Jul 29 15:11:06.744592 ip-10-0-135-116 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 15:11:07.100707 ip-10-0-135-116 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 15:11:07.100707 ip-10-0-135-116 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 15:11:07.100707 ip-10-0-135-116 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 15:11:07.100707 ip-10-0-135-116 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 15:11:07.100707 ip-10-0-135-116 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 15:11:07.101589 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.101484    2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 15:11:07.105836 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105820    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:11:07.105836 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105835    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105840    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105843    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105846    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105849    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105852    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105855    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105858    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105861    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105863    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105866    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105868    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105871    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105874    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105877    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105879    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105882    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105885    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105889    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:11:07.105903 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105892    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105895    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105898    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105900    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105903    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105906    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105909    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105912    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105916    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105920    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105923    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105927    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105930    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105933    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105935    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105938    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105941    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105944    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105947    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:11:07.106373 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105949    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105952    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105955    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105957    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105960    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105963    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105965    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105968    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105970    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105972    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105975    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105978    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105980    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105983    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105986    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105989    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105992    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105995    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.105997    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106000    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:11:07.106832 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106002    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106005    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106008    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106010    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106013    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106016    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106018    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106021    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106025    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106027    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106029    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106032    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106034    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106037    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106040    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106042    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106045    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106047    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106050    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106052    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:11:07.107371 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106054    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106057    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106060    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106062    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106065    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106068    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106071    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106074    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106077    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106080    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106082    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106085    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106087    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106090    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106092    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106095    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106097    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106100    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106103    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106105    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:11:07.107861 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106818    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106825    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106829    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106832    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106835    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106837    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106841    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106843    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106846    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106849    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106852    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106855    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106857    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106860    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106863    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106866    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106868    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106871    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106873    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106876    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:11:07.108359 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106879    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106882    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106886    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106889    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106892    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106895    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106898    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106900    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106904    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106906    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106909    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106912    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106914    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106917    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106919    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106922    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106924    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106927    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106929    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:11:07.108860 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106931    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106935    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106938    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106941    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106944    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106947    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106949    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106952    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106955    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106957    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106960    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106962    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106966    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106969    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106971    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106974    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106976    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106979    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106981    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:11:07.109334 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106983    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106986    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106988    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106991    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106993    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106996    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.106999    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107001    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107004    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107009    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107012    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107015    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107017    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107020    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107022    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107025    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107027    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107030    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107033    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:11:07.109855 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107035    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107038    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107040    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107043    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107045    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107048    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107052    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107055    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107057    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107060    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107062    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107065    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107068    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107070    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107073    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107075    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107078    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107080    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107083    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:11:07.110309 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107085    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107089    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107091    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107459    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107464    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107467    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107469    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107472    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107475    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107478    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107481    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107483    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107486    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107489    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107491    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107494    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107496    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107499    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107502    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107505    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:11:07.110779 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107508    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107510    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107513    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107516    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107518    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107521    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107523    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107526    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107529    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107531    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107533    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107536    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107538    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107546    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107549    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107552    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107554    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107558    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107561    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:11:07.111265 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107564    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107567    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107569    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107572    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107574    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107577    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107579    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107582    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107584    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107586    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107589    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107591    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107595    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107598    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107600    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107603    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107605    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107608    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107611    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107613    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:11:07.111772 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107616    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107619    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107621    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107623    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107626    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107628    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107631    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107635    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107638    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107641    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107643    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107646    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107649    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107652    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107654    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107657    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107660    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107663    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107665    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107667    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:11:07.112269 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107670    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107672    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107675    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107677    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107680    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107683    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107686    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107688    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107690    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107693    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107695    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107698    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107700    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107702    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107705    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107707    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107710    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107712    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107715    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107717    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:11:07.112769 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107720    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107724    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.107727    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107788    2575 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107799    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107805    2575 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107809    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107813    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107816    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107820    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107824    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107827    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107833    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107836    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107840    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107843    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107845    2575 flags.go:64] FLAG: --cgroup-root=""
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107848    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107851    2575 flags.go:64] FLAG: --client-ca-file=""
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107854    2575 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107857    2575 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107861    2575 flags.go:64] FLAG: --cluster-domain=""
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107864    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 15:11:07.113244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107867    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107871    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107874    2575 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107877    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107880    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107883    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107888    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107891    2575 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107894    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107897    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107901    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107904    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107908    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107911    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107913    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107916    2575 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107919    2575 flags.go:64] FLAG: --enable-server="true"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107922    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107926    2575 flags.go:64] FLAG: --event-burst="100"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107929    2575 flags.go:64] FLAG: --event-qps="50"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107932    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107936    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107938    2575 flags.go:64] FLAG: --eviction-hard=""
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107942    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 15:11:07.113839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107952    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107955    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107958    2575 flags.go:64] FLAG: --eviction-soft=""
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107961    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107964    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107967    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107970    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107972    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107975    2575 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107978    2575 flags.go:64] FLAG: --feature-gates=""
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107981    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107984    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107987    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107990    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107995    2575 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.107998    2575 flags.go:64] FLAG: --help="false"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108001    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108004    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108007    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108010    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108014    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108018    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108020    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 15:11:07.114512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108023    2575 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108026    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108028    2575 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108031    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108034    2575 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108037    2575 flags.go:64] FLAG: --kube-reserved=""
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108040    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108043    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108046    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108049    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108052    2575 flags.go:64] FLAG: --lock-file=""
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108055    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108058    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108060    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108066    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108069    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108071    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108074    2575 flags.go:64] FLAG: --logging-format="text"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108077    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108080    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108083    2575 flags.go:64] FLAG: --manifest-url=""
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108086    2575 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108090    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108096    2575 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108101    2575 flags.go:64] FLAG: --max-pods="110"
Jul 29 15:11:07.115313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108103    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108106    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108109    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108112    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108121    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108124    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108127    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108133    2575 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108136    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108139    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108142    2575 flags.go:64] FLAG: --pod-cidr=""
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108145    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108150    2575 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108153    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108156    2575 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108159    2575 flags.go:64] FLAG: --port="10250"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108162    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108165    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0248cf8cf0927e4f3"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108168    2575 flags.go:64] FLAG: --qos-reserved=""
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108171    2575 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108174    2575 flags.go:64] FLAG: --register-node="true"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108177    2575 flags.go:64] FLAG: --register-with-taints=""
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108180    2575 flags.go:64] FLAG: --registry-burst="10"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108183    2575 flags.go:64] FLAG: --registry-qps="5"
Jul 29 15:11:07.116124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108186    2575 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108189    2575 flags.go:64] FLAG: --reserved-memory=""
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108192    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108195    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108198    2575 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108201    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108205    2575 flags.go:64] FLAG: --runonce="false"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108208    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108211    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108214    2575 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108216    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108219    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108222    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108232    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108235    2575 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108238    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108241    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108244    2575 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108246    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108249    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108252    2575 flags.go:64] FLAG: --system-cgroups=""
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108255    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108260    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108263    2575 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108266    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 15:11:07.117089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108272    2575 flags.go:64] FLAG: --tls-min-version=""
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108275    2575 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108277    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108280    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108283    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108286    2575 flags.go:64] FLAG: --v="2"
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108289    2575 flags.go:64] FLAG: --version="false"
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108293    2575 flags.go:64] FLAG: --vmodule=""
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108296    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108299    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108441    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108446    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108449    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108454    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108457    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108459    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108462    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108465    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108467    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108470    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108472    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108476    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:11:07.117835 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108478    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108480    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108483    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108486    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108489    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108491    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108494    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108497    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108500    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108503    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108506    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108508    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108511    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108514    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108516    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108519    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108521    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108524    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108527    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108529    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:11:07.118414 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108531    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108534    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108537    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108540    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108543    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108545    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108548    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108551    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108553    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108556    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108558    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108561    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108564    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108566    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108569    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108572    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108574    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108577    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108579    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108583    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:11:07.118912 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108586    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108588    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108591    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108593    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108596    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108598    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108601    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108604    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108606    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108609    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108611    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108614    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108616    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108619    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108621    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108625    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108629    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108633    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108635    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:11:07.119402 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108638    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108641    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108644    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108646    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108649    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108654    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108658    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108660    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108664    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108668    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108672    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108675    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108679    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108682    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108684    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108687    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108690    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108692    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108695    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:11:07.119906 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108697    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:11:07.120447 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108700    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:11:07.120447 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108702    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:11:07.120447 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108704    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:11:07.120447 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108707    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:11:07.120447 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108710    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:11:07.120447 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108712    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:11:07.120447 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108715    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:11:07.120447 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.108717    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:11:07.120447 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.108724    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 15:11:07.120447 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.115676    2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 15:11:07.120447 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.115694    2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 15:11:07.120447 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.115710    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 15:11:07.120447 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.115716    2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 15:11:07.120447 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.116923    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:11:07.120447 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.116952    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117094    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117100    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117104    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117108    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117111    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117114    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117117    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117120    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117123    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117126    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117129    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117131    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117134    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117136    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117139    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117142    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117145    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117147    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:11:07.120822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117150    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117152    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117155    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117158    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117160    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117163    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117165    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117169    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117171    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117174    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117176    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117179    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117182    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117188    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117192    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117194    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117197    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117200    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117203    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117205    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:11:07.121274 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117208    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117211    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117213    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117216    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117219    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117221    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117224    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117227    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117230    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117232    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117237    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117241    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117244    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117247    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117250    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117253    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117256    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117259    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117262    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117264    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:11:07.121881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117267    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117269    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117272    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117275    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117277    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117280    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117284    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117287    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117290    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117293    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117295    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117298    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117301    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117303    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117306    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117310    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117314    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117330    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117335    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:11:07.122393 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117338    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117342    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117345    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117349    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117369    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117373    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117375    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117378    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117380    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117383    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117385    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117388    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117391    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117393    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117396    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117399    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117401    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117404    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117407    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:11:07.123099 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117411    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:11:07.123881 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.117417    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 15:11:07.123881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117556    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 15:11:07.123881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117561    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 15:11:07.123881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117564    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 15:11:07.123881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117567    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 15:11:07.123881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117570    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 15:11:07.123881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117573    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 15:11:07.123881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117576    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 15:11:07.123881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117578    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 15:11:07.123881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117581    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 15:11:07.123881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117584    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 15:11:07.123881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117586    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 15:11:07.123881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117589    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 15:11:07.123881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117592    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 15:11:07.123881 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117595    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117598    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117600    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117609    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117611    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117615    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117618    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117621    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117624    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117626    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117630    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117632    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117635    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117637    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117640    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117642    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117645    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117647    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117650    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 15:11:07.124260 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117653    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117655    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117657    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117660    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117662    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117665    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117667    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117670    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117672    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117674    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117677    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117680    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117682    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117684    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117687    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117689    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117692    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117701    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117704    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117706    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 15:11:07.125035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117709    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117711    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117714    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117716    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117719    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117721    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117724    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117726    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117729    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117731    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117734    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117737    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117739    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117741    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117744    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117748    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117751    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117754    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117757    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117759    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 15:11:07.125600 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117762    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117764    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117768    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117772    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117774    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117777    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117780    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117783    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117785    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117788    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117796    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117799    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117802    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117805    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117807    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117810    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117813    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117815    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117817    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 15:11:07.126085 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117820    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 15:11:07.126661 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117822    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 15:11:07.126661 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117825    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 15:11:07.126661 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117828    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 15:11:07.126661 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117830    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 15:11:07.126661 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117833    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 15:11:07.126661 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117836    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 15:11:07.126661 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:07.117838    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 15:11:07.126661 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.117843    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 15:11:07.126661 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.117938    2575 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 15:11:07.126661 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.120158    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 15:11:07.126661 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.121506    2575 server.go:1019] "Starting client certificate rotation"
Jul 29 15:11:07.126661 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.121620    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 15:11:07.126661 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.121664    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 15:11:07.127030 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.125237    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 15:11:07.127923 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.127906    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 15:11:07.140409 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.140392    2575 log.go:25] "Validated CRI v1 runtime API"
Jul 29 15:11:07.142492 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.142478    2575 log.go:25] "Validated CRI v1 image API"
Jul 29 15:11:07.143932 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.143914    2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 15:11:07.148583 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.148555    2575 fs.go:135] Filesystem UUIDs: map[45ecd465-de0a-459e-8ae1-32d59acb9f13:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 89adee62-4534-44d3-90e3-84d9e4feb828:/dev/nvme0n1p4]
Jul 29 15:11:07.148678 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.148583    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 15:11:07.153238 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.153215    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 15:11:07.153982 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.153851    2575 manager.go:216] Machine: {Timestamp:2026-07-29 15:11:07.153229372 +0000 UTC m=+0.302523268 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3079286 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b906d7f7f9fb27fe8ab941816214d SystemUUID:ec2b906d-7f7f-9fb2-7fe8-ab941816214d BootID:bce915f8-827c-43e0-a851-2bdf11f18ad8 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ca:d2:06:dd:c1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ca:d2:06:dd:c1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:49:6d:4a:9b:c5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 15:11:07.153982 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.153975    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 15:11:07.154157 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.154080    2575 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 15:11:07.156534 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.156509    2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 15:11:07.156693 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.156544    2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-116.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 15:11:07.157371 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.157361    2575 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 15:11:07.157422 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.157374    2575 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 15:11:07.157422 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.157382    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 15:11:07.158095 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.158083    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 15:11:07.159217 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.159206    2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 15:11:07.161998 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.161986    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 15:11:07.162124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.162113    2575 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 15:11:07.162169 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.162144    2575 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 15:11:07.162169 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.162154    2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 15:11:07.162169 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.162168    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 15:11:07.162287 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.162178    2575 kubelet.go:398] "Adding apiserver pod source"
Jul 29 15:11:07.162287 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.162186    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 15:11:07.162675 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.162656    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 15:11:07.162742 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.162694    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 15:11:07.166191 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.166168    2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 15:11:07.168507 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.168490    2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 15:11:07.168590 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.168531    2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 15:11:07.170565 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.170551    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 15:11:07.170637 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.170573    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 15:11:07.170637 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.170582    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 15:11:07.170637 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.170591    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 15:11:07.170637 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.170604    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 15:11:07.170637 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.170613    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 15:11:07.170637 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.170626    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 15:11:07.170637 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.170635    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 15:11:07.170877 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.170645    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 15:11:07.170877 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.170654    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 15:11:07.170877 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.170667    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 15:11:07.170877 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.170676    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 15:11:07.171519 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.171507    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 15:11:07.171570 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.171521    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 15:11:07.174795 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.174779    2575 server.go:1268] "Started kubelet"
Jul 29 15:11:07.174916 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.174868    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 15:11:07.174986 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.174939    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 15:11:07.175586 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.175552    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 15:11:07.175613 ip-10-0-135-116 systemd[1]: Started Kubernetes Kubelet.
Jul 29 15:11:07.176872 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.176855    2575 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 15:11:07.177157 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.177140    2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 15:11:07.177315 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.177296    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-116.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 15:11:07.177463 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.177442    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-116.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 15:11:07.177544 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.177507    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 15:11:07.180502 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.180478    2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 15:11:07.184409 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.183451    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-116.ec2.internal.18c6cb54c1012d2b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-116.ec2.internal,UID:ip-10-0-135-116.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-116.ec2.internal,},FirstTimestamp:2026-07-29 15:11:07.174751531 +0000 UTC m=+0.324045425,LastTimestamp:2026-07-29 15:11:07.174751531 +0000 UTC m=+0.324045425,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-116.ec2.internal,}"
Jul 29 15:11:07.186385 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.186368    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 15:11:07.186463 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.186388    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 15:11:07.186533 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.186479    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-116.ec2.internal\" not found"
Jul 29 15:11:07.187292 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.187162    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 15:11:07.187292 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.187197    2575 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 15:11:07.187292 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.187208    2575 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 15:11:07.187506 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.187339    2575 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 15:11:07.187506 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.187349    2575 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 15:11:07.187603 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.187563    2575 factory.go:55] Registering systemd factory
Jul 29 15:11:07.187603 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.187578    2575 factory.go:223] Registration of the systemd container factory successfully
Jul 29 15:11:07.187842 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.187812    2575 factory.go:153] Registering CRI-O factory
Jul 29 15:11:07.187842 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.187834    2575 factory.go:223] Registration of the crio container factory successfully
Jul 29 15:11:07.187981 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.187876    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6m94p"
Jul 29 15:11:07.187981 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.187913    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 15:11:07.187981 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.187939    2575 factory.go:103] Registering Raw factory
Jul 29 15:11:07.187981 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.187957    2575 manager.go:1196] Started watching for new ooms in manager
Jul 29 15:11:07.188568 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.188546    2575 manager.go:349] Starting recovery of all containers
Jul 29 15:11:07.188654 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.188617    2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-116.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 29 15:11:07.188823 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.188801    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 29 15:11:07.197039 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.197012    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6m94p"
Jul 29 15:11:07.201144 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.201125    2575 manager.go:354] Recovery completed
Jul 29 15:11:07.206817 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.206803    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:11:07.208964 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.208945    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:11:07.209026 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.208979    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:11:07.209026 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.208990    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:11:07.209576 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.209561    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 15:11:07.209637 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.209579    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 15:11:07.209637 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.209596    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 15:11:07.209637 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.209631    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:11:07.210357 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.210340    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:11:07.210418 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.210385    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:11:07.210418 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.210398    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:11:07.212386 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.212374    2575 policy_none.go:49] "None policy: Start"
Jul 29 15:11:07.212449 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.212390    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 15:11:07.212449 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.212415    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 15:11:07.214495 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.214481    2575 policy_none.go:47] "Start"
Jul 29 15:11:07.250039 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.249919    2575 manager.go:337] "Starting Device Plugin manager"
Jul 29 15:11:07.250123 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.250054    2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 15:11:07.250123 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.250070    2575 server.go:87] "Starting device plugin registration server"
Jul 29 15:11:07.250321 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.250307    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 15:11:07.250418 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.250322    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 15:11:07.250471 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.250450    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 15:11:07.250538 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.250524    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 15:11:07.250538 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.250539    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 15:11:07.251062 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.251039    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 15:11:07.251142 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.251093    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-116.ec2.internal\" not found"
Jul 29 15:11:07.287229 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.287197    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 15:11:07.295844 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.288494    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 15:11:07.295844 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.288532    2575 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 15:11:07.295844 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.288555    2575 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 15:11:07.295844 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.288589    2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 15:11:07.295844 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.291194    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:11:07.354138 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.354083    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:11:07.355011 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.354995    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:11:07.355091 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.355024    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:11:07.355091 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.355036    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:11:07.355091 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.355060    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.364775 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.364756    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.364821 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.364781    2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-116.ec2.internal\": node \"ip-10-0-135-116.ec2.internal\" not found"
Jul 29 15:11:07.382796 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.382770    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-116.ec2.internal\" not found"
Jul 29 15:11:07.388916 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.388882    2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-116.ec2.internal"]
Jul 29 15:11:07.388993 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.388957    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:11:07.391241 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.391226    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:11:07.391319 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.391255    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:11:07.391319 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.391270    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:11:07.393690 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.393676    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:11:07.393803 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.393776    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.393847 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.393815    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:11:07.394491 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.394468    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:11:07.394570 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.394499    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:11:07.394570 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.394510    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:11:07.394570 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.394476    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:11:07.394682 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.394573    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:11:07.394682 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.394588    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:11:07.396957 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.396940    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.397005 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.396976    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 15:11:07.397731 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.397714    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 15:11:07.397794 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.397750    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 15:11:07.397794 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.397767    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeHasSufficientPID"
Jul 29 15:11:07.416971 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.416949    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-116.ec2.internal\" not found" node="ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.421282 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.421264    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-116.ec2.internal\" not found" node="ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.484222 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.484195    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-116.ec2.internal\" not found"
Jul 29 15:11:07.487812 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.487788    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fc390c647fd85218bf5e68bb219f108a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal\" (UID: \"fc390c647fd85218bf5e68bb219f108a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.487918 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.487816    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc390c647fd85218bf5e68bb219f108a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal\" (UID: \"fc390c647fd85218bf5e68bb219f108a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.487918 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.487836    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5da59de62b315dbd806daa423c5548a3-config\") pod \"kube-apiserver-proxy-ip-10-0-135-116.ec2.internal\" (UID: \"5da59de62b315dbd806daa423c5548a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.584488 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.584453    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-116.ec2.internal\" not found"
Jul 29 15:11:07.588108 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.588086    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fc390c647fd85218bf5e68bb219f108a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal\" (UID: \"fc390c647fd85218bf5e68bb219f108a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.588174 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.588118    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc390c647fd85218bf5e68bb219f108a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal\" (UID: \"fc390c647fd85218bf5e68bb219f108a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.588174 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.588139    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5da59de62b315dbd806daa423c5548a3-config\") pod \"kube-apiserver-proxy-ip-10-0-135-116.ec2.internal\" (UID: \"5da59de62b315dbd806daa423c5548a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.588248 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.588188    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5da59de62b315dbd806daa423c5548a3-config\") pod \"kube-apiserver-proxy-ip-10-0-135-116.ec2.internal\" (UID: \"5da59de62b315dbd806daa423c5548a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.588248 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.588189    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fc390c647fd85218bf5e68bb219f108a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal\" (UID: \"fc390c647fd85218bf5e68bb219f108a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.588248 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.588211    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc390c647fd85218bf5e68bb219f108a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal\" (UID: \"fc390c647fd85218bf5e68bb219f108a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.685890 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.685813    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-116.ec2.internal\" not found"
Jul 29 15:11:07.721573 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.721549    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.725196 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:07.725175    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-116.ec2.internal"
Jul 29 15:11:07.787771 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.787746    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-116.ec2.internal\" not found"
Jul 29 15:11:07.891756 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.891722    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-116.ec2.internal\" not found"
Jul 29 15:11:07.992853 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:07.992788    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-116.ec2.internal\" not found"
Jul 29 15:11:08.092864 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:08.092840    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-116.ec2.internal\" not found"
Jul 29 15:11:08.095400 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:08.095368    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:11:08.121556 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:08.121519    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 15:11:08.122009 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:08.121682    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 15:11:08.122009 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:08.121725    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 15:11:08.193136 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:08.193108    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-116.ec2.internal\" not found"
Jul 29 15:11:08.203446 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:08.203410    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 15:06:07 +0000 UTC" deadline="2028-02-26 04:51:30.03036139 +0000 UTC"
Jul 29 15:11:08.203446 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:08.203440    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13837h40m21.826924625s"
Jul 29 15:11:08.293657 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:08.293592    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-116.ec2.internal\" not found"
Jul 29 15:11:08.394162 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:08.394124    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-116.ec2.internal\" not found"
Jul 29 15:11:08.415609 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:08.415588    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:11:08.488422 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:08.488385    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal"
Jul 29 15:11:08.503571 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:08.503552    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 15:11:08.505381 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:08.505364    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-116.ec2.internal"
Jul 29 15:11:08.517263 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:08.517243    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 15:11:08.527482 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:08.527454    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5da59de62b315dbd806daa423c5548a3.slice/crio-2068e064b092ac78812b827f2c338344dbd8cf4d27ebf3a8eb8064add8407f9c WatchSource:0}: Error finding container 2068e064b092ac78812b827f2c338344dbd8cf4d27ebf3a8eb8064add8407f9c: Status 404 returned error can't find the container with id 2068e064b092ac78812b827f2c338344dbd8cf4d27ebf3a8eb8064add8407f9c
Jul 29 15:11:08.534158 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:08.534137    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 15:11:08.539183 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:08.539158    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc390c647fd85218bf5e68bb219f108a.slice/crio-eafbd93a2e631b8e3dd598e7b13b2b0639b2f054ef8f9047259af81e76e995fc WatchSource:0}: Error finding container eafbd93a2e631b8e3dd598e7b13b2b0639b2f054ef8f9047259af81e76e995fc: Status 404 returned error can't find the container with id eafbd93a2e631b8e3dd598e7b13b2b0639b2f054ef8f9047259af81e76e995fc
Jul 29 15:11:08.647970 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:08.647887    2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:11:09.166129 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.166087    2575 apiserver.go:52] "Watching apiserver"
Jul 29 15:11:09.172082 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.172057    2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 15:11:09.174534 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.174504    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-8xf5b","openshift-image-registry/node-ca-m6bcg","openshift-multus/multus-vx7cn","openshift-network-diagnostics/network-check-target-bs7zd","openshift-ovn-kubernetes/ovnkube-node-ksmm5","kube-system/global-pull-secret-syncer-rnrfc","kube-system/konnectivity-agent-zwf85","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b","openshift-cluster-node-tuning-operator/tuned-q6prv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal","openshift-multus/multus-additional-cni-plugins-n9mpt","openshift-multus/network-metrics-daemon-psdvf","openshift-network-operator/iptables-alerter-fkv5t","kube-system/kube-apiserver-proxy-ip-10-0-135-116.ec2.internal"]
Jul 29 15:11:09.177230 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.177211    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zwf85"
Jul 29 15:11:09.180059 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.179637    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fkv5t"
Jul 29 15:11:09.180059 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.180019    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 15:11:09.180437 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.180412    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8vgg9\""
Jul 29 15:11:09.181186 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.181160    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 15:11:09.181723 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.181686    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 15:11:09.181982 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.181953    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 15:11:09.181982 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.181975    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-psjgp\""
Jul 29 15:11:09.182281 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.182262    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 15:11:09.182547 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.182515    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.184862 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.184421    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 15:11:09.184862 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.184464    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vt2bp\""
Jul 29 15:11:09.184862 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.184734    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 15:11:09.184862 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.184819    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 15:11:09.185165 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.184983    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:09.185165 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.185044    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs7zd" podUID="3294f075-e06b-44fa-b813-85ccc30860c1"
Jul 29 15:11:09.185165 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.185058    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 15:11:09.187413 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.187378    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 15:11:09.188210 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.188187    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.190174 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.190146    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 15:11:09.190667 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.190648    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 15:11:09.190741 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.190679    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nb92h\""
Jul 29 15:11:09.190839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.190823    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 15:11:09.190884 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.190863    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 15:11:09.191204 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.191020    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:09.191204 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.191055    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 15:11:09.191204 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.191068    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 15:11:09.191204 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.191099    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rnrfc" podUID="a2f60801-5312-4da2-a438-6103942ccbd7"
Jul 29 15:11:09.193719 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.193691    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.195890 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.195869    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 15:11:09.196019 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.195998    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8xf5b"
Jul 29 15:11:09.196128 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.196027    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 15:11:09.196128 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.196045    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-55v2m\""
Jul 29 15:11:09.198283 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.198262    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kg7nz\""
Jul 29 15:11:09.198379 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.198292    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-run-ovn\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.198379 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.198322    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-log-socket\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.198379 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.198348    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-cni-bin\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.198379 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.198374    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.198615 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.198482    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qmrv\" (UniqueName: \"kubernetes.io/projected/3aee2e64-bc45-401a-a783-23c2feda642e-kube-api-access-5qmrv\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.198615 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.198509    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-systemd-units\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.198615 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.198520    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 15:11:09.198615 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.198534    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-os-release\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.198615 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.198561    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b36df5c1-83d8-4ac9-8be0-2c7aa7086502-agent-certs\") pod \"konnectivity-agent-zwf85\" (UID: \"b36df5c1-83d8-4ac9-8be0-2c7aa7086502\") " pod="kube-system/konnectivity-agent-zwf85"
Jul 29 15:11:09.198615 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.198610    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5e7304d8-98a8-4f52-9798-5690011cf2b1-host-slash\") pod \"iptables-alerter-fkv5t\" (UID: \"5e7304d8-98a8-4f52-9798-5690011cf2b1\") " pod="openshift-network-operator/iptables-alerter-fkv5t"
Jul 29 15:11:09.198905 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.198634    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-var-lib-cni-bin\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.198905 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.198658    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-var-lib-kubelet\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.198905 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.198688    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-cnibin\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.198905 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.198711    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-run-netns\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.199265 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.198932    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 15:11:09.199265 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.198735    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-run-multus-certs\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.199265 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199027    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9bhs\" (UniqueName: \"kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs\") pod \"network-check-target-bs7zd\" (UID: \"3294f075-e06b-44fa-b813-85ccc30860c1\") " pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:09.199265 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199054    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-run-openvswitch\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.199265 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199079    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a4d1a578-61d7-4a7d-b4c1-7389cb969155-env-overrides\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.199265 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199113    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a4d1a578-61d7-4a7d-b4c1-7389cb969155-ovnkube-script-lib\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.199265 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199139    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djn56\" (UniqueName: \"kubernetes.io/projected/a4d1a578-61d7-4a7d-b4c1-7389cb969155-kube-api-access-djn56\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.199265 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199169    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c5hs\" (UniqueName: \"kubernetes.io/projected/5e7304d8-98a8-4f52-9798-5690011cf2b1-kube-api-access-5c5hs\") pod \"iptables-alerter-fkv5t\" (UID: \"5e7304d8-98a8-4f52-9798-5690011cf2b1\") " pod="openshift-network-operator/iptables-alerter-fkv5t"
Jul 29 15:11:09.199265 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199209    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-var-lib-cni-multus\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.199265 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199254    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-etc-openvswitch\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.199804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199290    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-node-log\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.199804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199323    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a4d1a578-61d7-4a7d-b4c1-7389cb969155-ovn-node-metrics-cert\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.199804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199358    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-kubelet\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.199804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199370    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m6bcg"
Jul 29 15:11:09.199804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199392    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-slash\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.199804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199478    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-var-lib-openvswitch\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.199804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199507    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-cni-netd\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.199804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199543    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a4d1a578-61d7-4a7d-b4c1-7389cb969155-ovnkube-config\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.199804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199593    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5e7304d8-98a8-4f52-9798-5690011cf2b1-iptables-alerter-script\") pod \"iptables-alerter-fkv5t\" (UID: \"5e7304d8-98a8-4f52-9798-5690011cf2b1\") " pod="openshift-network-operator/iptables-alerter-fkv5t"
Jul 29 15:11:09.199804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199616    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-system-cni-dir\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.199804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199640    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-hostroot\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.199804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199673    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3aee2e64-bc45-401a-a783-23c2feda642e-multus-daemon-config\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.199804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199699    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-run-netns\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.199804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199777    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-multus-socket-dir-parent\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.199804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199804    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-etc-kubernetes\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.200485 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199827    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-multus-cni-dir\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.200485 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199852    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b36df5c1-83d8-4ac9-8be0-2c7aa7086502-konnectivity-ca\") pod \"konnectivity-agent-zwf85\" (UID: \"b36df5c1-83d8-4ac9-8be0-2c7aa7086502\") " pod="kube-system/konnectivity-agent-zwf85"
Jul 29 15:11:09.200485 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199877    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-run-ovn-kubernetes\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.200485 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199948    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3aee2e64-bc45-401a-a783-23c2feda642e-cni-binary-copy\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.200485 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.199978    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-run-k8s-cni-cncf-io\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.200485 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.200005    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-multus-conf-dir\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.200485 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.200030    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-run-systemd\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.200816 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.200654    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 15:11:09.201421 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.201369    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.201953 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.201938    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 15:11:09.202310 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.202292    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b5282\""
Jul 29 15:11:09.202417 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.202300    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 15:11:09.203094 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.203077    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 15:11:09.204099 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.204039    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.205762 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.205740    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 15:11:09.205902 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.205883    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q5vnt\""
Jul 29 15:11:09.206559 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.206539    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:09.206667 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.206609    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:09.206915 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.206743    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 15:11:09.206915 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.206796    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 15:11:09.207745 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.207725    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-n9hld\""
Jul 29 15:11:09.208353 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.208319    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 15:11:09.208823 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.208802    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 15:11:09.225776 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.225753    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-k2qjn"
Jul 29 15:11:09.236715 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.236693    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-k2qjn"
Jul 29 15:11:09.288923 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.288902    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 15:11:09.295316 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.295268    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal" event={"ID":"fc390c647fd85218bf5e68bb219f108a","Type":"ContainerStarted","Data":"eafbd93a2e631b8e3dd598e7b13b2b0639b2f054ef8f9047259af81e76e995fc"}
Jul 29 15:11:09.296875 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.296849    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-116.ec2.internal" event={"ID":"5da59de62b315dbd806daa423c5548a3","Type":"ContainerStarted","Data":"2068e064b092ac78812b827f2c338344dbd8cf4d27ebf3a8eb8064add8407f9c"}
Jul 29 15:11:09.300288 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300255    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3aee2e64-bc45-401a-a783-23c2feda642e-cni-binary-copy\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.300408 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300305    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-run-systemd\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.300408 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300340    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-run-ovn\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.300544 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300405    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-log-socket\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.300544 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300433    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-cni-bin\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.300544 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300468    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.300544 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300511    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t526g\" (UniqueName: \"kubernetes.io/projected/68f9c6d7-78ed-4086-8e7f-9244b109c1d7-kube-api-access-t526g\") pod \"node-resolver-8xf5b\" (UID: \"68f9c6d7-78ed-4086-8e7f-9244b109c1d7\") " pod="openshift-dns/node-resolver-8xf5b"
Jul 29 15:11:09.300740 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300548    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a2f60801-5312-4da2-a438-6103942ccbd7-kubelet-config\") pod \"global-pull-secret-syncer-rnrfc\" (UID: \"a2f60801-5312-4da2-a438-6103942ccbd7\") " pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:09.300740 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300583    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-systemd-units\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.300740 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300610    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-os-release\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.300740 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300645    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5e7304d8-98a8-4f52-9798-5690011cf2b1-host-slash\") pod \"iptables-alerter-fkv5t\" (UID: \"5e7304d8-98a8-4f52-9798-5690011cf2b1\") " pod="openshift-network-operator/iptables-alerter-fkv5t"
Jul 29 15:11:09.300740 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300679    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-var-lib-cni-bin\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.300740 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300713    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-host\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.301044 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300742    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m5fd\" (UniqueName: \"kubernetes.io/projected/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-kube-api-access-5m5fd\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.301044 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300778    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94d73f72-b4ea-454b-8162-b4d444385eea-cnibin\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.301044 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300818    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52c1f3d7-1c12-497f-82ed-3d5439d7dfbc-serviceca\") pod \"node-ca-m6bcg\" (UID: \"52c1f3d7-1c12-497f-82ed-3d5439d7dfbc\") " pod="openshift-image-registry/node-ca-m6bcg"
Jul 29 15:11:09.301044 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300854    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-cnibin\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.301044 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300889    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-run-netns\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.301044 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300916    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-run-multus-certs\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.301044 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300951    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9bhs\" (UniqueName: \"kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs\") pod \"network-check-target-bs7zd\" (UID: \"3294f075-e06b-44fa-b813-85ccc30860c1\") " pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:09.301044 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.300990    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-run-openvswitch\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.301044 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301025    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a4d1a578-61d7-4a7d-b4c1-7389cb969155-env-overrides\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.301552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301059    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-var-lib-cni-multus\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.301552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301088    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-etc-openvswitch\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.301552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301123    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret\") pod \"global-pull-secret-syncer-rnrfc\" (UID: \"a2f60801-5312-4da2-a438-6103942ccbd7\") " pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:09.301552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301161    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp7xt\" (UniqueName: \"kubernetes.io/projected/94d73f72-b4ea-454b-8162-b4d444385eea-kube-api-access-qp7xt\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.301552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301201    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-systemd\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.301552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301233    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-sys-fs\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.301552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301268    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-kubelet\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.301552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301303    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-slash\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.301552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301336    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-var-lib-openvswitch\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.301552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301370    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a4d1a578-61d7-4a7d-b4c1-7389cb969155-ovnkube-config\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.301552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301412    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-lib-modules\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.301552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301449    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-tuned\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.301552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301486    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-registration-dir\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.301552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301521    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94d73f72-b4ea-454b-8162-b4d444385eea-system-cni-dir\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.301552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301554    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-kubernetes\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.302281 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301589    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-sys\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.302281 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301622    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-tmp\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.302281 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301650    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.302281 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301689    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94d73f72-b4ea-454b-8162-b4d444385eea-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.302281 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301724    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/94d73f72-b4ea-454b-8162-b4d444385eea-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.302281 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301758    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-modprobe-d\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.302281 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301786    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-run\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.302281 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301819    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs\") pod \"network-metrics-daemon-psdvf\" (UID: \"925b0a18-77be-438d-bcfe-27609c648913\") " pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:09.302281 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301855    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b36df5c1-83d8-4ac9-8be0-2c7aa7086502-konnectivity-ca\") pod \"konnectivity-agent-zwf85\" (UID: \"b36df5c1-83d8-4ac9-8be0-2c7aa7086502\") " pod="kube-system/konnectivity-agent-zwf85"
Jul 29 15:11:09.302281 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301891    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-run-ovn-kubernetes\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.302281 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301928    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/94d73f72-b4ea-454b-8162-b4d444385eea-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.302281 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301962    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-socket-dir\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.302281 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.301996    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-device-dir\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.302281 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302032    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-run-k8s-cni-cncf-io\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.302281 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302066    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-multus-conf-dir\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.302281 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302102    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk78n\" (UniqueName: \"kubernetes.io/projected/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-kube-api-access-fk78n\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.303106 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302133    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qmrv\" (UniqueName: \"kubernetes.io/projected/3aee2e64-bc45-401a-a783-23c2feda642e-kube-api-access-5qmrv\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.303106 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302168    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b36df5c1-83d8-4ac9-8be0-2c7aa7086502-agent-certs\") pod \"konnectivity-agent-zwf85\" (UID: \"b36df5c1-83d8-4ac9-8be0-2c7aa7086502\") " pod="kube-system/konnectivity-agent-zwf85"
Jul 29 15:11:09.303106 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302220    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-var-lib-kubelet\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.303106 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302258    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djn56\" (UniqueName: \"kubernetes.io/projected/a4d1a578-61d7-4a7d-b4c1-7389cb969155-kube-api-access-djn56\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.303106 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302302    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-sysctl-d\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.303106 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302337    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68f9c6d7-78ed-4086-8e7f-9244b109c1d7-tmp-dir\") pod \"node-resolver-8xf5b\" (UID: \"68f9c6d7-78ed-4086-8e7f-9244b109c1d7\") " pod="openshift-dns/node-resolver-8xf5b"
Jul 29 15:11:09.303106 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302374    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a4d1a578-61d7-4a7d-b4c1-7389cb969155-ovnkube-script-lib\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.303106 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302423    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.303106 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302460    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnw4k\" (UniqueName: \"kubernetes.io/projected/925b0a18-77be-438d-bcfe-27609c648913-kube-api-access-bnw4k\") pod \"network-metrics-daemon-psdvf\" (UID: \"925b0a18-77be-438d-bcfe-27609c648913\") " pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:09.303106 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302491    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/94d73f72-b4ea-454b-8162-b4d444385eea-cni-binary-copy\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.303106 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302529    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c5hs\" (UniqueName: \"kubernetes.io/projected/5e7304d8-98a8-4f52-9798-5690011cf2b1-kube-api-access-5c5hs\") pod \"iptables-alerter-fkv5t\" (UID: \"5e7304d8-98a8-4f52-9798-5690011cf2b1\") " pod="openshift-network-operator/iptables-alerter-fkv5t"
Jul 29 15:11:09.303106 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302567    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-node-log\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.303106 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302603    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a4d1a578-61d7-4a7d-b4c1-7389cb969155-ovn-node-metrics-cert\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.303106 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302636    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52c1f3d7-1c12-497f-82ed-3d5439d7dfbc-host\") pod \"node-ca-m6bcg\" (UID: \"52c1f3d7-1c12-497f-82ed-3d5439d7dfbc\") " pod="openshift-image-registry/node-ca-m6bcg"
Jul 29 15:11:09.303106 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302664    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-sysconfig\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.303106 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302708    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/68f9c6d7-78ed-4086-8e7f-9244b109c1d7-hosts-file\") pod \"node-resolver-8xf5b\" (UID: \"68f9c6d7-78ed-4086-8e7f-9244b109c1d7\") " pod="openshift-dns/node-resolver-8xf5b"
Jul 29 15:11:09.303106 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302743    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-cni-netd\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.303917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302778    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/94d73f72-b4ea-454b-8162-b4d444385eea-os-release\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.303917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302811    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5e7304d8-98a8-4f52-9798-5690011cf2b1-iptables-alerter-script\") pod \"iptables-alerter-fkv5t\" (UID: \"5e7304d8-98a8-4f52-9798-5690011cf2b1\") " pod="openshift-network-operator/iptables-alerter-fkv5t"
Jul 29 15:11:09.303917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302847    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-system-cni-dir\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.303917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302885    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-hostroot\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.303917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302922    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3aee2e64-bc45-401a-a783-23c2feda642e-multus-daemon-config\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.303917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302953    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-run-netns\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.303917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.302986    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a2f60801-5312-4da2-a438-6103942ccbd7-dbus\") pod \"global-pull-secret-syncer-rnrfc\" (UID: \"a2f60801-5312-4da2-a438-6103942ccbd7\") " pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:09.303917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.303021    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-sysctl-conf\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.303917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.303059    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-multus-socket-dir-parent\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.303917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.303099    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-etc-kubernetes\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.303917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.303133    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-var-lib-kubelet\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.303917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.303170    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-multus-cni-dir\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.303917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.303206    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcp8h\" (UniqueName: \"kubernetes.io/projected/52c1f3d7-1c12-497f-82ed-3d5439d7dfbc-kube-api-access-pcp8h\") pod \"node-ca-m6bcg\" (UID: \"52c1f3d7-1c12-497f-82ed-3d5439d7dfbc\") " pod="openshift-image-registry/node-ca-m6bcg"
Jul 29 15:11:09.303917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.303604    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-system-cni-dir\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.303917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.303608    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-run-ovn\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.303917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.303635    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-run-systemd\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.304662 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.303925    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-hostroot\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.304662 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.304200    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3aee2e64-bc45-401a-a783-23c2feda642e-cni-binary-copy\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.304662 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.304562    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-cni-bin\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.304801 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.304662    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-multus-socket-dir-parent\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.304801 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.304738    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-os-release\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.304801 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.304790    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-var-lib-cni-bin\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.304933 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.304841    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.304933 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.304887    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-run-netns\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.305233 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.305204    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a4d1a578-61d7-4a7d-b4c1-7389cb969155-ovnkube-config\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.305314 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.305282    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5e7304d8-98a8-4f52-9798-5690011cf2b1-host-slash\") pod \"iptables-alerter-fkv5t\" (UID: \"5e7304d8-98a8-4f52-9798-5690011cf2b1\") " pod="openshift-network-operator/iptables-alerter-fkv5t"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.305456    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b36df5c1-83d8-4ac9-8be0-2c7aa7086502-konnectivity-ca\") pod \"konnectivity-agent-zwf85\" (UID: \"b36df5c1-83d8-4ac9-8be0-2c7aa7086502\") " pod="kube-system/konnectivity-agent-zwf85"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.305467    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-log-socket\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.305593    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-var-lib-cni-multus\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.305614    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-kubelet\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.305754    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-multus-cni-dir\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.305867    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-etc-openvswitch\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.305924    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3aee2e64-bc45-401a-a783-23c2feda642e-multus-daemon-config\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.306131    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-cnibin\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.306153    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5e7304d8-98a8-4f52-9798-5690011cf2b1-iptables-alerter-script\") pod \"iptables-alerter-fkv5t\" (UID: \"5e7304d8-98a8-4f52-9798-5690011cf2b1\") " pod="openshift-network-operator/iptables-alerter-fkv5t"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.306161    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-run-netns\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.306186    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-run-multus-certs\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.306220    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-run-k8s-cni-cncf-io\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.306243    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-var-lib-openvswitch\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.306261    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a4d1a578-61d7-4a7d-b4c1-7389cb969155-ovnkube-script-lib\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.306290    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-run-ovn-kubernetes\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.306286    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.306320    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-slash\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.306423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.306343    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-node-log\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.307465 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.306376    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-multus-conf-dir\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.307465 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.306484    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-systemd-units\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.307465 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.306655    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-host-var-lib-kubelet\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.307465 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.306725    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-run-openvswitch\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.307465 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.306782    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a4d1a578-61d7-4a7d-b4c1-7389cb969155-host-cni-netd\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.307465 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.306966    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a4d1a578-61d7-4a7d-b4c1-7389cb969155-env-overrides\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.307465 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.307361    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3aee2e64-bc45-401a-a783-23c2feda642e-etc-kubernetes\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.311697 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.311672    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a4d1a578-61d7-4a7d-b4c1-7389cb969155-ovn-node-metrics-cert\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.315447 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.314463    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:11:09.315447 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.314486    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:11:09.315447 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.314501    2575 projected.go:196] Error preparing data for projected volume kube-api-access-j9bhs for pod openshift-network-diagnostics/network-check-target-bs7zd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:09.315447 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.314585    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs podName:3294f075-e06b-44fa-b813-85ccc30860c1 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:09.814556636 +0000 UTC m=+2.963850534 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-j9bhs" (UniqueName: "kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs") pod "network-check-target-bs7zd" (UID: "3294f075-e06b-44fa-b813-85ccc30860c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:09.316739 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.316715    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b36df5c1-83d8-4ac9-8be0-2c7aa7086502-agent-certs\") pod \"konnectivity-agent-zwf85\" (UID: \"b36df5c1-83d8-4ac9-8be0-2c7aa7086502\") " pod="kube-system/konnectivity-agent-zwf85"
Jul 29 15:11:09.317866 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.317842    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djn56\" (UniqueName: \"kubernetes.io/projected/a4d1a578-61d7-4a7d-b4c1-7389cb969155-kube-api-access-djn56\") pod \"ovnkube-node-ksmm5\" (UID: \"a4d1a578-61d7-4a7d-b4c1-7389cb969155\") " pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.317968 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.317895    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c5hs\" (UniqueName: \"kubernetes.io/projected/5e7304d8-98a8-4f52-9798-5690011cf2b1-kube-api-access-5c5hs\") pod \"iptables-alerter-fkv5t\" (UID: \"5e7304d8-98a8-4f52-9798-5690011cf2b1\") " pod="openshift-network-operator/iptables-alerter-fkv5t"
Jul 29 15:11:09.319374 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.319312    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qmrv\" (UniqueName: \"kubernetes.io/projected/3aee2e64-bc45-401a-a783-23c2feda642e-kube-api-access-5qmrv\") pod \"multus-vx7cn\" (UID: \"3aee2e64-bc45-401a-a783-23c2feda642e\") " pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.387525 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.387496    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:11:09.403622 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.403590    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcp8h\" (UniqueName: \"kubernetes.io/projected/52c1f3d7-1c12-497f-82ed-3d5439d7dfbc-kube-api-access-pcp8h\") pod \"node-ca-m6bcg\" (UID: \"52c1f3d7-1c12-497f-82ed-3d5439d7dfbc\") " pod="openshift-image-registry/node-ca-m6bcg"
Jul 29 15:11:09.403760 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.403639    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t526g\" (UniqueName: \"kubernetes.io/projected/68f9c6d7-78ed-4086-8e7f-9244b109c1d7-kube-api-access-t526g\") pod \"node-resolver-8xf5b\" (UID: \"68f9c6d7-78ed-4086-8e7f-9244b109c1d7\") " pod="openshift-dns/node-resolver-8xf5b"
Jul 29 15:11:09.403760 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.403669    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a2f60801-5312-4da2-a438-6103942ccbd7-kubelet-config\") pod \"global-pull-secret-syncer-rnrfc\" (UID: \"a2f60801-5312-4da2-a438-6103942ccbd7\") " pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:09.403760 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.403697    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-host\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.403760 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.403724    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m5fd\" (UniqueName: \"kubernetes.io/projected/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-kube-api-access-5m5fd\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.403760 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.403748    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94d73f72-b4ea-454b-8162-b4d444385eea-cnibin\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.404031 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.403784    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52c1f3d7-1c12-497f-82ed-3d5439d7dfbc-serviceca\") pod \"node-ca-m6bcg\" (UID: \"52c1f3d7-1c12-497f-82ed-3d5439d7dfbc\") " pod="openshift-image-registry/node-ca-m6bcg"
Jul 29 15:11:09.404031 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.403829    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret\") pod \"global-pull-secret-syncer-rnrfc\" (UID: \"a2f60801-5312-4da2-a438-6103942ccbd7\") " pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:09.404031 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.403855    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp7xt\" (UniqueName: \"kubernetes.io/projected/94d73f72-b4ea-454b-8162-b4d444385eea-kube-api-access-qp7xt\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.404031 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.403894    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-systemd\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.404031 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.403922    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-sys-fs\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.404031 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.403949    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-lib-modules\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.404031 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.403973    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-tuned\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.404031 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.403997    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-registration-dir\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.404031 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404021    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94d73f72-b4ea-454b-8162-b4d444385eea-system-cni-dir\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.404581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404037    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a2f60801-5312-4da2-a438-6103942ccbd7-kubelet-config\") pod \"global-pull-secret-syncer-rnrfc\" (UID: \"a2f60801-5312-4da2-a438-6103942ccbd7\") " pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:09.404581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404045    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-kubernetes\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.404581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404098    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-sys\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.404581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404139    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-tmp\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.404581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404162    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/94d73f72-b4ea-454b-8162-b4d444385eea-system-cni-dir\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.404581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404170    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.404581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404207    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94d73f72-b4ea-454b-8162-b4d444385eea-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.404581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404235    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/94d73f72-b4ea-454b-8162-b4d444385eea-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.404581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404262    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-modprobe-d\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.404581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404279    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.404581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404285    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-run\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.404581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404312    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs\") pod \"network-metrics-daemon-psdvf\" (UID: \"925b0a18-77be-438d-bcfe-27609c648913\") " pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:09.404581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404338    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-host\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.404581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404099    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-systemd\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.404581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404521    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/94d73f72-b4ea-454b-8162-b4d444385eea-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.404581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404572    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-sys\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.405364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404341    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/94d73f72-b4ea-454b-8162-b4d444385eea-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.405364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404620    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/94d73f72-b4ea-454b-8162-b4d444385eea-cnibin\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.405364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404668    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-socket-dir\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.405364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404103    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-kubernetes\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.405364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404700    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-device-dir\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.405364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404733    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk78n\" (UniqueName: \"kubernetes.io/projected/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-kube-api-access-fk78n\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.405364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404766    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-sysctl-d\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.405364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404798    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68f9c6d7-78ed-4086-8e7f-9244b109c1d7-tmp-dir\") pod \"node-resolver-8xf5b\" (UID: \"68f9c6d7-78ed-4086-8e7f-9244b109c1d7\") " pod="openshift-dns/node-resolver-8xf5b"
Jul 29 15:11:09.405364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404801    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-modprobe-d\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.405364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404827    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.405364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404856    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnw4k\" (UniqueName: \"kubernetes.io/projected/925b0a18-77be-438d-bcfe-27609c648913-kube-api-access-bnw4k\") pod \"network-metrics-daemon-psdvf\" (UID: \"925b0a18-77be-438d-bcfe-27609c648913\") " pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:09.405364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404884    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/94d73f72-b4ea-454b-8162-b4d444385eea-cni-binary-copy\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.405364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404903    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-lib-modules\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.405364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404916    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52c1f3d7-1c12-497f-82ed-3d5439d7dfbc-host\") pod \"node-ca-m6bcg\" (UID: \"52c1f3d7-1c12-497f-82ed-3d5439d7dfbc\") " pod="openshift-image-registry/node-ca-m6bcg"
Jul 29 15:11:09.405364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404945    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-sysconfig\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.405364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404974    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/68f9c6d7-78ed-4086-8e7f-9244b109c1d7-hosts-file\") pod \"node-resolver-8xf5b\" (UID: \"68f9c6d7-78ed-4086-8e7f-9244b109c1d7\") " pod="openshift-dns/node-resolver-8xf5b"
Jul 29 15:11:09.405364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405001    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/94d73f72-b4ea-454b-8162-b4d444385eea-os-release\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405030    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a2f60801-5312-4da2-a438-6103942ccbd7-dbus\") pod \"global-pull-secret-syncer-rnrfc\" (UID: \"a2f60801-5312-4da2-a438-6103942ccbd7\") " pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405053    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-sysctl-conf\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.404828    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405082    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-var-lib-kubelet\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.405144    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs podName:925b0a18-77be-438d-bcfe-27609c648913 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:09.905117115 +0000 UTC m=+3.054411020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs") pod "network-metrics-daemon-psdvf" (UID: "925b0a18-77be-438d-bcfe-27609c648913") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405150    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/94d73f72-b4ea-454b-8162-b4d444385eea-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405174    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-var-lib-kubelet\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405214    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-sys-fs\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405419    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-socket-dir\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405428    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52c1f3d7-1c12-497f-82ed-3d5439d7dfbc-serviceca\") pod \"node-ca-m6bcg\" (UID: \"52c1f3d7-1c12-497f-82ed-3d5439d7dfbc\") " pod="openshift-image-registry/node-ca-m6bcg"
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.404737    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-run\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405507    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-sysconfig\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405573    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-sysctl-d\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405573    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/68f9c6d7-78ed-4086-8e7f-9244b109c1d7-hosts-file\") pod \"node-resolver-8xf5b\" (UID: \"68f9c6d7-78ed-4086-8e7f-9244b109c1d7\") " pod="openshift-dns/node-resolver-8xf5b"
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405634    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/94d73f72-b4ea-454b-8162-b4d444385eea-os-release\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405699    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a2f60801-5312-4da2-a438-6103942ccbd7-dbus\") pod \"global-pull-secret-syncer-rnrfc\" (UID: \"a2f60801-5312-4da2-a438-6103942ccbd7\") " pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405730    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/94d73f72-b4ea-454b-8162-b4d444385eea-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.406232 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.404881    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:09.407191 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.405799    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret podName:a2f60801-5312-4da2-a438-6103942ccbd7 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:09.905784092 +0000 UTC m=+3.055077990 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret") pod "global-pull-secret-syncer-rnrfc" (UID: "a2f60801-5312-4da2-a438-6103942ccbd7") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:09.407191 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405830    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-sysctl-conf\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.407191 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405871    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68f9c6d7-78ed-4086-8e7f-9244b109c1d7-tmp-dir\") pod \"node-resolver-8xf5b\" (UID: \"68f9c6d7-78ed-4086-8e7f-9244b109c1d7\") " pod="openshift-dns/node-resolver-8xf5b"
Jul 29 15:11:09.407191 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405882    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52c1f3d7-1c12-497f-82ed-3d5439d7dfbc-host\") pod \"node-ca-m6bcg\" (UID: \"52c1f3d7-1c12-497f-82ed-3d5439d7dfbc\") " pod="openshift-image-registry/node-ca-m6bcg"
Jul 29 15:11:09.407191 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405849    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.407191 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405914    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/94d73f72-b4ea-454b-8162-b4d444385eea-cni-binary-copy\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.407191 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.405943    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-device-dir\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.407191 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.406029    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-registration-dir\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.408560 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.408520    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-etc-tuned\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.408690 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.408667    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-tmp\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.417887 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.417825    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t526g\" (UniqueName: \"kubernetes.io/projected/68f9c6d7-78ed-4086-8e7f-9244b109c1d7-kube-api-access-t526g\") pod \"node-resolver-8xf5b\" (UID: \"68f9c6d7-78ed-4086-8e7f-9244b109c1d7\") " pod="openshift-dns/node-resolver-8xf5b"
Jul 29 15:11:09.417887 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.417864    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcp8h\" (UniqueName: \"kubernetes.io/projected/52c1f3d7-1c12-497f-82ed-3d5439d7dfbc-kube-api-access-pcp8h\") pod \"node-ca-m6bcg\" (UID: \"52c1f3d7-1c12-497f-82ed-3d5439d7dfbc\") " pod="openshift-image-registry/node-ca-m6bcg"
Jul 29 15:11:09.419795 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.419763    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk78n\" (UniqueName: \"kubernetes.io/projected/2d9b7491-9bcc-4295-b0a9-34b8b2f236cc-kube-api-access-fk78n\") pod \"aws-ebs-csi-driver-node-phb2b\" (UID: \"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.420584 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.420557    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m5fd\" (UniqueName: \"kubernetes.io/projected/d50cbf8c-dba5-437d-b8fa-5370de45c8c8-kube-api-access-5m5fd\") pod \"tuned-q6prv\" (UID: \"d50cbf8c-dba5-437d-b8fa-5370de45c8c8\") " pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.420691 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.420558    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnw4k\" (UniqueName: \"kubernetes.io/projected/925b0a18-77be-438d-bcfe-27609c648913-kube-api-access-bnw4k\") pod \"network-metrics-daemon-psdvf\" (UID: \"925b0a18-77be-438d-bcfe-27609c648913\") " pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:09.423345 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.423292    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp7xt\" (UniqueName: \"kubernetes.io/projected/94d73f72-b4ea-454b-8162-b4d444385eea-kube-api-access-qp7xt\") pod \"multus-additional-cni-plugins-n9mpt\" (UID: \"94d73f72-b4ea-454b-8162-b4d444385eea\") " pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.492170 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.492138    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zwf85"
Jul 29 15:11:09.504688 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:09.504655    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb36df5c1_83d8_4ac9_8be0_2c7aa7086502.slice/crio-7e3a16ebe91e30c16342c15b763113ee58b1d991db4af0909fa3e46019feb774 WatchSource:0}: Error finding container 7e3a16ebe91e30c16342c15b763113ee58b1d991db4af0909fa3e46019feb774: Status 404 returned error can't find the container with id 7e3a16ebe91e30c16342c15b763113ee58b1d991db4af0909fa3e46019feb774
Jul 29 15:11:09.505597 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.505576    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fkv5t"
Jul 29 15:11:09.515868 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.515707    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vx7cn"
Jul 29 15:11:09.516521 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:09.516498    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e7304d8_98a8_4f52_9798_5690011cf2b1.slice/crio-e69db37a69c8c6c3e49c2ba914876fa0c7f95c30fa798e4e8cedc1f3487905ab WatchSource:0}: Error finding container e69db37a69c8c6c3e49c2ba914876fa0c7f95c30fa798e4e8cedc1f3487905ab: Status 404 returned error can't find the container with id e69db37a69c8c6c3e49c2ba914876fa0c7f95c30fa798e4e8cedc1f3487905ab
Jul 29 15:11:09.522915 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.522894    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:09.525733 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:09.525707    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aee2e64_bc45_401a_a783_23c2feda642e.slice/crio-b51cfd4d016263a3e7a044b6fe97d279f6feb3fec4a454b7f5fce74ee99ae1cc WatchSource:0}: Error finding container b51cfd4d016263a3e7a044b6fe97d279f6feb3fec4a454b7f5fce74ee99ae1cc: Status 404 returned error can't find the container with id b51cfd4d016263a3e7a044b6fe97d279f6feb3fec4a454b7f5fce74ee99ae1cc
Jul 29 15:11:09.529314 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.529292    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q6prv"
Jul 29 15:11:09.537916 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.537893    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8xf5b"
Jul 29 15:11:09.546028 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.546010    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m6bcg"
Jul 29 15:11:09.554859 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.554838    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b"
Jul 29 15:11:09.562725 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.562707    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n9mpt"
Jul 29 15:11:09.908968 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.908896    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9bhs\" (UniqueName: \"kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs\") pod \"network-check-target-bs7zd\" (UID: \"3294f075-e06b-44fa-b813-85ccc30860c1\") " pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:09.908968 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.908937    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret\") pod \"global-pull-secret-syncer-rnrfc\" (UID: \"a2f60801-5312-4da2-a438-6103942ccbd7\") " pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:09.908968 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:09.908967    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs\") pod \"network-metrics-daemon-psdvf\" (UID: \"925b0a18-77be-438d-bcfe-27609c648913\") " pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:09.909170 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.909091    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:09.909170 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.909091    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:09.909170 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.909153    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret podName:a2f60801-5312-4da2-a438-6103942ccbd7 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:10.909134809 +0000 UTC m=+4.058428693 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret") pod "global-pull-secret-syncer-rnrfc" (UID: "a2f60801-5312-4da2-a438-6103942ccbd7") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:09.909285 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.909214    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:11:09.909285 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.909239    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:11:09.909285 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.909253    2575 projected.go:196] Error preparing data for projected volume kube-api-access-j9bhs for pod openshift-network-diagnostics/network-check-target-bs7zd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:09.909285 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.909218    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs podName:925b0a18-77be-438d-bcfe-27609c648913 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:10.909194878 +0000 UTC m=+4.058488773 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs") pod "network-metrics-daemon-psdvf" (UID: "925b0a18-77be-438d-bcfe-27609c648913") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:09.909439 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:09.909302    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs podName:3294f075-e06b-44fa-b813-85ccc30860c1 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:10.909291068 +0000 UTC m=+4.058584948 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9bhs" (UniqueName: "kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs") pod "network-check-target-bs7zd" (UID: "3294f075-e06b-44fa-b813-85ccc30860c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:10.217938 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:10.217868    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 15:11:10.240780 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:10.240752    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 15:06:09 +0000 UTC" deadline="2028-04-26 21:34:09.274192846 +0000 UTC"
Jul 29 15:11:10.240780 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:10.240776    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15294h22m59.033419998s"
Jul 29 15:11:10.289398 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:10.289371    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:10.289539 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:10.289517    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:10.299935 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:10.299886    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zwf85" event={"ID":"b36df5c1-83d8-4ac9-8be0-2c7aa7086502","Type":"ContainerStarted","Data":"7e3a16ebe91e30c16342c15b763113ee58b1d991db4af0909fa3e46019feb774"}
Jul 29 15:11:10.301060 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:10.301032    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vx7cn" event={"ID":"3aee2e64-bc45-401a-a783-23c2feda642e","Type":"ContainerStarted","Data":"b51cfd4d016263a3e7a044b6fe97d279f6feb3fec4a454b7f5fce74ee99ae1cc"}
Jul 29 15:11:10.302628 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:10.302603    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fkv5t" event={"ID":"5e7304d8-98a8-4f52-9798-5690011cf2b1","Type":"ContainerStarted","Data":"e69db37a69c8c6c3e49c2ba914876fa0c7f95c30fa798e4e8cedc1f3487905ab"}
Jul 29 15:11:10.590213 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:10.590178    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4d1a578_61d7_4a7d_b4c1_7389cb969155.slice/crio-0af0d071204e1d32780dadfe7fbf3367321660aa469d62dc4c11b7d8c117f3f2 WatchSource:0}: Error finding container 0af0d071204e1d32780dadfe7fbf3367321660aa469d62dc4c11b7d8c117f3f2: Status 404 returned error can't find the container with id 0af0d071204e1d32780dadfe7fbf3367321660aa469d62dc4c11b7d8c117f3f2
Jul 29 15:11:10.596073 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:10.596046    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd50cbf8c_dba5_437d_b8fa_5370de45c8c8.slice/crio-4f289cfb69444ded119030369508ab48ea86cc837d957758f35159ee894f5319 WatchSource:0}: Error finding container 4f289cfb69444ded119030369508ab48ea86cc837d957758f35159ee894f5319: Status 404 returned error can't find the container with id 4f289cfb69444ded119030369508ab48ea86cc837d957758f35159ee894f5319
Jul 29 15:11:10.598416 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:10.598392    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d9b7491_9bcc_4295_b0a9_34b8b2f236cc.slice/crio-6d5f299724d85ff4a0ac83b5564a71b456ee6d150d82f32e5572e10076c5d0ab WatchSource:0}: Error finding container 6d5f299724d85ff4a0ac83b5564a71b456ee6d150d82f32e5572e10076c5d0ab: Status 404 returned error can't find the container with id 6d5f299724d85ff4a0ac83b5564a71b456ee6d150d82f32e5572e10076c5d0ab
Jul 29 15:11:10.602495 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:10.602253    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94d73f72_b4ea_454b_8162_b4d444385eea.slice/crio-58f43f6edfb75f5bf41381ce5ccdd686b37ef3b510c074933c623418d82b6752 WatchSource:0}: Error finding container 58f43f6edfb75f5bf41381ce5ccdd686b37ef3b510c074933c623418d82b6752: Status 404 returned error can't find the container with id 58f43f6edfb75f5bf41381ce5ccdd686b37ef3b510c074933c623418d82b6752
Jul 29 15:11:10.918833 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:10.918367    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9bhs\" (UniqueName: \"kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs\") pod \"network-check-target-bs7zd\" (UID: \"3294f075-e06b-44fa-b813-85ccc30860c1\") " pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:10.918833 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:10.918513    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:11:10.918833 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:10.918618    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret\") pod \"global-pull-secret-syncer-rnrfc\" (UID: \"a2f60801-5312-4da2-a438-6103942ccbd7\") " pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:10.918833 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:10.918633    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:11:10.918833 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:10.918648    2575 projected.go:196] Error preparing data for projected volume kube-api-access-j9bhs for pod openshift-network-diagnostics/network-check-target-bs7zd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:10.918833 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:10.918669    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs\") pod \"network-metrics-daemon-psdvf\" (UID: \"925b0a18-77be-438d-bcfe-27609c648913\") " pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:10.918833 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:10.918711    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs podName:3294f075-e06b-44fa-b813-85ccc30860c1 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:12.918692375 +0000 UTC m=+6.067986264 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9bhs" (UniqueName: "kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs") pod "network-check-target-bs7zd" (UID: "3294f075-e06b-44fa-b813-85ccc30860c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:10.918833 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:10.918710    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:10.918833 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:10.918752    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:10.918833 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:10.918763    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret podName:a2f60801-5312-4da2-a438-6103942ccbd7 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:12.918750518 +0000 UTC m=+6.068044407 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret") pod "global-pull-secret-syncer-rnrfc" (UID: "a2f60801-5312-4da2-a438-6103942ccbd7") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:10.918833 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:10.918790    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs podName:925b0a18-77be-438d-bcfe-27609c648913 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:12.918779975 +0000 UTC m=+6.068073869 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs") pod "network-metrics-daemon-psdvf" (UID: "925b0a18-77be-438d-bcfe-27609c648913") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:11.241239 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:11.241101    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 15:06:09 +0000 UTC" deadline="2028-02-20 07:59:14.598819902 +0000 UTC"
Jul 29 15:11:11.241239 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:11.241167    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13696h48m3.357656964s"
Jul 29 15:11:11.291762 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:11.291728    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:11.291914 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:11.291869    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs7zd" podUID="3294f075-e06b-44fa-b813-85ccc30860c1"
Jul 29 15:11:11.292309 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:11.292287    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:11.292431 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:11.292410    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rnrfc" podUID="a2f60801-5312-4da2-a438-6103942ccbd7"
Jul 29 15:11:11.318508 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:11.318478    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8xf5b" event={"ID":"68f9c6d7-78ed-4086-8e7f-9244b109c1d7","Type":"ContainerStarted","Data":"f8980570f20d7d4261d9262bb1485584f4948e417c22c6683a3fed2457d87a53"}
Jul 29 15:11:11.327929 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:11.327904    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b" event={"ID":"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc","Type":"ContainerStarted","Data":"6d5f299724d85ff4a0ac83b5564a71b456ee6d150d82f32e5572e10076c5d0ab"}
Jul 29 15:11:11.335532 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:11.335502    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q6prv" event={"ID":"d50cbf8c-dba5-437d-b8fa-5370de45c8c8","Type":"ContainerStarted","Data":"4f289cfb69444ded119030369508ab48ea86cc837d957758f35159ee894f5319"}
Jul 29 15:11:11.340972 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:11.340916    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5" event={"ID":"a4d1a578-61d7-4a7d-b4c1-7389cb969155","Type":"ContainerStarted","Data":"0af0d071204e1d32780dadfe7fbf3367321660aa469d62dc4c11b7d8c117f3f2"}
Jul 29 15:11:11.349984 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:11.349960    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m6bcg" event={"ID":"52c1f3d7-1c12-497f-82ed-3d5439d7dfbc","Type":"ContainerStarted","Data":"36c73bedfbd1b495abecb249df374556b2b52587fdfecc1cc75a58ba6b390f4b"}
Jul 29 15:11:11.357442 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:11.357418    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n9mpt" event={"ID":"94d73f72-b4ea-454b-8162-b4d444385eea","Type":"ContainerStarted","Data":"58f43f6edfb75f5bf41381ce5ccdd686b37ef3b510c074933c623418d82b6752"}
Jul 29 15:11:11.366402 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:11.366371    2575 generic.go:356] "Generic (PLEG): container finished" podID="fc390c647fd85218bf5e68bb219f108a" containerID="de500ef98f37f9beda0efc189327950d69c57098a75ea0b416551df6863b0ba2" exitCode=0
Jul 29 15:11:11.366496 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:11.366454    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal" event={"ID":"fc390c647fd85218bf5e68bb219f108a","Type":"ContainerDied","Data":"de500ef98f37f9beda0efc189327950d69c57098a75ea0b416551df6863b0ba2"}
Jul 29 15:11:11.375301 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:11.374855    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-116.ec2.internal" event={"ID":"5da59de62b315dbd806daa423c5548a3","Type":"ContainerStarted","Data":"f3fd3f8334712a45fbdc9f6dfa115d6609371c81aa78590f1b9d8391eedc9461"}
Jul 29 15:11:12.289839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:12.289419    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:12.289839 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:12.289559    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:12.938673 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:12.938641    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9bhs\" (UniqueName: \"kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs\") pod \"network-check-target-bs7zd\" (UID: \"3294f075-e06b-44fa-b813-85ccc30860c1\") " pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:12.938825 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:12.938690    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret\") pod \"global-pull-secret-syncer-rnrfc\" (UID: \"a2f60801-5312-4da2-a438-6103942ccbd7\") " pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:12.938825 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:12.938729    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs\") pod \"network-metrics-daemon-psdvf\" (UID: \"925b0a18-77be-438d-bcfe-27609c648913\") " pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:12.938965 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:12.938866    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:12.938965 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:12.938915    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs podName:925b0a18-77be-438d-bcfe-27609c648913 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:16.938897443 +0000 UTC m=+10.088191335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs") pod "network-metrics-daemon-psdvf" (UID: "925b0a18-77be-438d-bcfe-27609c648913") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:12.939068 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:12.938992    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:11:12.939068 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:12.939005    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:11:12.939068 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:12.939016    2575 projected.go:196] Error preparing data for projected volume kube-api-access-j9bhs for pod openshift-network-diagnostics/network-check-target-bs7zd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:12.939068 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:12.939050    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs podName:3294f075-e06b-44fa-b813-85ccc30860c1 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:16.939038546 +0000 UTC m=+10.088332438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9bhs" (UniqueName: "kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs") pod "network-check-target-bs7zd" (UID: "3294f075-e06b-44fa-b813-85ccc30860c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:12.939268 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:12.939122    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:12.939268 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:12.939161    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret podName:a2f60801-5312-4da2-a438-6103942ccbd7 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:16.939150413 +0000 UTC m=+10.088444311 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret") pod "global-pull-secret-syncer-rnrfc" (UID: "a2f60801-5312-4da2-a438-6103942ccbd7") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:13.289595 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:13.289437    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:13.289595 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:13.289576    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs7zd" podUID="3294f075-e06b-44fa-b813-85ccc30860c1"
Jul 29 15:11:13.290016 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:13.289907    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:13.290494 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:13.290012    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rnrfc" podUID="a2f60801-5312-4da2-a438-6103942ccbd7"
Jul 29 15:11:13.391361 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:13.390487    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal" event={"ID":"fc390c647fd85218bf5e68bb219f108a","Type":"ContainerStarted","Data":"19b6bde887bcd1879fd3f3ed6fd8f068bc8deaf8339033cfa6f04fdaaee1754f"}
Jul 29 15:11:13.397341 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:13.396784    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zwf85" event={"ID":"b36df5c1-83d8-4ac9-8be0-2c7aa7086502","Type":"ContainerStarted","Data":"14edb7e832a4bc159b68a81511f0c5e2c620874ce346f0018555e399ba4926a0"}
Jul 29 15:11:13.411551 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:13.410095    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-116.ec2.internal" podStartSLOduration=5.410071438 podStartE2EDuration="5.410071438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:11.408122929 +0000 UTC m=+4.557416832" watchObservedRunningTime="2026-07-29 15:11:13.410071438 +0000 UTC m=+6.559365339"
Jul 29 15:11:13.426878 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:13.426820    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-116.ec2.internal" podStartSLOduration=5.426795664 podStartE2EDuration="5.426795664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:13.411347159 +0000 UTC m=+6.560641063" watchObservedRunningTime="2026-07-29 15:11:13.426795664 +0000 UTC m=+6.576089561"
Jul 29 15:11:14.289260 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:14.289218    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:14.289507 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:14.289465    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:15.289157 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:15.289121    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:15.289541 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:15.289271    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs7zd" podUID="3294f075-e06b-44fa-b813-85ccc30860c1"
Jul 29 15:11:15.289691 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:15.289673    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:15.289789 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:15.289771    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rnrfc" podUID="a2f60801-5312-4da2-a438-6103942ccbd7"
Jul 29 15:11:16.289220 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:16.289179    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:16.289710 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:16.289358    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:16.974488 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:16.974448    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9bhs\" (UniqueName: \"kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs\") pod \"network-check-target-bs7zd\" (UID: \"3294f075-e06b-44fa-b813-85ccc30860c1\") " pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:16.974708 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:16.974507    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret\") pod \"global-pull-secret-syncer-rnrfc\" (UID: \"a2f60801-5312-4da2-a438-6103942ccbd7\") " pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:16.974708 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:16.974546    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs\") pod \"network-metrics-daemon-psdvf\" (UID: \"925b0a18-77be-438d-bcfe-27609c648913\") " pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:16.974708 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:16.974632    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:16.974708 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:16.974634    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:11:16.974708 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:16.974661    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:11:16.974708 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:16.974675    2575 projected.go:196] Error preparing data for projected volume kube-api-access-j9bhs for pod openshift-network-diagnostics/network-check-target-bs7zd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:16.974708 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:16.974688    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs podName:925b0a18-77be-438d-bcfe-27609c648913 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:24.974674821 +0000 UTC m=+18.123968719 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs") pod "network-metrics-daemon-psdvf" (UID: "925b0a18-77be-438d-bcfe-27609c648913") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:16.975102 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:16.974772    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs podName:3294f075-e06b-44fa-b813-85ccc30860c1 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:24.974752464 +0000 UTC m=+18.124046341 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9bhs" (UniqueName: "kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs") pod "network-check-target-bs7zd" (UID: "3294f075-e06b-44fa-b813-85ccc30860c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:16.975102 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:16.974851    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:16.975102 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:16.974894    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret podName:a2f60801-5312-4da2-a438-6103942ccbd7 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:24.974880514 +0000 UTC m=+18.124174396 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret") pod "global-pull-secret-syncer-rnrfc" (UID: "a2f60801-5312-4da2-a438-6103942ccbd7") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:17.290150 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:17.290061    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:17.290703 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:17.290137    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:17.290703 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:17.290543    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rnrfc" podUID="a2f60801-5312-4da2-a438-6103942ccbd7"
Jul 29 15:11:17.290703 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:17.290394    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs7zd" podUID="3294f075-e06b-44fa-b813-85ccc30860c1"
Jul 29 15:11:18.004763 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:18.004702    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zwf85"
Jul 29 15:11:18.005759 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:18.005734    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zwf85"
Jul 29 15:11:18.023703 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:18.023643    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zwf85" podStartSLOduration=7.791314652 podStartE2EDuration="11.023621273s" totalImagesPullingTime="3.232306621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:13.427981647 +0000 UTC m=+6.577275548" watchObservedRunningTime="2026-07-29 15:11:18.023621273 +0000 UTC m=+11.172915174"
Jul 29 15:11:18.289262 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:18.289155    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:18.289443 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:18.289301    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:19.289102 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:19.289068    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:19.289605 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:19.289193    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs7zd" podUID="3294f075-e06b-44fa-b813-85ccc30860c1"
Jul 29 15:11:19.289881 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:19.289857    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:19.290004 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:19.289976    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rnrfc" podUID="a2f60801-5312-4da2-a438-6103942ccbd7"
Jul 29 15:11:20.288808 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:20.288772    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:20.288998 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:20.288892    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:21.289695 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:21.289651    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:21.290174 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:21.289656    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:21.290174 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:21.289796    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs7zd" podUID="3294f075-e06b-44fa-b813-85ccc30860c1"
Jul 29 15:11:21.290174 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:21.289879    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rnrfc" podUID="a2f60801-5312-4da2-a438-6103942ccbd7"
Jul 29 15:11:22.288921 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:22.288881    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:22.289094 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:22.289006    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:23.289469 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:23.289432    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:23.289469 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:23.289473    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:23.290084 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:23.289570    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs7zd" podUID="3294f075-e06b-44fa-b813-85ccc30860c1"
Jul 29 15:11:23.290084 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:23.289691    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rnrfc" podUID="a2f60801-5312-4da2-a438-6103942ccbd7"
Jul 29 15:11:24.289827 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:24.289788    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:24.290311 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:24.289920    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:25.037799 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:25.037754    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9bhs\" (UniqueName: \"kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs\") pod \"network-check-target-bs7zd\" (UID: \"3294f075-e06b-44fa-b813-85ccc30860c1\") " pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:25.037985 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:25.037809    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret\") pod \"global-pull-secret-syncer-rnrfc\" (UID: \"a2f60801-5312-4da2-a438-6103942ccbd7\") " pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:25.037985 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:25.037939    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:25.037985 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:25.037949    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs\") pod \"network-metrics-daemon-psdvf\" (UID: \"925b0a18-77be-438d-bcfe-27609c648913\") " pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:25.038129 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:25.037956    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:11:25.038129 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:25.038031    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:11:25.038129 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:25.038043    2575 projected.go:196] Error preparing data for projected volume kube-api-access-j9bhs for pod openshift-network-diagnostics/network-check-target-bs7zd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:25.038129 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:25.038011    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret podName:a2f60801-5312-4da2-a438-6103942ccbd7 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:41.037989512 +0000 UTC m=+34.187283415 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret") pod "global-pull-secret-syncer-rnrfc" (UID: "a2f60801-5312-4da2-a438-6103942ccbd7") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:25.038129 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:25.038095    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs podName:3294f075-e06b-44fa-b813-85ccc30860c1 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:41.038083807 +0000 UTC m=+34.187377685 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9bhs" (UniqueName: "kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs") pod "network-check-target-bs7zd" (UID: "3294f075-e06b-44fa-b813-85ccc30860c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:25.038129 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:25.038011    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:25.038129 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:25.038128    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs podName:925b0a18-77be-438d-bcfe-27609c648913 nodeName:}" failed. No retries permitted until 2026-07-29 15:11:41.038118479 +0000 UTC m=+34.187412361 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs") pod "network-metrics-daemon-psdvf" (UID: "925b0a18-77be-438d-bcfe-27609c648913") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:25.289731 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:25.289648    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:25.289883 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:25.289792    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rnrfc" podUID="a2f60801-5312-4da2-a438-6103942ccbd7"
Jul 29 15:11:25.289883 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:25.289837    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:25.290248 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:25.289958    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs7zd" podUID="3294f075-e06b-44fa-b813-85ccc30860c1"
Jul 29 15:11:26.289032 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:26.288997    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:26.289213 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:26.289118    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:27.290359 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:27.290311    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:27.290805 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:27.290427    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs7zd" podUID="3294f075-e06b-44fa-b813-85ccc30860c1"
Jul 29 15:11:27.290805 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:27.290485    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:27.290805 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:27.290608    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rnrfc" podUID="a2f60801-5312-4da2-a438-6103942ccbd7"
Jul 29 15:11:28.289611 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:28.289570    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:28.289768 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:28.289728    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:29.289306 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:29.289277    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:29.289944 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:29.289414    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs7zd" podUID="3294f075-e06b-44fa-b813-85ccc30860c1"
Jul 29 15:11:29.289944 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:29.289445    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:29.289944 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:29.289504    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rnrfc" podUID="a2f60801-5312-4da2-a438-6103942ccbd7"
Jul 29 15:11:29.432312 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:29.432274    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b" event={"ID":"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc","Type":"ContainerStarted","Data":"5af63e1789a1ab11eee05237a3108ceb0beb47d22a82bea931ce67f0b7a289ae"}
Jul 29 15:11:29.433517 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:29.433487    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q6prv" event={"ID":"d50cbf8c-dba5-437d-b8fa-5370de45c8c8","Type":"ContainerStarted","Data":"7c149ddb4ea40f60ba7301f07d49f9e669b86aac1792d710d2a1fbcc95595bd0"}
Jul 29 15:11:29.434872 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:29.434854    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:11:29.437992 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:29.435517    2575 generic.go:356] "Generic (PLEG): container finished" podID="a4d1a578-61d7-4a7d-b4c1-7389cb969155" containerID="9509743a48d400e7ff355c89609d6dc344834f080d6928180d82b07fdc0c715f" exitCode=1
Jul 29 15:11:29.437992 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:29.435647    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5" event={"ID":"a4d1a578-61d7-4a7d-b4c1-7389cb969155","Type":"ContainerDied","Data":"9509743a48d400e7ff355c89609d6dc344834f080d6928180d82b07fdc0c715f"}
Jul 29 15:11:29.437992 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:29.435780    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5" event={"ID":"a4d1a578-61d7-4a7d-b4c1-7389cb969155","Type":"ContainerStarted","Data":"e9e0a717bb5600e09e104fe40f8690758438543ee8d73a93f07c3ef9a81e1e99"}
Jul 29 15:11:29.439923 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:29.439895    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vx7cn" event={"ID":"3aee2e64-bc45-401a-a783-23c2feda642e","Type":"ContainerStarted","Data":"c39c47f2674ebd61c4e08a2feeabac60bacd053394ca0d82d211571837fb36b1"}
Jul 29 15:11:29.441470 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:29.441448    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m6bcg" event={"ID":"52c1f3d7-1c12-497f-82ed-3d5439d7dfbc","Type":"ContainerStarted","Data":"60b842a8a85f169b8b1cdb83e36b56986b2d0f4c5d9d6fd75a6cf1b78e3a9021"}
Jul 29 15:11:29.443081 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:29.443059    2575 generic.go:356] "Generic (PLEG): container finished" podID="94d73f72-b4ea-454b-8162-b4d444385eea" containerID="bc780f04553490dd24b7f9bea7daf8df30d73f9e679913a6051aa1f99d7f9be6" exitCode=0
Jul 29 15:11:29.443171 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:29.443151    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n9mpt" event={"ID":"94d73f72-b4ea-454b-8162-b4d444385eea","Type":"ContainerDied","Data":"bc780f04553490dd24b7f9bea7daf8df30d73f9e679913a6051aa1f99d7f9be6"}
Jul 29 15:11:29.444746 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:29.444628    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8xf5b" event={"ID":"68f9c6d7-78ed-4086-8e7f-9244b109c1d7","Type":"ContainerStarted","Data":"6064c026d6c8841f3a87c1c6d3129b8f4a4da5d80398afaf778a49aec05cbabb"}
Jul 29 15:11:29.454728 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:29.454548    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-q6prv" podStartSLOduration=4.213810016 podStartE2EDuration="22.454532537s" totalImagesPullingTime="18.240722521s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:29.45395717 +0000 UTC m=+22.603251069" watchObservedRunningTime="2026-07-29 15:11:29.454532537 +0000 UTC m=+22.603826437"
Jul 29 15:11:29.472709 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:29.472664    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vx7cn" podStartSLOduration=3.122359574 podStartE2EDuration="22.472646036s" totalImagesPullingTime="19.350286462s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:29.472039021 +0000 UTC m=+22.621332922" watchObservedRunningTime="2026-07-29 15:11:29.472646036 +0000 UTC m=+22.621939943"
Jul 29 15:11:29.489857 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:29.489668    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-m6bcg" podStartSLOduration=4.571649921 podStartE2EDuration="22.489652735s" totalImagesPullingTime="17.918002814s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:29.489200514 +0000 UTC m=+22.638494428" watchObservedRunningTime="2026-07-29 15:11:29.489652735 +0000 UTC m=+22.638946635"
Jul 29 15:11:29.533209 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:29.532985    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8xf5b" podStartSLOduration=4.319133275 podStartE2EDuration="22.532969465s" totalImagesPullingTime="18.21383619s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:29.532813538 +0000 UTC m=+22.682107439" watchObservedRunningTime="2026-07-29 15:11:29.532969465 +0000 UTC m=+22.682263365"
Jul 29 15:11:30.289815 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:30.289546    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:30.289815 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:30.289680    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:30.450410 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:30.450384    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:11:30.450876 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:30.450847    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5" event={"ID":"a4d1a578-61d7-4a7d-b4c1-7389cb969155","Type":"ContainerStarted","Data":"834fd44b364e98159b6b8987f905d1cb86b42b8c59d0153470dd57b2c47d7a90"}
Jul 29 15:11:30.450979 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:30.450940    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5" event={"ID":"a4d1a578-61d7-4a7d-b4c1-7389cb969155","Type":"ContainerStarted","Data":"aa98bb9f829c42d860b971494e029cd6c9001b146a3a89399731acf75436d81d"}
Jul 29 15:11:30.451041 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:30.450991    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5" event={"ID":"a4d1a578-61d7-4a7d-b4c1-7389cb969155","Type":"ContainerStarted","Data":"1771e8cc9bef21783fda4d1108a9ae6740ae50a383e524a92602826ffbddeb37"}
Jul 29 15:11:30.451083 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:30.451041    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5" event={"ID":"a4d1a578-61d7-4a7d-b4c1-7389cb969155","Type":"ContainerStarted","Data":"d854ad9dc246e118a4b79fb252a7a7b529189ef797b686edc4ddada7694a5659"}
Jul 29 15:11:30.452512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:30.452473    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fkv5t" event={"ID":"5e7304d8-98a8-4f52-9798-5690011cf2b1","Type":"ContainerStarted","Data":"0dc0ae8024580347b1d95157a0040a098f61700bbeb02ba84fe9cd6242d1ce0a"}
Jul 29 15:11:30.468979 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:30.468928    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fkv5t" podStartSLOduration=4.444259118 podStartE2EDuration="23.468909758s" totalImagesPullingTime="19.02465064s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:30.468652333 +0000 UTC m=+23.617946234" watchObservedRunningTime="2026-07-29 15:11:30.468909758 +0000 UTC m=+23.618203657"
Jul 29 15:11:30.783590 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:30.783558    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 15:11:31.264282 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:31.264093    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T15:11:30.78358007Z","UUID":"e7ef6fe7-7ed5-42f0-8c5b-de57e5183387","Handler":null,"Name":"","Endpoint":""}
Jul 29 15:11:31.266078 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:31.266046    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 15:11:31.266078 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:31.266084    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 15:11:31.289974 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:31.289354    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:31.289974 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:31.289527    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs7zd" podUID="3294f075-e06b-44fa-b813-85ccc30860c1"
Jul 29 15:11:31.289974 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:31.289820    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:31.289974 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:31.289928    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rnrfc" podUID="a2f60801-5312-4da2-a438-6103942ccbd7"
Jul 29 15:11:31.456989 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:31.456955    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b" event={"ID":"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc","Type":"ContainerStarted","Data":"a3babe42dd9fb2e9ba761b12a307a9bb02b6d3941d8f773cb04bcf26f5aaa2ac"}
Jul 29 15:11:32.289602 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:32.289516    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:32.289770 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:32.289642    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:32.462229 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:32.461964    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b" event={"ID":"2d9b7491-9bcc-4295-b0a9-34b8b2f236cc","Type":"ContainerStarted","Data":"4d0ddb4af10a893c5a36722d629777bcc26a3d7b49954d19cbe3c88fdda67c04"}
Jul 29 15:11:32.465658 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:32.465625    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:11:32.466039 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:32.466015    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5" event={"ID":"a4d1a578-61d7-4a7d-b4c1-7389cb969155","Type":"ContainerStarted","Data":"a900e9eb67f0bdba1d0435237dc8309011e61d3eaa98cba016903d1efae564d5"}
Jul 29 15:11:33.289438 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:33.289399    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:33.289438 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:33.289428    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:33.289679 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:33.289538    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rnrfc" podUID="a2f60801-5312-4da2-a438-6103942ccbd7"
Jul 29 15:11:33.289679 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:33.289645    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs7zd" podUID="3294f075-e06b-44fa-b813-85ccc30860c1"
Jul 29 15:11:34.288949 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:34.288910    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:34.289567 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:34.289042    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:34.472907 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:34.472662    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:11:34.473203 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:34.473176    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5" event={"ID":"a4d1a578-61d7-4a7d-b4c1-7389cb969155","Type":"ContainerStarted","Data":"69b6f724a4f4c72057fc73da8e5809db2237a636bc914a89ea54696ad47c6777"}
Jul 29 15:11:34.473427 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:34.473397    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:34.473709 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:34.473690    2575 scope.go:117] "RemoveContainer" containerID="9509743a48d400e7ff355c89609d6dc344834f080d6928180d82b07fdc0c715f"
Jul 29 15:11:34.475194 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:34.475165    2575 generic.go:356] "Generic (PLEG): container finished" podID="94d73f72-b4ea-454b-8162-b4d444385eea" containerID="21af438b851352b0f7f5e046b4d234101f682af0612605091c38776920299090" exitCode=0
Jul 29 15:11:34.475290 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:34.475206    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n9mpt" event={"ID":"94d73f72-b4ea-454b-8162-b4d444385eea","Type":"ContainerDied","Data":"21af438b851352b0f7f5e046b4d234101f682af0612605091c38776920299090"}
Jul 29 15:11:34.490275 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:34.490255    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:34.513988 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:34.513934    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phb2b" podStartSLOduration=6.6793343830000005 podStartE2EDuration="27.513913783s" totalImagesPullingTime="20.8345794s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:32.482557344 +0000 UTC m=+25.631851246" watchObservedRunningTime="2026-07-29 15:11:34.513913783 +0000 UTC m=+27.663207681"
Jul 29 15:11:35.289466 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:35.289430    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:35.289891 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:35.289436    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:35.289891 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:35.289565    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs7zd" podUID="3294f075-e06b-44fa-b813-85ccc30860c1"
Jul 29 15:11:35.289891 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:35.289658    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rnrfc" podUID="a2f60801-5312-4da2-a438-6103942ccbd7"
Jul 29 15:11:35.488532 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:35.488452    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:11:35.488919 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:35.488888    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5" event={"ID":"a4d1a578-61d7-4a7d-b4c1-7389cb969155","Type":"ContainerStarted","Data":"4f2eeb6788026c36e9d843e5fa31fff429a9c1efa4bd371e69a30512e1beedd7"}
Jul 29 15:11:35.489138 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:35.489111    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:35.489217 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:35.489205    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:35.511745 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:35.511636    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:11:35.524212 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:35.524159    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5" podStartSLOduration=10.027381473 podStartE2EDuration="28.524142817s" totalImagesPullingTime="18.496761344s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:35.523997355 +0000 UTC m=+28.673291290" watchObservedRunningTime="2026-07-29 15:11:35.524142817 +0000 UTC m=+28.673436746"
Jul 29 15:11:36.219883 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:36.219847    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zwf85"
Jul 29 15:11:36.220044 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:36.219977    2575 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 29 15:11:36.220440 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:36.220420    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zwf85"
Jul 29 15:11:36.289664 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:36.289628    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:36.290042 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:36.289750    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:36.493927 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:36.493839    2575 generic.go:356] "Generic (PLEG): container finished" podID="94d73f72-b4ea-454b-8162-b4d444385eea" containerID="a252d678c8d29ea6b7b59f561d793500c5c4fa1ba92dbf06af31e6c3bb463ab1" exitCode=0
Jul 29 15:11:36.494085 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:36.493921    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n9mpt" event={"ID":"94d73f72-b4ea-454b-8162-b4d444385eea","Type":"ContainerDied","Data":"a252d678c8d29ea6b7b59f561d793500c5c4fa1ba92dbf06af31e6c3bb463ab1"}
Jul 29 15:11:36.590586 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:36.590552    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bs7zd"]
Jul 29 15:11:36.590764 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:36.590653    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:36.590764 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:36.590738    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs7zd" podUID="3294f075-e06b-44fa-b813-85ccc30860c1"
Jul 29 15:11:36.594031 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:36.594005    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rnrfc"]
Jul 29 15:11:36.594181 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:36.594102    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:36.594232 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:36.594199    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rnrfc" podUID="a2f60801-5312-4da2-a438-6103942ccbd7"
Jul 29 15:11:36.594570 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:36.594549    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-psdvf"]
Jul 29 15:11:36.594648 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:36.594637    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:36.594739 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:36.594721    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:38.288963 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:38.288753    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:38.289469 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:38.288790    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:38.289469 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:38.289056    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs7zd" podUID="3294f075-e06b-44fa-b813-85ccc30860c1"
Jul 29 15:11:38.289469 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:38.288808    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:38.289469 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:38.289119    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rnrfc" podUID="a2f60801-5312-4da2-a438-6103942ccbd7"
Jul 29 15:11:38.289469 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:38.289201    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:38.501793 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:38.501705    2575 generic.go:356] "Generic (PLEG): container finished" podID="94d73f72-b4ea-454b-8162-b4d444385eea" containerID="36165f92f08f999d186fbb21cf64fa29cba6244baf5f1fb07e74d10ca6065d7b" exitCode=0
Jul 29 15:11:38.501793 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:38.501763    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n9mpt" event={"ID":"94d73f72-b4ea-454b-8162-b4d444385eea","Type":"ContainerDied","Data":"36165f92f08f999d186fbb21cf64fa29cba6244baf5f1fb07e74d10ca6065d7b"}
Jul 29 15:11:40.289628 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:40.289595    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:40.290303 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:40.289640    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:40.290303 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:40.289640    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:40.290303 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:40.289738    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rnrfc" podUID="a2f60801-5312-4da2-a438-6103942ccbd7"
Jul 29 15:11:40.290303 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:40.289824    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bs7zd" podUID="3294f075-e06b-44fa-b813-85ccc30860c1"
Jul 29 15:11:40.290303 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:40.289905    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-psdvf" podUID="925b0a18-77be-438d-bcfe-27609c648913"
Jul 29 15:11:41.062980 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.062912    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs\") pod \"network-metrics-daemon-psdvf\" (UID: \"925b0a18-77be-438d-bcfe-27609c648913\") " pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:41.063179 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.063024    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9bhs\" (UniqueName: \"kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs\") pod \"network-check-target-bs7zd\" (UID: \"3294f075-e06b-44fa-b813-85ccc30860c1\") " pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:41.063179 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.063056    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret\") pod \"global-pull-secret-syncer-rnrfc\" (UID: \"a2f60801-5312-4da2-a438-6103942ccbd7\") " pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:41.063179 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:41.063070    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:41.063179 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:41.063135    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs podName:925b0a18-77be-438d-bcfe-27609c648913 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:13.063119166 +0000 UTC m=+66.212413048 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs") pod "network-metrics-daemon-psdvf" (UID: "925b0a18-77be-438d-bcfe-27609c648913") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 15:11:41.063179 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:41.063152    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:41.063421 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:41.063207    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret podName:a2f60801-5312-4da2-a438-6103942ccbd7 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:13.063190835 +0000 UTC m=+66.212484716 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret") pod "global-pull-secret-syncer-rnrfc" (UID: "a2f60801-5312-4da2-a438-6103942ccbd7") : object "kube-system"/"original-pull-secret" not registered
Jul 29 15:11:41.063421 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:41.063216    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 15:11:41.063421 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:41.063237    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 15:11:41.063421 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:41.063250    2575 projected.go:196] Error preparing data for projected volume kube-api-access-j9bhs for pod openshift-network-diagnostics/network-check-target-bs7zd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:41.063421 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:41.063297    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs podName:3294f075-e06b-44fa-b813-85ccc30860c1 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:13.063283036 +0000 UTC m=+66.212576920 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9bhs" (UniqueName: "kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs") pod "network-check-target-bs7zd" (UID: "3294f075-e06b-44fa-b813-85ccc30860c1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 15:11:41.663188 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.663101    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-116.ec2.internal" event="NodeReady"
Jul 29 15:11:41.663810 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.663298    2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 15:11:41.711239 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.711079    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zvjbc"]
Jul 29 15:11:41.715231 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.715202    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:11:41.715532 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.715513    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tps4w"]
Jul 29 15:11:41.717237 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.717217    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 15:11:41.717539 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.717519    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 15:11:41.717921 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.717884    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dw7hg\""
Jul 29 15:11:41.718376 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.718353    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tps4w"
Jul 29 15:11:41.720617 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.720484    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2gcqd\""
Jul 29 15:11:41.720617 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.720535    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 15:11:41.720767 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.720735    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 15:11:41.721000 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.720824    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 15:11:41.726295 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.726230    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zvjbc"]
Jul 29 15:11:41.733669 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.733644    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tps4w"]
Jul 29 15:11:41.869894 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.869860    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a897ee8-304b-48a3-8afe-c09656ab3028-cert\") pod \"ingress-canary-tps4w\" (UID: \"4a897ee8-304b-48a3-8afe-c09656ab3028\") " pod="openshift-ingress-canary/ingress-canary-tps4w"
Jul 29 15:11:41.870065 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.869907    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-config-volume\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:11:41.870065 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.869938    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:11:41.870065 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.869965    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-tmp-dir\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:11:41.870065 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.870001    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f49rg\" (UniqueName: \"kubernetes.io/projected/4a897ee8-304b-48a3-8afe-c09656ab3028-kube-api-access-f49rg\") pod \"ingress-canary-tps4w\" (UID: \"4a897ee8-304b-48a3-8afe-c09656ab3028\") " pod="openshift-ingress-canary/ingress-canary-tps4w"
Jul 29 15:11:41.870065 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.870036    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwhgb\" (UniqueName: \"kubernetes.io/projected/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-kube-api-access-xwhgb\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:11:41.971221 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.971121    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a897ee8-304b-48a3-8afe-c09656ab3028-cert\") pod \"ingress-canary-tps4w\" (UID: \"4a897ee8-304b-48a3-8afe-c09656ab3028\") " pod="openshift-ingress-canary/ingress-canary-tps4w"
Jul 29 15:11:41.971221 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.971209    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-config-volume\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:11:41.971481 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.971236    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:11:41.971481 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.971307    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-tmp-dir\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:11:41.971481 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:41.971403    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:11:41.971481 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:41.971473    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls podName:4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e nodeName:}" failed. No retries permitted until 2026-07-29 15:11:42.471452128 +0000 UTC m=+35.620746023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls") pod "dns-default-zvjbc" (UID: "4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e") : secret "dns-default-metrics-tls" not found
Jul 29 15:11:41.971694 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.971495    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f49rg\" (UniqueName: \"kubernetes.io/projected/4a897ee8-304b-48a3-8afe-c09656ab3028-kube-api-access-f49rg\") pod \"ingress-canary-tps4w\" (UID: \"4a897ee8-304b-48a3-8afe-c09656ab3028\") " pod="openshift-ingress-canary/ingress-canary-tps4w"
Jul 29 15:11:41.971694 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.971540    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwhgb\" (UniqueName: \"kubernetes.io/projected/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-kube-api-access-xwhgb\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:11:41.971694 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.971596    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-tmp-dir\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:11:41.971938 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.971906    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-config-volume\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:11:41.976093 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.976075    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4a897ee8-304b-48a3-8afe-c09656ab3028-cert\") pod \"ingress-canary-tps4w\" (UID: \"4a897ee8-304b-48a3-8afe-c09656ab3028\") " pod="openshift-ingress-canary/ingress-canary-tps4w"
Jul 29 15:11:41.981542 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.981521    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwhgb\" (UniqueName: \"kubernetes.io/projected/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-kube-api-access-xwhgb\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:11:41.981698 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:41.981677    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f49rg\" (UniqueName: \"kubernetes.io/projected/4a897ee8-304b-48a3-8afe-c09656ab3028-kube-api-access-f49rg\") pod \"ingress-canary-tps4w\" (UID: \"4a897ee8-304b-48a3-8afe-c09656ab3028\") " pod="openshift-ingress-canary/ingress-canary-tps4w"
Jul 29 15:11:42.035712 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:42.035674    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tps4w"
Jul 29 15:11:42.289407 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:42.289151    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:11:42.289566 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:42.289172    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:11:42.289566 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:42.289208    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:11:42.291833 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:42.291814    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 15:11:42.292489 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:42.292474    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 15:11:42.292657 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:42.292635    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sgb7s\""
Jul 29 15:11:42.292769 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:42.292679    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 15:11:42.292769 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:42.292682    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 15:11:42.292769 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:42.292683    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cnmln\""
Jul 29 15:11:42.476268 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:42.475891    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:11:42.476268 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:42.476063    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:11:42.476268 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:42.476133    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls podName:4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e nodeName:}" failed. No retries permitted until 2026-07-29 15:11:43.476111578 +0000 UTC m=+36.625405477 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls") pod "dns-default-zvjbc" (UID: "4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e") : secret "dns-default-metrics-tls" not found
Jul 29 15:11:42.542825 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:42.542796    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tps4w"]
Jul 29 15:11:42.548354 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:11:42.548312    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a897ee8_304b_48a3_8afe_c09656ab3028.slice/crio-056d8c5431f9be61eeec2011a1ccaece0626c7b850289911a4df696e07f6b23d WatchSource:0}: Error finding container 056d8c5431f9be61eeec2011a1ccaece0626c7b850289911a4df696e07f6b23d: Status 404 returned error can't find the container with id 056d8c5431f9be61eeec2011a1ccaece0626c7b850289911a4df696e07f6b23d
Jul 29 15:11:43.484127 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:43.484091    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:11:43.484600 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:43.484226    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:11:43.484600 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:43.484287    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls podName:4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e nodeName:}" failed. No retries permitted until 2026-07-29 15:11:45.484271539 +0000 UTC m=+38.633565418 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls") pod "dns-default-zvjbc" (UID: "4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e") : secret "dns-default-metrics-tls" not found
Jul 29 15:11:43.517129 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:43.517093    2575 generic.go:356] "Generic (PLEG): container finished" podID="94d73f72-b4ea-454b-8162-b4d444385eea" containerID="506394be9920e0e191ed0dbf787b9d62dd376ffdd84daa367a893dee791a9aa7" exitCode=0
Jul 29 15:11:43.517287 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:43.517174    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n9mpt" event={"ID":"94d73f72-b4ea-454b-8162-b4d444385eea","Type":"ContainerDied","Data":"506394be9920e0e191ed0dbf787b9d62dd376ffdd84daa367a893dee791a9aa7"}
Jul 29 15:11:43.518621 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:43.518495    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tps4w" event={"ID":"4a897ee8-304b-48a3-8afe-c09656ab3028","Type":"ContainerStarted","Data":"056d8c5431f9be61eeec2011a1ccaece0626c7b850289911a4df696e07f6b23d"}
Jul 29 15:11:44.524428 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:44.524389    2575 generic.go:356] "Generic (PLEG): container finished" podID="94d73f72-b4ea-454b-8162-b4d444385eea" containerID="10edb953fc11275c93aa3977cb4af0f5ec891cdbaa6de3fbe45af2067a970558" exitCode=0
Jul 29 15:11:44.524827 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:44.524442    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n9mpt" event={"ID":"94d73f72-b4ea-454b-8162-b4d444385eea","Type":"ContainerDied","Data":"10edb953fc11275c93aa3977cb4af0f5ec891cdbaa6de3fbe45af2067a970558"}
Jul 29 15:11:45.501349 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:45.501163    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:11:45.501513 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:45.501311    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:11:45.501513 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:45.501450    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls podName:4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e nodeName:}" failed. No retries permitted until 2026-07-29 15:11:49.501428255 +0000 UTC m=+42.650722135 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls") pod "dns-default-zvjbc" (UID: "4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e") : secret "dns-default-metrics-tls" not found
Jul 29 15:11:45.527563 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:45.527532    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tps4w" event={"ID":"4a897ee8-304b-48a3-8afe-c09656ab3028","Type":"ContainerStarted","Data":"50f0f9e458913135a4791940beaeddb84a4c67158175bb1b572227a4b8d7c9e5"}
Jul 29 15:11:45.530814 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:45.530790    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n9mpt" event={"ID":"94d73f72-b4ea-454b-8162-b4d444385eea","Type":"ContainerStarted","Data":"19d91c16eacb3dc838b3b36cd8e995ad8704fee18c2a510443b75c9eccbc05a2"}
Jul 29 15:11:45.545796 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:45.545739    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tps4w" podStartSLOduration=2.097318699 podStartE2EDuration="4.5457266s" totalImagesPullingTime="2.448407901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:45.545420439 +0000 UTC m=+38.694714339" watchObservedRunningTime="2026-07-29 15:11:45.5457266 +0000 UTC m=+38.695020500"
Jul 29 15:11:45.573172 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:45.573124    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-n9mpt" podStartSLOduration=9.514122126 podStartE2EDuration="38.573105713s" totalImagesPullingTime="29.058983587s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:11:45.572023538 +0000 UTC m=+38.721317439" watchObservedRunningTime="2026-07-29 15:11:45.573105713 +0000 UTC m=+38.722399614"
Jul 29 15:11:49.528398 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:49.528358    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:11:49.528824 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:49.528486    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:11:49.528824 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:49.528542    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls podName:4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e nodeName:}" failed. No retries permitted until 2026-07-29 15:11:57.528526214 +0000 UTC m=+50.677820092 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls") pod "dns-default-zvjbc" (UID: "4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e") : secret "dns-default-metrics-tls" not found
Jul 29 15:11:57.582503 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:11:57.582467    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:11:57.582943 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:57.582624    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:11:57.582943 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:11:57.582698    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls podName:4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e nodeName:}" failed. No retries permitted until 2026-07-29 15:12:13.582682628 +0000 UTC m=+66.731976506 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls") pod "dns-default-zvjbc" (UID: "4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e") : secret "dns-default-metrics-tls" not found
Jul 29 15:12:07.514013 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:07.513983    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ksmm5"
Jul 29 15:12:13.090054 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:13.090003    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9bhs\" (UniqueName: \"kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs\") pod \"network-check-target-bs7zd\" (UID: \"3294f075-e06b-44fa-b813-85ccc30860c1\") " pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:12:13.090054 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:13.090060    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret\") pod \"global-pull-secret-syncer-rnrfc\" (UID: \"a2f60801-5312-4da2-a438-6103942ccbd7\") " pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:12:13.090504 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:13.090095    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs\") pod \"network-metrics-daemon-psdvf\" (UID: \"925b0a18-77be-438d-bcfe-27609c648913\") " pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:12:13.095723 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:13.095700    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 15:12:13.095774 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:13.095741    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 15:12:13.097717 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:13.097704    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 15:12:13.100849 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:13.100834    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 29 15:12:13.100900 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:13.100890    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs podName:925b0a18-77be-438d-bcfe-27609c648913 nodeName:}" failed. No retries permitted until 2026-07-29 15:13:17.1008724 +0000 UTC m=+130.250166278 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs") pod "network-metrics-daemon-psdvf" (UID: "925b0a18-77be-438d-bcfe-27609c648913") : secret "metrics-daemon-secret" not found
Jul 29 15:12:13.102922 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:13.102907    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 15:12:13.104520 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:13.104504    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a2f60801-5312-4da2-a438-6103942ccbd7-original-pull-secret\") pod \"global-pull-secret-syncer-rnrfc\" (UID: \"a2f60801-5312-4da2-a438-6103942ccbd7\") " pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:12:13.113620 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:13.113593    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9bhs\" (UniqueName: \"kubernetes.io/projected/3294f075-e06b-44fa-b813-85ccc30860c1-kube-api-access-j9bhs\") pod \"network-check-target-bs7zd\" (UID: \"3294f075-e06b-44fa-b813-85ccc30860c1\") " pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:12:13.201140 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:13.201090    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rnrfc"
Jul 29 15:12:13.218383 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:13.218354    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cnmln\""
Jul 29 15:12:13.226675 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:13.226648    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:12:13.330400 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:13.330371    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rnrfc"]
Jul 29 15:12:13.334084 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:12:13.334058    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2f60801_5312_4da2_a438_6103942ccbd7.slice/crio-3f4ecff149ebd9483cbd97203fa2d509db890b428b0ded2f390a0beb991664b7 WatchSource:0}: Error finding container 3f4ecff149ebd9483cbd97203fa2d509db890b428b0ded2f390a0beb991664b7: Status 404 returned error can't find the container with id 3f4ecff149ebd9483cbd97203fa2d509db890b428b0ded2f390a0beb991664b7
Jul 29 15:12:13.376239 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:13.376213    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bs7zd"]
Jul 29 15:12:13.379261 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:12:13.379238    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3294f075_e06b_44fa_b813_85ccc30860c1.slice/crio-e30b28ab938d67ca865a2db9d9a390c672bcca4687733a4b8bb986ef0fb504de WatchSource:0}: Error finding container e30b28ab938d67ca865a2db9d9a390c672bcca4687733a4b8bb986ef0fb504de: Status 404 returned error can't find the container with id e30b28ab938d67ca865a2db9d9a390c672bcca4687733a4b8bb986ef0fb504de
Jul 29 15:12:13.592573 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:13.592542    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:12:13.592730 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:13.592685    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 15:12:13.592794 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:13.592747    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls podName:4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e nodeName:}" failed. No retries permitted until 2026-07-29 15:12:45.592729255 +0000 UTC m=+98.742023132 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls") pod "dns-default-zvjbc" (UID: "4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e") : secret "dns-default-metrics-tls" not found
Jul 29 15:12:13.592794 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:13.592774    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bs7zd" event={"ID":"3294f075-e06b-44fa-b813-85ccc30860c1","Type":"ContainerStarted","Data":"e30b28ab938d67ca865a2db9d9a390c672bcca4687733a4b8bb986ef0fb504de"}
Jul 29 15:12:13.593967 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:13.593944    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rnrfc" event={"ID":"a2f60801-5312-4da2-a438-6103942ccbd7","Type":"ContainerStarted","Data":"3f4ecff149ebd9483cbd97203fa2d509db890b428b0ded2f390a0beb991664b7"}
Jul 29 15:12:14.006759 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.006721    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-zfj9l"]
Jul 29 15:12:14.020626 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.020599    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.023926 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.023897    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 29 15:12:14.025965 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.025740    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 15:12:14.029991 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.029968    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mbrd7\""
Jul 29 15:12:14.030085 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.030013    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 15:12:14.030085 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.030034    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 29 15:12:14.036514 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.036497    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 29 15:12:14.047063 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.047037    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-zfj9l"]
Jul 29 15:12:14.096899 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.096865    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.097406 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.096916    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-service-ca-bundle\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.097406 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.096999    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-tmp\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.097406 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.097032    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrpbx\" (UniqueName: \"kubernetes.io/projected/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-kube-api-access-jrpbx\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.097406 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.097066    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-snapshots\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.097406 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.097092    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-serving-cert\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.127763 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.127732    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-ztzjg"]
Jul 29 15:12:14.164186 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.164150    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-ztzjg"]
Jul 29 15:12:14.164368 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.164249    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:14.166537 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.166514    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-zvn2v\""
Jul 29 15:12:14.167315 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.167294    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 29 15:12:14.168216 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.168192    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 29 15:12:14.168346 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.168315    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 29 15:12:14.168488 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.168467    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 29 15:12:14.178138 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.178111    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 29 15:12:14.197836 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.197804    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.197954 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.197863    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-service-ca-bundle\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.197954 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.197905    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f268c925-4fbb-42bd-9d51-b2c74d55c34e-config\") pod \"console-operator-584dcb6b54-ztzjg\" (UID: \"f268c925-4fbb-42bd-9d51-b2c74d55c34e\") " pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:14.197954 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.197926    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f268c925-4fbb-42bd-9d51-b2c74d55c34e-trusted-ca\") pod \"console-operator-584dcb6b54-ztzjg\" (UID: \"f268c925-4fbb-42bd-9d51-b2c74d55c34e\") " pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:14.198117 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.197972    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-tmp\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.198117 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.198013    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrpbx\" (UniqueName: \"kubernetes.io/projected/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-kube-api-access-jrpbx\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.198117 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.198057    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-snapshots\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.198117 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.198084    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-serving-cert\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.198314 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.198142    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f268c925-4fbb-42bd-9d51-b2c74d55c34e-serving-cert\") pod \"console-operator-584dcb6b54-ztzjg\" (UID: \"f268c925-4fbb-42bd-9d51-b2c74d55c34e\") " pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:14.198314 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.198178    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq9f9\" (UniqueName: \"kubernetes.io/projected/f268c925-4fbb-42bd-9d51-b2c74d55c34e-kube-api-access-mq9f9\") pod \"console-operator-584dcb6b54-ztzjg\" (UID: \"f268c925-4fbb-42bd-9d51-b2c74d55c34e\") " pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:14.198437 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.198415    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-tmp\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.198617 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.198595    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-service-ca-bundle\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.198764 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.198737    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-snapshots\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.198878 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.198831    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.200551 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.200529    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-serving-cert\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.208587 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.208563    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrpbx\" (UniqueName: \"kubernetes.io/projected/d3ffac2a-3208-473f-9d7f-ca7819f1ecff-kube-api-access-jrpbx\") pod \"insights-operator-5885fdb759-zfj9l\" (UID: \"d3ffac2a-3208-473f-9d7f-ca7819f1ecff\") " pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.214949 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.214932    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8649b6ff55-9t5ss"]
Jul 29 15:12:14.234332 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.234303    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.236839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.236819    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8649b6ff55-9t5ss"]
Jul 29 15:12:14.237107 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.237086    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 15:12:14.237198 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.237184    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-snrjc\""
Jul 29 15:12:14.237253 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.237209    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 15:12:14.237472 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.237453    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 15:12:14.242986 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.242940    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 15:12:14.298832 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.298770    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f268c925-4fbb-42bd-9d51-b2c74d55c34e-config\") pod \"console-operator-584dcb6b54-ztzjg\" (UID: \"f268c925-4fbb-42bd-9d51-b2c74d55c34e\") " pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:14.298832 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.298798    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f268c925-4fbb-42bd-9d51-b2c74d55c34e-trusted-ca\") pod \"console-operator-584dcb6b54-ztzjg\" (UID: \"f268c925-4fbb-42bd-9d51-b2c74d55c34e\") " pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:14.298832 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.298822    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.299097 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.299051    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1b70bba-d880-4c9d-94fc-a78c3b95302e-installation-pull-secrets\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.299097 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.299091    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4dh4\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-kube-api-access-l4dh4\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.299200 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.299145    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f268c925-4fbb-42bd-9d51-b2c74d55c34e-serving-cert\") pod \"console-operator-584dcb6b54-ztzjg\" (UID: \"f268c925-4fbb-42bd-9d51-b2c74d55c34e\") " pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:14.299200 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.299179    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1b70bba-d880-4c9d-94fc-a78c3b95302e-image-registry-private-configuration\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.299315 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.299222    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq9f9\" (UniqueName: \"kubernetes.io/projected/f268c925-4fbb-42bd-9d51-b2c74d55c34e-kube-api-access-mq9f9\") pod \"console-operator-584dcb6b54-ztzjg\" (UID: \"f268c925-4fbb-42bd-9d51-b2c74d55c34e\") " pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:14.299315 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.299249    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1b70bba-d880-4c9d-94fc-a78c3b95302e-ca-trust-extracted\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.299315 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.299290    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-certificates\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.299488 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.299348    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1b70bba-d880-4c9d-94fc-a78c3b95302e-trusted-ca\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.299488 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.299459    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-bound-sa-token\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.299649 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.299629    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f268c925-4fbb-42bd-9d51-b2c74d55c34e-config\") pod \"console-operator-584dcb6b54-ztzjg\" (UID: \"f268c925-4fbb-42bd-9d51-b2c74d55c34e\") " pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:14.299810 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.299788    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f268c925-4fbb-42bd-9d51-b2c74d55c34e-trusted-ca\") pod \"console-operator-584dcb6b54-ztzjg\" (UID: \"f268c925-4fbb-42bd-9d51-b2c74d55c34e\") " pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:14.301939 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.301918    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f268c925-4fbb-42bd-9d51-b2c74d55c34e-serving-cert\") pod \"console-operator-584dcb6b54-ztzjg\" (UID: \"f268c925-4fbb-42bd-9d51-b2c74d55c34e\") " pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:14.310103 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.310080    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq9f9\" (UniqueName: \"kubernetes.io/projected/f268c925-4fbb-42bd-9d51-b2c74d55c34e-kube-api-access-mq9f9\") pod \"console-operator-584dcb6b54-ztzjg\" (UID: \"f268c925-4fbb-42bd-9d51-b2c74d55c34e\") " pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:14.330809 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.330790    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-zfj9l"
Jul 29 15:12:14.400694 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.400659    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.400797 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.400753    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1b70bba-d880-4c9d-94fc-a78c3b95302e-installation-pull-secrets\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.400797 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.400783    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4dh4\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-kube-api-access-l4dh4\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.400870 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.400815    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1b70bba-d880-4c9d-94fc-a78c3b95302e-image-registry-private-configuration\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.400870 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.400851    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1b70bba-d880-4c9d-94fc-a78c3b95302e-ca-trust-extracted\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.400938 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.400880    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-certificates\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.400938 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.400905    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1b70bba-d880-4c9d-94fc-a78c3b95302e-trusted-ca\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.400938 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.400931    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-bound-sa-token\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.401388 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:14.401367    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:12:14.401452 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:14.401390    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8649b6ff55-9t5ss: secret "image-registry-tls" not found
Jul 29 15:12:14.401490 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:14.401459    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls podName:a1b70bba-d880-4c9d-94fc-a78c3b95302e nodeName:}" failed. No retries permitted until 2026-07-29 15:12:14.901438793 +0000 UTC m=+68.050732688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls") pod "image-registry-8649b6ff55-9t5ss" (UID: "a1b70bba-d880-4c9d-94fc-a78c3b95302e") : secret "image-registry-tls" not found
Jul 29 15:12:14.402300 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.402277    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1b70bba-d880-4c9d-94fc-a78c3b95302e-ca-trust-extracted\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.402966 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.402904    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-certificates\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.403485 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.403443    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1b70bba-d880-4c9d-94fc-a78c3b95302e-trusted-ca\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.405194 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.405151    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1b70bba-d880-4c9d-94fc-a78c3b95302e-image-registry-private-configuration\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.406460 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.406440    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1b70bba-d880-4c9d-94fc-a78c3b95302e-installation-pull-secrets\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.414689 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.414635    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-bound-sa-token\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.414986 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.414950    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4dh4\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-kube-api-access-l4dh4\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.475053 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.475022    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:14.487366 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.487303    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-zfj9l"]
Jul 29 15:12:14.491127 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:12:14.491104    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3ffac2a_3208_473f_9d7f_ca7819f1ecff.slice/crio-a9f5349d70fcfe2afa4f9a233d0356509feed8fa81c4eb2842112bb1782f1b9b WatchSource:0}: Error finding container a9f5349d70fcfe2afa4f9a233d0356509feed8fa81c4eb2842112bb1782f1b9b: Status 404 returned error can't find the container with id a9f5349d70fcfe2afa4f9a233d0356509feed8fa81c4eb2842112bb1782f1b9b
Jul 29 15:12:14.599167 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.599135    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-zfj9l" event={"ID":"d3ffac2a-3208-473f-9d7f-ca7819f1ecff","Type":"ContainerStarted","Data":"a9f5349d70fcfe2afa4f9a233d0356509feed8fa81c4eb2842112bb1782f1b9b"}
Jul 29 15:12:14.645961 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.645924    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-ztzjg"]
Jul 29 15:12:14.649303 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:12:14.649274    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf268c925_4fbb_42bd_9d51_b2c74d55c34e.slice/crio-5f4a18b0efaafd0fde71c7ba55ddb2c9302cd60e620785aeb70f6f5a1ce5a4cd WatchSource:0}: Error finding container 5f4a18b0efaafd0fde71c7ba55ddb2c9302cd60e620785aeb70f6f5a1ce5a4cd: Status 404 returned error can't find the container with id 5f4a18b0efaafd0fde71c7ba55ddb2c9302cd60e620785aeb70f6f5a1ce5a4cd
Jul 29 15:12:14.904977 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:14.904939    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:14.905129 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:14.905075    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:12:14.905129 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:14.905089    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8649b6ff55-9t5ss: secret "image-registry-tls" not found
Jul 29 15:12:14.905200 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:14.905141    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls podName:a1b70bba-d880-4c9d-94fc-a78c3b95302e nodeName:}" failed. No retries permitted until 2026-07-29 15:12:15.905127555 +0000 UTC m=+69.054421433 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls") pod "image-registry-8649b6ff55-9t5ss" (UID: "a1b70bba-d880-4c9d-94fc-a78c3b95302e") : secret "image-registry-tls" not found
Jul 29 15:12:15.604587 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:15.604547    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg" event={"ID":"f268c925-4fbb-42bd-9d51-b2c74d55c34e","Type":"ContainerStarted","Data":"5f4a18b0efaafd0fde71c7ba55ddb2c9302cd60e620785aeb70f6f5a1ce5a4cd"}
Jul 29 15:12:15.913889 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:15.913847    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:15.914074 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:15.914039    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:12:15.914074 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:15.914064    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8649b6ff55-9t5ss: secret "image-registry-tls" not found
Jul 29 15:12:15.914200 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:15.914138    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls podName:a1b70bba-d880-4c9d-94fc-a78c3b95302e nodeName:}" failed. No retries permitted until 2026-07-29 15:12:17.914116373 +0000 UTC m=+71.063410269 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls") pod "image-registry-8649b6ff55-9t5ss" (UID: "a1b70bba-d880-4c9d-94fc-a78c3b95302e") : secret "image-registry-tls" not found
Jul 29 15:12:17.927615 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:17.927574    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:17.928020 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:17.927741    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:12:17.928020 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:17.927763    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8649b6ff55-9t5ss: secret "image-registry-tls" not found
Jul 29 15:12:17.928020 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:17.927847    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls podName:a1b70bba-d880-4c9d-94fc-a78c3b95302e nodeName:}" failed. No retries permitted until 2026-07-29 15:12:21.927824858 +0000 UTC m=+75.077118738 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls") pod "image-registry-8649b6ff55-9t5ss" (UID: "a1b70bba-d880-4c9d-94fc-a78c3b95302e") : secret "image-registry-tls" not found
Jul 29 15:12:19.618688 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:19.618650    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rnrfc" event={"ID":"a2f60801-5312-4da2-a438-6103942ccbd7","Type":"ContainerStarted","Data":"c224bf01e92eddbccc000820797239a7572e9325656cf36dd1add3afe71046b6"}
Jul 29 15:12:19.620552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:19.620519    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-zfj9l" event={"ID":"d3ffac2a-3208-473f-9d7f-ca7819f1ecff","Type":"ContainerStarted","Data":"888b6f152e74968cff7456048b5c957e68d48424bcc809a410f413fea7ed8ec0"}
Jul 29 15:12:19.622312 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:19.622259    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bs7zd" event={"ID":"3294f075-e06b-44fa-b813-85ccc30860c1","Type":"ContainerStarted","Data":"4b771cb71134a13f1bf299ee1dfba1bae78ed9c9c42adf17c597c0c8e298b0bc"}
Jul 29 15:12:19.622470 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:19.622401    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:12:19.638654 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:19.638600    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rnrfc" podStartSLOduration=67.011108795 podStartE2EDuration="1m12.63858024s" totalImagesPullingTime="5.627471445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:19.638559488 +0000 UTC m=+72.787853389" watchObservedRunningTime="2026-07-29 15:12:19.63858024 +0000 UTC m=+72.787874141"
Jul 29 15:12:19.659231 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:19.659171    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-zfj9l" podStartSLOduration=2.1867714400000002 podStartE2EDuration="6.659154453s" totalImagesPullingTime="4.472383013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:12:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:19.657753061 +0000 UTC m=+72.807046963" watchObservedRunningTime="2026-07-29 15:12:19.659154453 +0000 UTC m=+72.808448354"
Jul 29 15:12:19.676298 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:19.676241    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bs7zd" podStartSLOduration=67.083088335 podStartE2EDuration="1m12.676220791s" totalImagesPullingTime="5.593132456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:19.675717695 +0000 UTC m=+72.825011596" watchObservedRunningTime="2026-07-29 15:12:19.676220791 +0000 UTC m=+72.825514922"
Jul 29 15:12:20.626811 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:20.626783    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/0.log"
Jul 29 15:12:20.627206 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:20.626823    2575 generic.go:356] "Generic (PLEG): container finished" podID="f268c925-4fbb-42bd-9d51-b2c74d55c34e" containerID="1d2d7a53aa5702552affd10516cbc3b3aa75ff73f4fdca7e4246aef7038fc6d2" exitCode=255
Jul 29 15:12:20.627206 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:20.626902    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg" event={"ID":"f268c925-4fbb-42bd-9d51-b2c74d55c34e","Type":"ContainerDied","Data":"1d2d7a53aa5702552affd10516cbc3b3aa75ff73f4fdca7e4246aef7038fc6d2"}
Jul 29 15:12:20.627206 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:20.627101    2575 scope.go:117] "RemoveContainer" containerID="1d2d7a53aa5702552affd10516cbc3b3aa75ff73f4fdca7e4246aef7038fc6d2"
Jul 29 15:12:21.106074 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.106038    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fsfps"]
Jul 29 15:12:21.109428 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.109407    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:21.112881 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.112862    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 15:12:21.113046 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.113029    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 15:12:21.113137 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.113120    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ztgq7\""
Jul 29 15:12:21.122305 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.122286    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fsfps"]
Jul 29 15:12:21.153984 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.153958    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:21.154155 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.153991    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7rrv\" (UniqueName: \"kubernetes.io/projected/3ce93a47-6362-4a83-b069-a12163c25159-kube-api-access-f7rrv\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:21.154155 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.154020    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:21.154155 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.154067    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ce93a47-6362-4a83-b069-a12163c25159-data-volume\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:21.154303 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.154204    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ce93a47-6362-4a83-b069-a12163c25159-crio-socket\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:21.255528 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.255491    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ce93a47-6362-4a83-b069-a12163c25159-crio-socket\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:21.255709 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.255536    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:21.255709 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:21.255616    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 15:12:21.255709 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.255617    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ce93a47-6362-4a83-b069-a12163c25159-crio-socket\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:21.255709 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.255643    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7rrv\" (UniqueName: \"kubernetes.io/projected/3ce93a47-6362-4a83-b069-a12163c25159-kube-api-access-f7rrv\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:21.255709 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:21.255682    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm podName:3ce93a47-6362-4a83-b069-a12163c25159 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:21.75566293 +0000 UTC m=+74.904956821 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fsfps" (UID: "3ce93a47-6362-4a83-b069-a12163c25159") : configmap "kube-rbac-proxy" not found
Jul 29 15:12:21.255965 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.255720    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:21.255965 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.255753    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ce93a47-6362-4a83-b069-a12163c25159-data-volume\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:21.255965 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:21.255843    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:21.255965 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:21.255890    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls podName:3ce93a47-6362-4a83-b069-a12163c25159 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:21.755874295 +0000 UTC m=+74.905168179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fsfps" (UID: "3ce93a47-6362-4a83-b069-a12163c25159") : secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:21.256163 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.256114    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ce93a47-6362-4a83-b069-a12163c25159-data-volume\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:21.267415 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.267385    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7rrv\" (UniqueName: \"kubernetes.io/projected/3ce93a47-6362-4a83-b069-a12163c25159-kube-api-access-f7rrv\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:21.631407 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.631382    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/1.log"
Jul 29 15:12:21.631837 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.631804    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/0.log"
Jul 29 15:12:21.631905 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.631846    2575 generic.go:356] "Generic (PLEG): container finished" podID="f268c925-4fbb-42bd-9d51-b2c74d55c34e" containerID="45892dd45b1c0a19b208dab79d9b0f518b4b3f01255c6f81947acb6e9e0b1607" exitCode=255
Jul 29 15:12:21.631961 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.631912    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg" event={"ID":"f268c925-4fbb-42bd-9d51-b2c74d55c34e","Type":"ContainerDied","Data":"45892dd45b1c0a19b208dab79d9b0f518b4b3f01255c6f81947acb6e9e0b1607"}
Jul 29 15:12:21.632016 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.631975    2575 scope.go:117] "RemoveContainer" containerID="1d2d7a53aa5702552affd10516cbc3b3aa75ff73f4fdca7e4246aef7038fc6d2"
Jul 29 15:12:21.632243 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.632229    2575 scope.go:117] "RemoveContainer" containerID="45892dd45b1c0a19b208dab79d9b0f518b4b3f01255c6f81947acb6e9e0b1607"
Jul 29 15:12:21.632457 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:21.632433    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-ztzjg_openshift-console-operator(f268c925-4fbb-42bd-9d51-b2c74d55c34e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg" podUID="f268c925-4fbb-42bd-9d51-b2c74d55c34e"
Jul 29 15:12:21.758962 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.758912    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:21.758962 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.758968    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:21.759174 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:21.759036    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 15:12:21.759174 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:21.759062    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:21.759174 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:21.759109    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm podName:3ce93a47-6362-4a83-b069-a12163c25159 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:22.759092972 +0000 UTC m=+75.908386851 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fsfps" (UID: "3ce93a47-6362-4a83-b069-a12163c25159") : configmap "kube-rbac-proxy" not found
Jul 29 15:12:21.759174 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:21.759122    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls podName:3ce93a47-6362-4a83-b069-a12163c25159 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:22.759116352 +0000 UTC m=+75.908410230 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fsfps" (UID: "3ce93a47-6362-4a83-b069-a12163c25159") : secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:21.960640 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:21.960549    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:21.960798 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:21.960691    2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 15:12:21.960798 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:21.960708    2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8649b6ff55-9t5ss: secret "image-registry-tls" not found
Jul 29 15:12:21.960798 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:21.960760    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls podName:a1b70bba-d880-4c9d-94fc-a78c3b95302e nodeName:}" failed. No retries permitted until 2026-07-29 15:12:29.960744922 +0000 UTC m=+83.110038801 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls") pod "image-registry-8649b6ff55-9t5ss" (UID: "a1b70bba-d880-4c9d-94fc-a78c3b95302e") : secret "image-registry-tls" not found
Jul 29 15:12:22.636095 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:22.636071    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/1.log"
Jul 29 15:12:22.636580 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:22.636492    2575 scope.go:117] "RemoveContainer" containerID="45892dd45b1c0a19b208dab79d9b0f518b4b3f01255c6f81947acb6e9e0b1607"
Jul 29 15:12:22.636702 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:22.636678    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-ztzjg_openshift-console-operator(f268c925-4fbb-42bd-9d51-b2c74d55c34e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg" podUID="f268c925-4fbb-42bd-9d51-b2c74d55c34e"
Jul 29 15:12:22.765368 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:22.765303    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:22.765537 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:22.765446    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:22.765537 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:22.765471    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:22.765537 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:22.765511    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls podName:3ce93a47-6362-4a83-b069-a12163c25159 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:24.765493456 +0000 UTC m=+77.914787334 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fsfps" (UID: "3ce93a47-6362-4a83-b069-a12163c25159") : secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:22.765677 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:22.765540    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 15:12:22.765677 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:22.765597    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm podName:3ce93a47-6362-4a83-b069-a12163c25159 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:24.765580509 +0000 UTC m=+77.914874391 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fsfps" (UID: "3ce93a47-6362-4a83-b069-a12163c25159") : configmap "kube-rbac-proxy" not found
Jul 29 15:12:24.079490 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:24.079460    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8xf5b_68f9c6d7-78ed-4086-8e7f-9244b109c1d7/dns-node-resolver/0.log"
Jul 29 15:12:24.475878 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:24.475849    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:24.476031 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:24.475928    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:24.476242 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:24.476229    2575 scope.go:117] "RemoveContainer" containerID="45892dd45b1c0a19b208dab79d9b0f518b4b3f01255c6f81947acb6e9e0b1607"
Jul 29 15:12:24.476423 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:24.476394    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-ztzjg_openshift-console-operator(f268c925-4fbb-42bd-9d51-b2c74d55c34e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg" podUID="f268c925-4fbb-42bd-9d51-b2c74d55c34e"
Jul 29 15:12:24.779144 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:24.779051    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:24.779144 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:24.779112    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:24.779337 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:24.779179    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 15:12:24.779337 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:24.779236    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:24.779337 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:24.779253    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm podName:3ce93a47-6362-4a83-b069-a12163c25159 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:28.779233737 +0000 UTC m=+81.928527632 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fsfps" (UID: "3ce93a47-6362-4a83-b069-a12163c25159") : configmap "kube-rbac-proxy" not found
Jul 29 15:12:24.779337 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:24.779272    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls podName:3ce93a47-6362-4a83-b069-a12163c25159 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:28.779262404 +0000 UTC m=+81.928556282 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fsfps" (UID: "3ce93a47-6362-4a83-b069-a12163c25159") : secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:25.079421 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:25.079349    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m6bcg_52c1f3d7-1c12-497f-82ed-3d5439d7dfbc/node-ca/0.log"
Jul 29 15:12:25.884686 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:25.884657    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tps4w_4a897ee8-304b-48a3-8afe-c09656ab3028/serve-healthcheck-canary/0.log"
Jul 29 15:12:28.813075 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:28.812962    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:28.813075 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:28.813044    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:28.813502 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:28.813108    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:28.813502 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:28.813176    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls podName:3ce93a47-6362-4a83-b069-a12163c25159 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:36.813158523 +0000 UTC m=+89.962452401 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fsfps" (UID: "3ce93a47-6362-4a83-b069-a12163c25159") : secret "insights-runtime-extractor-tls" not found
Jul 29 15:12:28.813502 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:28.813112    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 15:12:28.813502 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:28.813264    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm podName:3ce93a47-6362-4a83-b069-a12163c25159 nodeName:}" failed. No retries permitted until 2026-07-29 15:12:36.81324898 +0000 UTC m=+89.962542871 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fsfps" (UID: "3ce93a47-6362-4a83-b069-a12163c25159") : configmap "kube-rbac-proxy" not found
Jul 29 15:12:30.022522 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:30.022486    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:30.024765 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:30.024746    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls\") pod \"image-registry-8649b6ff55-9t5ss\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") " pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:30.145778 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:30.145743    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:30.273582 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:30.273509    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8649b6ff55-9t5ss"]
Jul 29 15:12:30.277676 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:12:30.277653    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1b70bba_d880_4c9d_94fc_a78c3b95302e.slice/crio-81d14ec35cc2c60b8966acd28012cc9ce637eca6d5f7fc03a339895dd6d45adb WatchSource:0}: Error finding container 81d14ec35cc2c60b8966acd28012cc9ce637eca6d5f7fc03a339895dd6d45adb: Status 404 returned error can't find the container with id 81d14ec35cc2c60b8966acd28012cc9ce637eca6d5f7fc03a339895dd6d45adb
Jul 29 15:12:30.660535 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:30.660498    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss" event={"ID":"a1b70bba-d880-4c9d-94fc-a78c3b95302e","Type":"ContainerStarted","Data":"c0c6a1d8358de0dec3f272b338b0fb9f50001db2229a83203a003bf6c743ab86"}
Jul 29 15:12:30.660674 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:30.660557    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss" event={"ID":"a1b70bba-d880-4c9d-94fc-a78c3b95302e","Type":"ContainerStarted","Data":"81d14ec35cc2c60b8966acd28012cc9ce637eca6d5f7fc03a339895dd6d45adb"}
Jul 29 15:12:30.660674 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:30.660628    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:12:30.682146 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:30.682071    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss" podStartSLOduration=16.682052344 podStartE2EDuration="16.682052344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:12:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:30.681418042 +0000 UTC m=+83.830711942" watchObservedRunningTime="2026-07-29 15:12:30.682052344 +0000 UTC m=+83.831346246"
Jul 29 15:12:36.872353 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:36.872305    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:36.872723 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:36.872395    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:36.872855 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:36.872838    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ce93a47-6362-4a83-b069-a12163c25159-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:36.874501 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:36.874472    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ce93a47-6362-4a83-b069-a12163c25159-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fsfps\" (UID: \"3ce93a47-6362-4a83-b069-a12163c25159\") " pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:37.018438 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:37.018387    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fsfps"
Jul 29 15:12:37.147312 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:37.147245    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fsfps"]
Jul 29 15:12:37.150583 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:12:37.150562    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ce93a47_6362_4a83_b069_a12163c25159.slice/crio-c4eaf7f8dd6db234f23e359a1f41247d802ad23d2df925120b9e73276133780e WatchSource:0}: Error finding container c4eaf7f8dd6db234f23e359a1f41247d802ad23d2df925120b9e73276133780e: Status 404 returned error can't find the container with id c4eaf7f8dd6db234f23e359a1f41247d802ad23d2df925120b9e73276133780e
Jul 29 15:12:37.685950 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:37.685909    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fsfps" event={"ID":"3ce93a47-6362-4a83-b069-a12163c25159","Type":"ContainerStarted","Data":"c4eaf7f8dd6db234f23e359a1f41247d802ad23d2df925120b9e73276133780e"}
Jul 29 15:12:39.289744 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:39.289705    2575 scope.go:117] "RemoveContainer" containerID="45892dd45b1c0a19b208dab79d9b0f518b4b3f01255c6f81947acb6e9e0b1607"
Jul 29 15:12:39.695270 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:39.695237    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:12:39.695739 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:39.695714    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/1.log"
Jul 29 15:12:39.695857 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:39.695754    2575 generic.go:356] "Generic (PLEG): container finished" podID="f268c925-4fbb-42bd-9d51-b2c74d55c34e" containerID="c41dde5a575a219e6f72f8fc6328af1c462b4ff96a8b0f0f61ae6f373fcd336f" exitCode=255
Jul 29 15:12:39.695857 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:39.695803    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg" event={"ID":"f268c925-4fbb-42bd-9d51-b2c74d55c34e","Type":"ContainerDied","Data":"c41dde5a575a219e6f72f8fc6328af1c462b4ff96a8b0f0f61ae6f373fcd336f"}
Jul 29 15:12:39.695954 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:39.695856    2575 scope.go:117] "RemoveContainer" containerID="45892dd45b1c0a19b208dab79d9b0f518b4b3f01255c6f81947acb6e9e0b1607"
Jul 29 15:12:39.696299 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:39.696276    2575 scope.go:117] "RemoveContainer" containerID="c41dde5a575a219e6f72f8fc6328af1c462b4ff96a8b0f0f61ae6f373fcd336f"
Jul 29 15:12:39.696980 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:39.696493    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-ztzjg_openshift-console-operator(f268c925-4fbb-42bd-9d51-b2c74d55c34e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg" podUID="f268c925-4fbb-42bd-9d51-b2c74d55c34e"
Jul 29 15:12:40.702071 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:40.702041    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:12:44.357763 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.357736    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"]
Jul 29 15:12:44.360853 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.360833    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.362936 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.362911    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 29 15:12:44.363364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.363347    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 29 15:12:44.363618 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.363600    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 29 15:12:44.364125 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.364111    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 29 15:12:44.364230 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.364216    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 29 15:12:44.364626 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.364608    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 29 15:12:44.365060 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.365044    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 29 15:12:44.383467 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.383444    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"]
Jul 29 15:12:44.432643 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.432613    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8649b6ff55-9t5ss"]
Jul 29 15:12:44.434516 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.434495    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx5zw\" (UniqueName: \"kubernetes.io/projected/74ff5bf7-87c2-4d82-8077-32f30abff37c-kube-api-access-zx5zw\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.434610 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.434550    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/74ff5bf7-87c2-4d82-8077-32f30abff37c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.434610 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.434579    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ff5bf7-87c2-4d82-8077-32f30abff37c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.434730 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.434620    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/74ff5bf7-87c2-4d82-8077-32f30abff37c-ca\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.434730 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.434650    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/74ff5bf7-87c2-4d82-8077-32f30abff37c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.434730 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.434677    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/74ff5bf7-87c2-4d82-8077-32f30abff37c-hub\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.476545 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.475817    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:44.476545 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.476142    2575 scope.go:117] "RemoveContainer" containerID="c41dde5a575a219e6f72f8fc6328af1c462b4ff96a8b0f0f61ae6f373fcd336f"
Jul 29 15:12:44.476545 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:44.476294    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-ztzjg_openshift-console-operator(f268c925-4fbb-42bd-9d51-b2c74d55c34e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg" podUID="f268c925-4fbb-42bd-9d51-b2c74d55c34e"
Jul 29 15:12:44.476545 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.476496    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:12:44.535671 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.535337    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx5zw\" (UniqueName: \"kubernetes.io/projected/74ff5bf7-87c2-4d82-8077-32f30abff37c-kube-api-access-zx5zw\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.535671 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.535421    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/74ff5bf7-87c2-4d82-8077-32f30abff37c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.535671 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.535475    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ff5bf7-87c2-4d82-8077-32f30abff37c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.535671 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.535514    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/74ff5bf7-87c2-4d82-8077-32f30abff37c-ca\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.535671 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.535548    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/74ff5bf7-87c2-4d82-8077-32f30abff37c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.535671 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.535577    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/74ff5bf7-87c2-4d82-8077-32f30abff37c-hub\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.537693 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.537641    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/74ff5bf7-87c2-4d82-8077-32f30abff37c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.538997 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.538971    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/74ff5bf7-87c2-4d82-8077-32f30abff37c-hub\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.539676 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.539650    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/74ff5bf7-87c2-4d82-8077-32f30abff37c-ca\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.540296 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.540275    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/74ff5bf7-87c2-4d82-8077-32f30abff37c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.541342 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.541308    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/74ff5bf7-87c2-4d82-8077-32f30abff37c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.544129 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.544105    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx5zw\" (UniqueName: \"kubernetes.io/projected/74ff5bf7-87c2-4d82-8077-32f30abff37c-kube-api-access-zx5zw\") pod \"cluster-proxy-proxy-agent-f6cbdc869-qzq4g\" (UID: \"74ff5bf7-87c2-4d82-8077-32f30abff37c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.678154 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.678077    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"
Jul 29 15:12:44.719830 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.719794    2575 scope.go:117] "RemoveContainer" containerID="c41dde5a575a219e6f72f8fc6328af1c462b4ff96a8b0f0f61ae6f373fcd336f"
Jul 29 15:12:44.720388 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:12:44.719993    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-ztzjg_openshift-console-operator(f268c925-4fbb-42bd-9d51-b2c74d55c34e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg" podUID="f268c925-4fbb-42bd-9d51-b2c74d55c34e"
Jul 29 15:12:44.720388 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.719792    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fsfps" event={"ID":"3ce93a47-6362-4a83-b069-a12163c25159","Type":"ContainerStarted","Data":"282182bebb3c8db6d39c9b2e12246677ee36af397c67c58c671a4b09bda0c2ad"}
Jul 29 15:12:44.833548 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:44.833516    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g"]
Jul 29 15:12:44.838035 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:12:44.838008    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74ff5bf7_87c2_4d82_8077_32f30abff37c.slice/crio-d0905015cf307c39b18b1dcd10e369fd46319be0769ddd71dc1f253df7a93073 WatchSource:0}: Error finding container d0905015cf307c39b18b1dcd10e369fd46319be0769ddd71dc1f253df7a93073: Status 404 returned error can't find the container with id d0905015cf307c39b18b1dcd10e369fd46319be0769ddd71dc1f253df7a93073
Jul 29 15:12:45.646657 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:45.646621    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:12:45.649230 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:45.649206    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e-metrics-tls\") pod \"dns-default-zvjbc\" (UID: \"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e\") " pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:12:45.724674 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:45.724635    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g" event={"ID":"74ff5bf7-87c2-4d82-8077-32f30abff37c","Type":"ContainerStarted","Data":"d0905015cf307c39b18b1dcd10e369fd46319be0769ddd71dc1f253df7a93073"}
Jul 29 15:12:45.930924 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:45.930844    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dw7hg\""
Jul 29 15:12:45.938660 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:45.938624    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:12:49.834217 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:49.834187    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zvjbc"]
Jul 29 15:12:49.836676 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:12:49.836653    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c9df7c8_57b4_4ec0_b1fb_85149f5ec60e.slice/crio-956e99fb1b216b2923a3216e84be8cbf74c72c646215f540e38a83a7a8276976 WatchSource:0}: Error finding container 956e99fb1b216b2923a3216e84be8cbf74c72c646215f540e38a83a7a8276976: Status 404 returned error can't find the container with id 956e99fb1b216b2923a3216e84be8cbf74c72c646215f540e38a83a7a8276976
Jul 29 15:12:50.629301 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:50.629273    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bs7zd"
Jul 29 15:12:50.751529 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:50.751487    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g" event={"ID":"74ff5bf7-87c2-4d82-8077-32f30abff37c","Type":"ContainerStarted","Data":"85081d636033164f9bf389220176d4d32b7bfa2d7f167d4fd10445b93dbad36b"}
Jul 29 15:12:50.751696 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:50.751541    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g" event={"ID":"74ff5bf7-87c2-4d82-8077-32f30abff37c","Type":"ContainerStarted","Data":"06c436b55a2038938c577dc59404225550a24f538ab2a9a64d569d2da2a4c083"}
Jul 29 15:12:50.751696 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:50.751563    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g" event={"ID":"74ff5bf7-87c2-4d82-8077-32f30abff37c","Type":"ContainerStarted","Data":"e1f48c5ff1145492b152558d45fd62a8bc9d06d6054989f23e9217e6115d27d1"}
Jul 29 15:12:50.753591 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:50.753540    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fsfps" event={"ID":"3ce93a47-6362-4a83-b069-a12163c25159","Type":"ContainerStarted","Data":"c042d065aacb2b3e092b129c6a3fac07790a1a6db1244f7430aba28f8219e37b"}
Jul 29 15:12:50.754890 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:50.754854    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zvjbc" event={"ID":"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e","Type":"ContainerStarted","Data":"956e99fb1b216b2923a3216e84be8cbf74c72c646215f540e38a83a7a8276976"}
Jul 29 15:12:50.776636 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:50.776584    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6cbdc869-qzq4g" podStartSLOduration=1.58722491 podStartE2EDuration="6.77656683s" totalImagesPullingTime="5.18934192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:12:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:50.774517919 +0000 UTC m=+103.923811819" watchObservedRunningTime="2026-07-29 15:12:50.77656683 +0000 UTC m=+103.925860730"
Jul 29 15:12:52.763171 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:52.763137    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zvjbc" event={"ID":"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e","Type":"ContainerStarted","Data":"648414bb124bfdc216bd6cf1e792624495dad5069ddecdde161ed281e831b8f7"}
Jul 29 15:12:52.763624 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:52.763188    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zvjbc" event={"ID":"4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e","Type":"ContainerStarted","Data":"3212a4953d49bc4da0b32fe1e10dc790a3834d604b94d18872d2ea78f9f3e424"}
Jul 29 15:12:52.763624 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:52.763268    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:12:52.790526 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:52.790467    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zvjbc" podStartSLOduration=69.722763846 podStartE2EDuration="1m11.790310137s" totalImagesPullingTime="2.067546291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:11:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:52.789088602 +0000 UTC m=+105.938382697" watchObservedRunningTime="2026-07-29 15:12:52.790310137 +0000 UTC m=+105.939604038"
Jul 29 15:12:54.438741 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:54.438705    2575 patch_prober.go:28] interesting pod/image-registry-8649b6ff55-9t5ss container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 29 15:12:54.439136 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:54.438764    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss" podUID="a1b70bba-d880-4c9d-94fc-a78c3b95302e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 15:12:55.775452 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:55.775418    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fsfps" event={"ID":"3ce93a47-6362-4a83-b069-a12163c25159","Type":"ContainerStarted","Data":"6f2f696279f700181368cd2c5395a40073b115cc08ff4650a73be0d61f738583"}
Jul 29 15:12:55.799879 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:12:55.799828    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fsfps" podStartSLOduration=16.987349348 podStartE2EDuration="34.799809175s" totalImagesPullingTime="17.812459827s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 15:12:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:12:55.799517537 +0000 UTC m=+108.948811436" watchObservedRunningTime="2026-07-29 15:12:55.799809175 +0000 UTC m=+108.949103079"
Jul 29 15:13:00.289594 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:00.289564    2575 scope.go:117] "RemoveContainer" containerID="c41dde5a575a219e6f72f8fc6328af1c462b4ff96a8b0f0f61ae6f373fcd336f"
Jul 29 15:13:00.791767 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:00.791736    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:13:00.791963 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:00.791824    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg" event={"ID":"f268c925-4fbb-42bd-9d51-b2c74d55c34e","Type":"ContainerStarted","Data":"708ceb87acc7b5fa61401e1ee6d3bc6ebbc4a669f1977b99f62555c9868263b4"}
Jul 29 15:13:00.792135 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:00.792113    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:13:00.809938 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:00.809887    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg" podStartSLOduration=41.705613684 podStartE2EDuration="46.809868448s" totalImagesPullingTime="5.104254764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:12:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:13:00.809409996 +0000 UTC m=+113.958703896" watchObservedRunningTime="2026-07-29 15:13:00.809868448 +0000 UTC m=+113.959162349"
Jul 29 15:13:01.200233 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:01.200201    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-ztzjg"
Jul 29 15:13:02.769310 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:02.769280    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zvjbc"
Jul 29 15:13:04.437512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.437484    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:13:04.483870 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.483844    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kfrcd"]
Jul 29 15:13:04.486886 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.486864    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.489681 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.489662    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 15:13:04.489899 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.489878    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rlsh5\""
Jul 29 15:13:04.490012 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.489906    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 15:13:04.490012 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.489886    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 15:13:04.490111 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.490061    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 15:13:04.490162 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.490121    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 15:13:04.490650 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.490631    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 15:13:04.596983 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.596949    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t9v4\" (UniqueName: \"kubernetes.io/projected/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-kube-api-access-4t9v4\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.596983 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.596981    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-metrics-client-ca\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.597144 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.597009    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-node-exporter-accelerators-collector-config\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.597144 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.597128    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-root\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.597217 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.597163    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-node-exporter-wtmp\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.597217 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.597209    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-node-exporter-textfile\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.597283 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.597231    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.597283 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.597249    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-node-exporter-tls\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.597283 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.597266    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-sys\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.698575 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.698296    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-node-exporter-accelerators-collector-config\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.698575 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.698461    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-root\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.698575 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.698493    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-node-exporter-wtmp\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.698575 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.698528    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-node-exporter-textfile\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.698575 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.698563    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-root\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.698941 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.698565    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.698941 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.698636    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-node-exporter-wtmp\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.698941 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.698677    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-node-exporter-tls\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.698941 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.698705    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-sys\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.698941 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.698769    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t9v4\" (UniqueName: \"kubernetes.io/projected/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-kube-api-access-4t9v4\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.698941 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.698796    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-metrics-client-ca\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.698941 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.698866    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-node-exporter-textfile\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.698941 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.698915    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-sys\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.699301 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.698961    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-node-exporter-accelerators-collector-config\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.699301 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.699202    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-metrics-client-ca\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.700962 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.700940    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.701020 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.700955    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-node-exporter-tls\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.715261 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.715239    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t9v4\" (UniqueName: \"kubernetes.io/projected/5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e-kube-api-access-4t9v4\") pod \"node-exporter-kfrcd\" (UID: \"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e\") " pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.796307 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:04.796284    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kfrcd"
Jul 29 15:13:04.808530 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:13:04.808510    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d01ae0c_9529_4b6c_9f7d_0ab0d3e7507e.slice/crio-6c0126f2b2f2baf25847043ded13b6e8cc85e89452263af619588944829a658c WatchSource:0}: Error finding container 6c0126f2b2f2baf25847043ded13b6e8cc85e89452263af619588944829a658c: Status 404 returned error can't find the container with id 6c0126f2b2f2baf25847043ded13b6e8cc85e89452263af619588944829a658c
Jul 29 15:13:05.515595 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.515565    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 15:13:05.519467 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.519447    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.521780 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.521750    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 15:13:05.522036 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.522018    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 15:13:05.522112 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.522036    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 15:13:05.522112 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.522025    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 15:13:05.522112 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.522020    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 15:13:05.522383 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.522350    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 15:13:05.522463 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.522388    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 15:13:05.522838 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.522819    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 15:13:05.522942 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.522926    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 15:13:05.523036 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.523017    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-ts2w5\""
Jul 29 15:13:05.536496 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.536476    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 15:13:05.605407 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.605376    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-web-config\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.605562 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.605416    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.605562 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.605445    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afe522ca-f2b0-4988-9986-ba0fd523b84f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.605562 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.605471    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp25c\" (UniqueName: \"kubernetes.io/projected/afe522ca-f2b0-4988-9986-ba0fd523b84f-kube-api-access-rp25c\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.605562 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.605493    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-config-volume\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.605562 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.605514    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.605562 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.605541    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.605834 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.605629    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/afe522ca-f2b0-4988-9986-ba0fd523b84f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.605834 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.605660    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/afe522ca-f2b0-4988-9986-ba0fd523b84f-config-out\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.605834 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.605699    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.605834 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.605719    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/afe522ca-f2b0-4988-9986-ba0fd523b84f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.605834 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.605759    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.605834 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.605796    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afe522ca-f2b0-4988-9986-ba0fd523b84f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.706342 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.706293    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.706481 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.706355    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afe522ca-f2b0-4988-9986-ba0fd523b84f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.706481 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.706394    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-web-config\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.706481 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.706429    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.706481 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.706459    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afe522ca-f2b0-4988-9986-ba0fd523b84f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.706699 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.706483    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp25c\" (UniqueName: \"kubernetes.io/projected/afe522ca-f2b0-4988-9986-ba0fd523b84f-kube-api-access-rp25c\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.706699 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.706509    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-config-volume\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.706699 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.706540    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.706699 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.706578    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.706699 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:13:05.706599    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afe522ca-f2b0-4988-9986-ba0fd523b84f-alertmanager-trusted-ca-bundle podName:afe522ca-f2b0-4988-9986-ba0fd523b84f nodeName:}" failed. No retries permitted until 2026-07-29 15:13:06.206575089 +0000 UTC m=+119.355868972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/afe522ca-f2b0-4988-9986-ba0fd523b84f-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "afe522ca-f2b0-4988-9986-ba0fd523b84f") : configmap references non-existent config key: ca-bundle.crt
Jul 29 15:13:05.706699 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.706641    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/afe522ca-f2b0-4988-9986-ba0fd523b84f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.706699 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.706678    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/afe522ca-f2b0-4988-9986-ba0fd523b84f-config-out\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.707054 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.706742    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.707054 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.706766    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/afe522ca-f2b0-4988-9986-ba0fd523b84f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.707304 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.707271    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/afe522ca-f2b0-4988-9986-ba0fd523b84f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.707657 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:13:05.707539    2575 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 29 15:13:05.707657 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:13:05.707606    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-main-tls podName:afe522ca-f2b0-4988-9986-ba0fd523b84f nodeName:}" failed. No retries permitted until 2026-07-29 15:13:06.20758316 +0000 UTC m=+119.356877039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "afe522ca-f2b0-4988-9986-ba0fd523b84f") : secret "alertmanager-main-tls" not found
Jul 29 15:13:05.708381 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.708354    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afe522ca-f2b0-4988-9986-ba0fd523b84f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.709678 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.709636    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.709777 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.709689    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-web-config\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.709838 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.709826    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.709896 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.709827    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.710212 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.710188    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/afe522ca-f2b0-4988-9986-ba0fd523b84f-config-out\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.710361 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.710343    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/afe522ca-f2b0-4988-9986-ba0fd523b84f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.710783 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.710758    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-config-volume\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.711083 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.711064    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.717301 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.717278    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp25c\" (UniqueName: \"kubernetes.io/projected/afe522ca-f2b0-4988-9986-ba0fd523b84f-kube-api-access-rp25c\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:05.809257 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:05.809199    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kfrcd" event={"ID":"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e","Type":"ContainerStarted","Data":"6c0126f2b2f2baf25847043ded13b6e8cc85e89452263af619588944829a658c"}
Jul 29 15:13:06.212334 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:06.212292    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afe522ca-f2b0-4988-9986-ba0fd523b84f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:06.212482 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:06.212360    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:06.213027 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:06.212994    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afe522ca-f2b0-4988-9986-ba0fd523b84f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:06.214660 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:06.214637    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:06.428604 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:06.428572    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:13:06.585204 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:06.585156    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 15:13:06.587517 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:13:06.587495    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafe522ca_f2b0_4988_9986_ba0fd523b84f.slice/crio-11434e89b6134435c67066f3dd1310d3cd034d7a6f88a70d32dca1d59b16ba33 WatchSource:0}: Error finding container 11434e89b6134435c67066f3dd1310d3cd034d7a6f88a70d32dca1d59b16ba33: Status 404 returned error can't find the container with id 11434e89b6134435c67066f3dd1310d3cd034d7a6f88a70d32dca1d59b16ba33
Jul 29 15:13:06.813690 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:06.813601    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afe522ca-f2b0-4988-9986-ba0fd523b84f","Type":"ContainerStarted","Data":"11434e89b6134435c67066f3dd1310d3cd034d7a6f88a70d32dca1d59b16ba33"}
Jul 29 15:13:07.818170 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:07.818080    2575 generic.go:356] "Generic (PLEG): container finished" podID="5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e" containerID="62018e9672dbabcbfdae386f0538ce59b7b6ecbf0dea07aba47891f414889345" exitCode=0
Jul 29 15:13:07.818170 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:07.818149    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kfrcd" event={"ID":"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e","Type":"ContainerDied","Data":"62018e9672dbabcbfdae386f0538ce59b7b6ecbf0dea07aba47891f414889345"}
Jul 29 15:13:08.823012 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:08.822981    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kfrcd" event={"ID":"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e","Type":"ContainerStarted","Data":"2658294bba8c56bf005c131bdef019fc0bd23aa6d38afc6322b7f7fadb5b5bf2"}
Jul 29 15:13:08.823437 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:08.823031    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kfrcd" event={"ID":"5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e","Type":"ContainerStarted","Data":"40acd405dabdf3f5f423af3a2275264ba8070259e9b862542cf05cb34082ab25"}
Jul 29 15:13:08.849436 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:08.849383    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kfrcd" podStartSLOduration=2.148573197 podStartE2EDuration="4.849366779s" totalImagesPullingTime="2.639265996s" totalInitContainerRuntime="61.527586ms" isStatefulPod=true podCreationTimestamp="2026-07-29 15:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:13:08.848958064 +0000 UTC m=+121.998251965" watchObservedRunningTime="2026-07-29 15:13:08.849366779 +0000 UTC m=+121.998660679"
Jul 29 15:13:09.250049 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.250008    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-4kcv6"]
Jul 29 15:13:09.253418 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.253396    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4kcv6"
Jul 29 15:13:09.257697 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.257677    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 29 15:13:09.257964 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.257945    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-wrpr4\""
Jul 29 15:13:09.264552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.264526    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-4kcv6"]
Jul 29 15:13:09.438260 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.438233    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97sgd\" (UniqueName: \"kubernetes.io/projected/640f7311-506d-4d5c-bb8c-59facc070d1a-kube-api-access-97sgd\") pod \"monitoring-plugin-68b7fc9574-4kcv6\" (UID: \"640f7311-506d-4d5c-bb8c-59facc070d1a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4kcv6"
Jul 29 15:13:09.438456 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.438347    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/640f7311-506d-4d5c-bb8c-59facc070d1a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-4kcv6\" (UID: \"640f7311-506d-4d5c-bb8c-59facc070d1a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4kcv6"
Jul 29 15:13:09.459502 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.459473    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss" podUID="a1b70bba-d880-4c9d-94fc-a78c3b95302e" containerName="registry" containerID="cri-o://c0c6a1d8358de0dec3f272b338b0fb9f50001db2229a83203a003bf6c743ab86" gracePeriod=30
Jul 29 15:13:09.539183 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.539159    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97sgd\" (UniqueName: \"kubernetes.io/projected/640f7311-506d-4d5c-bb8c-59facc070d1a-kube-api-access-97sgd\") pod \"monitoring-plugin-68b7fc9574-4kcv6\" (UID: \"640f7311-506d-4d5c-bb8c-59facc070d1a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4kcv6"
Jul 29 15:13:09.539293 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.539237    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/640f7311-506d-4d5c-bb8c-59facc070d1a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-4kcv6\" (UID: \"640f7311-506d-4d5c-bb8c-59facc070d1a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4kcv6"
Jul 29 15:13:09.539374 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:13:09.539359    2575 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 29 15:13:09.539433 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:13:09.539422    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/640f7311-506d-4d5c-bb8c-59facc070d1a-monitoring-plugin-cert podName:640f7311-506d-4d5c-bb8c-59facc070d1a nodeName:}" failed. No retries permitted until 2026-07-29 15:13:10.03940536 +0000 UTC m=+123.188699242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/640f7311-506d-4d5c-bb8c-59facc070d1a-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-4kcv6" (UID: "640f7311-506d-4d5c-bb8c-59facc070d1a") : secret "monitoring-plugin-cert" not found
Jul 29 15:13:09.551830 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.551806    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97sgd\" (UniqueName: \"kubernetes.io/projected/640f7311-506d-4d5c-bb8c-59facc070d1a-kube-api-access-97sgd\") pod \"monitoring-plugin-68b7fc9574-4kcv6\" (UID: \"640f7311-506d-4d5c-bb8c-59facc070d1a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4kcv6"
Jul 29 15:13:09.685877 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.685852    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:13:09.827469 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.827388    2575 generic.go:356] "Generic (PLEG): container finished" podID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerID="51b19e52b616256e5021412bb126b924ad6db1a5608684da7689ed3ccccdf26e" exitCode=0
Jul 29 15:13:09.827469 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.827454    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afe522ca-f2b0-4988-9986-ba0fd523b84f","Type":"ContainerDied","Data":"51b19e52b616256e5021412bb126b924ad6db1a5608684da7689ed3ccccdf26e"}
Jul 29 15:13:09.828707 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.828687    2575 generic.go:356] "Generic (PLEG): container finished" podID="a1b70bba-d880-4c9d-94fc-a78c3b95302e" containerID="c0c6a1d8358de0dec3f272b338b0fb9f50001db2229a83203a003bf6c743ab86" exitCode=0
Jul 29 15:13:09.828775 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.828755    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss" event={"ID":"a1b70bba-d880-4c9d-94fc-a78c3b95302e","Type":"ContainerDied","Data":"c0c6a1d8358de0dec3f272b338b0fb9f50001db2229a83203a003bf6c743ab86"}
Jul 29 15:13:09.828840 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.828773    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss"
Jul 29 15:13:09.828840 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.828803    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8649b6ff55-9t5ss" event={"ID":"a1b70bba-d880-4c9d-94fc-a78c3b95302e","Type":"ContainerDied","Data":"81d14ec35cc2c60b8966acd28012cc9ce637eca6d5f7fc03a339895dd6d45adb"}
Jul 29 15:13:09.828840 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.828831    2575 scope.go:117] "RemoveContainer" containerID="c0c6a1d8358de0dec3f272b338b0fb9f50001db2229a83203a003bf6c743ab86"
Jul 29 15:13:09.836578 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.836551    2575 scope.go:117] "RemoveContainer" containerID="c0c6a1d8358de0dec3f272b338b0fb9f50001db2229a83203a003bf6c743ab86"
Jul 29 15:13:09.836844 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:13:09.836820    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0c6a1d8358de0dec3f272b338b0fb9f50001db2229a83203a003bf6c743ab86\": container with ID starting with c0c6a1d8358de0dec3f272b338b0fb9f50001db2229a83203a003bf6c743ab86 not found: ID does not exist" containerID="c0c6a1d8358de0dec3f272b338b0fb9f50001db2229a83203a003bf6c743ab86"
Jul 29 15:13:09.836895 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.836859    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0c6a1d8358de0dec3f272b338b0fb9f50001db2229a83203a003bf6c743ab86"} err="failed to get container status \"c0c6a1d8358de0dec3f272b338b0fb9f50001db2229a83203a003bf6c743ab86\": rpc error: code = NotFound desc = could not find container \"c0c6a1d8358de0dec3f272b338b0fb9f50001db2229a83203a003bf6c743ab86\": container with ID starting with c0c6a1d8358de0dec3f272b338b0fb9f50001db2229a83203a003bf6c743ab86 not found: ID does not exist"
Jul 29 15:13:09.843122 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.843102    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1b70bba-d880-4c9d-94fc-a78c3b95302e-trusted-ca\") pod \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") "
Jul 29 15:13:09.843230 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.843137    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1b70bba-d880-4c9d-94fc-a78c3b95302e-installation-pull-secrets\") pod \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") "
Jul 29 15:13:09.843230 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.843162    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls\") pod \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") "
Jul 29 15:13:09.843230 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.843197    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1b70bba-d880-4c9d-94fc-a78c3b95302e-image-registry-private-configuration\") pod \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") "
Jul 29 15:13:09.843402 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.843233    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4dh4\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-kube-api-access-l4dh4\") pod \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") "
Jul 29 15:13:09.843402 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.843293    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-bound-sa-token\") pod \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") "
Jul 29 15:13:09.843402 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.843350    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1b70bba-d880-4c9d-94fc-a78c3b95302e-ca-trust-extracted\") pod \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") "
Jul 29 15:13:09.843402 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.843386    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-certificates\") pod \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\" (UID: \"a1b70bba-d880-4c9d-94fc-a78c3b95302e\") "
Jul 29 15:13:09.843996 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.843947    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1b70bba-d880-4c9d-94fc-a78c3b95302e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a1b70bba-d880-4c9d-94fc-a78c3b95302e" (UID: "a1b70bba-d880-4c9d-94fc-a78c3b95302e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:13:09.843996 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.843957    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a1b70bba-d880-4c9d-94fc-a78c3b95302e" (UID: "a1b70bba-d880-4c9d-94fc-a78c3b95302e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:13:09.845768 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.845740    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-kube-api-access-l4dh4" (OuterVolumeSpecName: "kube-api-access-l4dh4") pod "a1b70bba-d880-4c9d-94fc-a78c3b95302e" (UID: "a1b70bba-d880-4c9d-94fc-a78c3b95302e"). InnerVolumeSpecName "kube-api-access-l4dh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:13:09.845925 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.845906    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a1b70bba-d880-4c9d-94fc-a78c3b95302e" (UID: "a1b70bba-d880-4c9d-94fc-a78c3b95302e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:13:09.845980 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.845919    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1b70bba-d880-4c9d-94fc-a78c3b95302e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a1b70bba-d880-4c9d-94fc-a78c3b95302e" (UID: "a1b70bba-d880-4c9d-94fc-a78c3b95302e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:13:09.845980 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.845904    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a1b70bba-d880-4c9d-94fc-a78c3b95302e" (UID: "a1b70bba-d880-4c9d-94fc-a78c3b95302e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:13:09.846040 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.845974    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1b70bba-d880-4c9d-94fc-a78c3b95302e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a1b70bba-d880-4c9d-94fc-a78c3b95302e" (UID: "a1b70bba-d880-4c9d-94fc-a78c3b95302e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:13:09.851533 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.851505    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1b70bba-d880-4c9d-94fc-a78c3b95302e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a1b70bba-d880-4c9d-94fc-a78c3b95302e" (UID: "a1b70bba-d880-4c9d-94fc-a78c3b95302e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:13:09.944484 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.944448    2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1b70bba-d880-4c9d-94fc-a78c3b95302e-trusted-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:13:09.944634 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.944488    2575 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1b70bba-d880-4c9d-94fc-a78c3b95302e-installation-pull-secrets\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:13:09.944634 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.944506    2575 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:13:09.944634 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.944532    2575 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1b70bba-d880-4c9d-94fc-a78c3b95302e-image-registry-private-configuration\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:13:09.944634 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.944548    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4dh4\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-kube-api-access-l4dh4\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:13:09.944634 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.944563    2575 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1b70bba-d880-4c9d-94fc-a78c3b95302e-bound-sa-token\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:13:09.944634 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.944579    2575 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1b70bba-d880-4c9d-94fc-a78c3b95302e-ca-trust-extracted\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:13:09.944634 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:09.944592    2575 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1b70bba-d880-4c9d-94fc-a78c3b95302e-registry-certificates\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:13:10.045231 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.045198    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/640f7311-506d-4d5c-bb8c-59facc070d1a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-4kcv6\" (UID: \"640f7311-506d-4d5c-bb8c-59facc070d1a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4kcv6"
Jul 29 15:13:10.047466 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.047446    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/640f7311-506d-4d5c-bb8c-59facc070d1a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-4kcv6\" (UID: \"640f7311-506d-4d5c-bb8c-59facc070d1a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4kcv6"
Jul 29 15:13:10.154917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.154891    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8649b6ff55-9t5ss"]
Jul 29 15:13:10.160333 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.160303    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8649b6ff55-9t5ss"]
Jul 29 15:13:10.163387 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.163373    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4kcv6"
Jul 29 15:13:10.285868 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.285834    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-4kcv6"]
Jul 29 15:13:10.288582 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:13:10.288558    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod640f7311_506d_4d5c_bb8c_59facc070d1a.slice/crio-71daf92b7d4fcbfcfb24a3569be840ca1ea3e596770bc4a8f2e3f2588ddb7fe7 WatchSource:0}: Error finding container 71daf92b7d4fcbfcfb24a3569be840ca1ea3e596770bc4a8f2e3f2588ddb7fe7: Status 404 returned error can't find the container with id 71daf92b7d4fcbfcfb24a3569be840ca1ea3e596770bc4a8f2e3f2588ddb7fe7
Jul 29 15:13:10.703562 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.703485    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 15:13:10.703857 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.703838    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1b70bba-d880-4c9d-94fc-a78c3b95302e" containerName="registry"
Jul 29 15:13:10.703857 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.703860    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1b70bba-d880-4c9d-94fc-a78c3b95302e" containerName="registry"
Jul 29 15:13:10.704018 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.703907    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1b70bba-d880-4c9d-94fc-a78c3b95302e" containerName="registry"
Jul 29 15:13:10.707333 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.707291    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.710294 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.710186    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 29 15:13:10.710431 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.710313    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 29 15:13:10.710863 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.710843    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 29 15:13:10.711026 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.711013    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 29 15:13:10.711262 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.711223    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5k6nn\""
Jul 29 15:13:10.711338 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.711265    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 29 15:13:10.711338 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.711280    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 29 15:13:10.711338 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.711288    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 29 15:13:10.711666 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.711614    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 29 15:13:10.711905 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.711868    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 29 15:13:10.712132 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.712108    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 29 15:13:10.712534 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.712461    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 29 15:13:10.712534 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.712517    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-27d1ht214qgs7\""
Jul 29 15:13:10.712746 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.712708    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 29 15:13:10.713825 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.713673    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 29 15:13:10.714887 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.714868    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 29 15:13:10.718787 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.718765    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 29 15:13:10.729525 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.729505    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 15:13:10.836169 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.836131    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4kcv6" event={"ID":"640f7311-506d-4d5c-bb8c-59facc070d1a","Type":"ContainerStarted","Data":"71daf92b7d4fcbfcfb24a3569be840ca1ea3e596770bc4a8f2e3f2588ddb7fe7"}
Jul 29 15:13:10.854539 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.854497    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-config\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.854692 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.854545    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.854692 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.854572    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.854692 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.854591    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.854692 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.854610    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcpph\" (UniqueName: \"kubernetes.io/projected/c618a894-ab7d-4d1c-a493-23f63a6cfa55-kube-api-access-mcpph\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.854692 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.854640    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.854692 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.854688    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.854919 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.854752    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.854919 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.854776    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.854919 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.854794    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.854919 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.854822    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c618a894-ab7d-4d1c-a493-23f63a6cfa55-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.854919 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.854839    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.854919 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.854857    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.854919 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.854902    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.855257 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.854949    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.855257 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.854984    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.855257 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.855013    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.855257 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.855032    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-web-config\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.855257 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.855060    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c618a894-ab7d-4d1c-a493-23f63a6cfa55-config-out\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.855257 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.855134    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.955924 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.955840    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c618a894-ab7d-4d1c-a493-23f63a6cfa55-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.955924 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.955880    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.955924 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.955905    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.956186 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.955948    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.956186 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.955984    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.956186 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.956005    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.956186 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.956037    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.956186 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.956064    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-web-config\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.956186 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.956092    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c618a894-ab7d-4d1c-a493-23f63a6cfa55-config-out\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.956186 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.956137    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.956186 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.956169    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-config\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.956588 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.956194    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.956588 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.956234    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.956588 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.956262    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.956588 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.956285    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcpph\" (UniqueName: \"kubernetes.io/projected/c618a894-ab7d-4d1c-a493-23f63a6cfa55-kube-api-access-mcpph\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.956588 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.956314    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.957531 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.957009    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.957531 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.957033    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.957531 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.957250    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.957531 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.957297    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.957531 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.957383    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.957531 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.957414    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.957531 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.957442    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.958912 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.958777    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.958912 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.958855    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.960230 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.959280    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.960230 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.959754    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.960230 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.960190    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.960482 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.960436    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c618a894-ab7d-4d1c-a493-23f63a6cfa55-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.961489 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.961446    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.963170 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.963122    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-config\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.963486 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.963463    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.963747 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.963726    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c618a894-ab7d-4d1c-a493-23f63a6cfa55-config-out\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.964057 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.964033    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-web-config\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.964150 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.964085    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.964215 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.964173    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.964526 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.964503    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.964949 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.964930    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.966132 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.966111    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:10.967704 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:10.967681    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcpph\" (UniqueName: \"kubernetes.io/projected/c618a894-ab7d-4d1c-a493-23f63a6cfa55-kube-api-access-mcpph\") pod \"prometheus-k8s-0\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:11.019384 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:11.019355    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:11.188204 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:11.188169    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 15:13:11.294243 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:11.294168    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1b70bba-d880-4c9d-94fc-a78c3b95302e" path="/var/lib/kubelet/pods/a1b70bba-d880-4c9d-94fc-a78c3b95302e/volumes"
Jul 29 15:13:11.392495 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:13:11.392464    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc618a894_ab7d_4d1c_a493_23f63a6cfa55.slice/crio-8ab56a5e3301c7b9372a53ffdcaf212e38b88777049a62c9f31099a16fa27a75 WatchSource:0}: Error finding container 8ab56a5e3301c7b9372a53ffdcaf212e38b88777049a62c9f31099a16fa27a75: Status 404 returned error can't find the container with id 8ab56a5e3301c7b9372a53ffdcaf212e38b88777049a62c9f31099a16fa27a75
Jul 29 15:13:11.841365 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:11.841314    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c618a894-ab7d-4d1c-a493-23f63a6cfa55","Type":"ContainerStarted","Data":"8ab56a5e3301c7b9372a53ffdcaf212e38b88777049a62c9f31099a16fa27a75"}
Jul 29 15:13:12.849217 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:12.849181    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afe522ca-f2b0-4988-9986-ba0fd523b84f","Type":"ContainerStarted","Data":"527a443b09f840860cdd5f77874ca3ea802cfde1733f8641c5200ab7f24ca8f6"}
Jul 29 15:13:12.849554 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:12.849252    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afe522ca-f2b0-4988-9986-ba0fd523b84f","Type":"ContainerStarted","Data":"e9ff5a7a7c7079801f51d1947bb9c3f9676692f48994453553986f4732734e3c"}
Jul 29 15:13:12.849554 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:12.849293    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afe522ca-f2b0-4988-9986-ba0fd523b84f","Type":"ContainerStarted","Data":"c21ea86d3156f5838d7c71f2f5a684a5e3900404abe91f289385a1db3cc64a57"}
Jul 29 15:13:12.849554 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:12.849354    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afe522ca-f2b0-4988-9986-ba0fd523b84f","Type":"ContainerStarted","Data":"57267101eaa4d38405101575f2c34435bd0ff5c08bd72a512569b1a63b731b99"}
Jul 29 15:13:12.849554 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:12.849381    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afe522ca-f2b0-4988-9986-ba0fd523b84f","Type":"ContainerStarted","Data":"54f85c1860eeb21df1c77ea951fbe13d6257969aa450c4d1f6cc4aa680a6ba15"}
Jul 29 15:13:12.850950 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:12.850930    2575 generic.go:356] "Generic (PLEG): container finished" podID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerID="18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd" exitCode=0
Jul 29 15:13:12.851037 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:12.851014    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c618a894-ab7d-4d1c-a493-23f63a6cfa55","Type":"ContainerDied","Data":"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd"}
Jul 29 15:13:12.852744 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:12.852720    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4kcv6" event={"ID":"640f7311-506d-4d5c-bb8c-59facc070d1a","Type":"ContainerStarted","Data":"c23a5fb49820d335da7893136e1076285a20c9fc299fccb51f8c22eae6acdc80"}
Jul 29 15:13:12.852948 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:12.852930    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4kcv6"
Jul 29 15:13:12.857894 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:12.857875    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4kcv6"
Jul 29 15:13:12.908886 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:12.908840    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-4kcv6" podStartSLOduration=2.433406439 podStartE2EDuration="3.908827599s" totalImagesPullingTime="1.47542116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:13:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:13:12.907585817 +0000 UTC m=+126.056879717" watchObservedRunningTime="2026-07-29 15:13:12.908827599 +0000 UTC m=+126.058121498"
Jul 29 15:13:13.861789 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:13.861749    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afe522ca-f2b0-4988-9986-ba0fd523b84f","Type":"ContainerStarted","Data":"88a5b5030fb75fe575b115df27413e15edcfa62b064d8831a6a65fbf58be89b5"}
Jul 29 15:13:13.893695 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:13.893643    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.414714889 podStartE2EDuration="8.893622599s" totalImagesPullingTime="5.463534142s" totalInitContainerRuntime="15.373568ms" isStatefulPod=false podCreationTimestamp="2026-07-29 15:13:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:13:13.89270986 +0000 UTC m=+127.042003761" watchObservedRunningTime="2026-07-29 15:13:13.893622599 +0000 UTC m=+127.042916500"
Jul 29 15:13:16.876487 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:16.876454    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c618a894-ab7d-4d1c-a493-23f63a6cfa55","Type":"ContainerStarted","Data":"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf"}
Jul 29 15:13:16.876914 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:16.876520    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c618a894-ab7d-4d1c-a493-23f63a6cfa55","Type":"ContainerStarted","Data":"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7"}
Jul 29 15:13:17.115068 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:17.115033    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs\") pod \"network-metrics-daemon-psdvf\" (UID: \"925b0a18-77be-438d-bcfe-27609c648913\") " pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:13:17.117454 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:17.117431    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/925b0a18-77be-438d-bcfe-27609c648913-metrics-certs\") pod \"network-metrics-daemon-psdvf\" (UID: \"925b0a18-77be-438d-bcfe-27609c648913\") " pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:13:17.412396 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:17.412370    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sgb7s\""
Jul 29 15:13:17.420499 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:17.420471    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-psdvf"
Jul 29 15:13:17.610008 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:17.609967    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-psdvf"]
Jul 29 15:13:17.614844 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:13:17.614791    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod925b0a18_77be_438d_bcfe_27609c648913.slice/crio-a3384cf7d4c7218dc895f4f55017a9497152351d3237e3d6bfcab2d4a0d99a24 WatchSource:0}: Error finding container a3384cf7d4c7218dc895f4f55017a9497152351d3237e3d6bfcab2d4a0d99a24: Status 404 returned error can't find the container with id a3384cf7d4c7218dc895f4f55017a9497152351d3237e3d6bfcab2d4a0d99a24
Jul 29 15:13:17.881903 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:17.881868    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-psdvf" event={"ID":"925b0a18-77be-438d-bcfe-27609c648913","Type":"ContainerStarted","Data":"a3384cf7d4c7218dc895f4f55017a9497152351d3237e3d6bfcab2d4a0d99a24"}
Jul 29 15:13:19.893492 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:19.893409    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c618a894-ab7d-4d1c-a493-23f63a6cfa55","Type":"ContainerStarted","Data":"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757"}
Jul 29 15:13:19.893492 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:19.893486    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c618a894-ab7d-4d1c-a493-23f63a6cfa55","Type":"ContainerStarted","Data":"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04"}
Jul 29 15:13:19.893966 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:19.893531    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c618a894-ab7d-4d1c-a493-23f63a6cfa55","Type":"ContainerStarted","Data":"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85"}
Jul 29 15:13:19.893966 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:19.893572    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c618a894-ab7d-4d1c-a493-23f63a6cfa55","Type":"ContainerStarted","Data":"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c"}
Jul 29 15:13:19.895134 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:19.895106    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-psdvf" event={"ID":"925b0a18-77be-438d-bcfe-27609c648913","Type":"ContainerStarted","Data":"ca912acd572a043ad91f7f436449a4d97861dacaad076d4b89665c4509e00a7e"}
Jul 29 15:13:19.895240 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:19.895151    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-psdvf" event={"ID":"925b0a18-77be-438d-bcfe-27609c648913","Type":"ContainerStarted","Data":"bdb625d15b3ace3a3d078367d5332991e84dc90b2ca239e857968dc7f4c03447"}
Jul 29 15:13:19.931390 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:19.931332    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.601633301 podStartE2EDuration="9.931295798s" totalImagesPullingTime="6.306942633s" totalInitContainerRuntime="22.719864ms" isStatefulPod=false podCreationTimestamp="2026-07-29 15:13:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:13:19.929220078 +0000 UTC m=+133.078513976" watchObservedRunningTime="2026-07-29 15:13:19.931295798 +0000 UTC m=+133.080589699"
Jul 29 15:13:19.952385 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:19.952309    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-psdvf" podStartSLOduration=131.197129111 podStartE2EDuration="2m12.952293876s" totalImagesPullingTime="1.755164765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:11:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:13:19.95166092 +0000 UTC m=+133.100954831" watchObservedRunningTime="2026-07-29 15:13:19.952293876 +0000 UTC m=+133.101587775"
Jul 29 15:13:21.019611 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:21.019570    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:13:35.952921 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:35.952890    2575 generic.go:356] "Generic (PLEG): container finished" podID="d3ffac2a-3208-473f-9d7f-ca7819f1ecff" containerID="888b6f152e74968cff7456048b5c957e68d48424bcc809a410f413fea7ed8ec0" exitCode=0
Jul 29 15:13:35.953359 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:35.952966    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-zfj9l" event={"ID":"d3ffac2a-3208-473f-9d7f-ca7819f1ecff","Type":"ContainerDied","Data":"888b6f152e74968cff7456048b5c957e68d48424bcc809a410f413fea7ed8ec0"}
Jul 29 15:13:35.953359 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:35.953350    2575 scope.go:117] "RemoveContainer" containerID="888b6f152e74968cff7456048b5c957e68d48424bcc809a410f413fea7ed8ec0"
Jul 29 15:13:36.959381 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:13:36.959348    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-zfj9l" event={"ID":"d3ffac2a-3208-473f-9d7f-ca7819f1ecff","Type":"ContainerStarted","Data":"73c03822c96bdb5c1f29dfc5875f1a32b7aaa0c995c7308d48deae752ba398fe"}
Jul 29 15:14:11.019709 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:11.019668    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:11.038425 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:11.038396    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:11.097527 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:11.097501    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:24.979288 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:24.979253    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 15:14:25.132696 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:25.132612    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="alertmanager" containerID="cri-o://54f85c1860eeb21df1c77ea951fbe13d6257969aa450c4d1f6cc4aa680a6ba15" gracePeriod=120
Jul 29 15:14:25.132696 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:25.132650    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="kube-rbac-proxy" containerID="cri-o://e9ff5a7a7c7079801f51d1947bb9c3f9676692f48994453553986f4732734e3c" gracePeriod=120
Jul 29 15:14:25.132696 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:25.132686    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="config-reloader" containerID="cri-o://57267101eaa4d38405101575f2c34435bd0ff5c08bd72a512569b1a63b731b99" gracePeriod=120
Jul 29 15:14:25.133005 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:25.132643    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="kube-rbac-proxy-metric" containerID="cri-o://527a443b09f840860cdd5f77874ca3ea802cfde1733f8641c5200ab7f24ca8f6" gracePeriod=120
Jul 29 15:14:25.133005 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:25.132661    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="prom-label-proxy" containerID="cri-o://88a5b5030fb75fe575b115df27413e15edcfa62b064d8831a6a65fbf58be89b5" gracePeriod=120
Jul 29 15:14:25.133005 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:25.132643    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="kube-rbac-proxy-web" containerID="cri-o://c21ea86d3156f5838d7c71f2f5a684a5e3900404abe91f289385a1db3cc64a57" gracePeriod=120
Jul 29 15:14:26.138777 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.138742    2575 generic.go:356] "Generic (PLEG): container finished" podID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerID="88a5b5030fb75fe575b115df27413e15edcfa62b064d8831a6a65fbf58be89b5" exitCode=0
Jul 29 15:14:26.138777 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.138768    2575 generic.go:356] "Generic (PLEG): container finished" podID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerID="e9ff5a7a7c7079801f51d1947bb9c3f9676692f48994453553986f4732734e3c" exitCode=0
Jul 29 15:14:26.138777 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.138776    2575 generic.go:356] "Generic (PLEG): container finished" podID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerID="c21ea86d3156f5838d7c71f2f5a684a5e3900404abe91f289385a1db3cc64a57" exitCode=0
Jul 29 15:14:26.138777 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.138784    2575 generic.go:356] "Generic (PLEG): container finished" podID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerID="57267101eaa4d38405101575f2c34435bd0ff5c08bd72a512569b1a63b731b99" exitCode=0
Jul 29 15:14:26.139219 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.138791    2575 generic.go:356] "Generic (PLEG): container finished" podID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerID="54f85c1860eeb21df1c77ea951fbe13d6257969aa450c4d1f6cc4aa680a6ba15" exitCode=0
Jul 29 15:14:26.139219 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.138812    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afe522ca-f2b0-4988-9986-ba0fd523b84f","Type":"ContainerDied","Data":"88a5b5030fb75fe575b115df27413e15edcfa62b064d8831a6a65fbf58be89b5"}
Jul 29 15:14:26.139219 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.138873    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afe522ca-f2b0-4988-9986-ba0fd523b84f","Type":"ContainerDied","Data":"e9ff5a7a7c7079801f51d1947bb9c3f9676692f48994453553986f4732734e3c"}
Jul 29 15:14:26.139219 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.138900    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afe522ca-f2b0-4988-9986-ba0fd523b84f","Type":"ContainerDied","Data":"c21ea86d3156f5838d7c71f2f5a684a5e3900404abe91f289385a1db3cc64a57"}
Jul 29 15:14:26.139219 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.138927    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afe522ca-f2b0-4988-9986-ba0fd523b84f","Type":"ContainerDied","Data":"57267101eaa4d38405101575f2c34435bd0ff5c08bd72a512569b1a63b731b99"}
Jul 29 15:14:26.139219 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.138952    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afe522ca-f2b0-4988-9986-ba0fd523b84f","Type":"ContainerDied","Data":"54f85c1860eeb21df1c77ea951fbe13d6257969aa450c4d1f6cc4aa680a6ba15"}
Jul 29 15:14:26.390413 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.390362    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:14:26.470344 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.470299    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afe522ca-f2b0-4988-9986-ba0fd523b84f-alertmanager-trusted-ca-bundle\") pod \"afe522ca-f2b0-4988-9986-ba0fd523b84f\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") "
Jul 29 15:14:26.470495 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.470396    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-cluster-tls-config\") pod \"afe522ca-f2b0-4988-9986-ba0fd523b84f\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") "
Jul 29 15:14:26.470495 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.470431    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/afe522ca-f2b0-4988-9986-ba0fd523b84f-alertmanager-main-db\") pod \"afe522ca-f2b0-4988-9986-ba0fd523b84f\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") "
Jul 29 15:14:26.470495 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.470458    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"afe522ca-f2b0-4988-9986-ba0fd523b84f\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") "
Jul 29 15:14:26.470495 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.470485    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/afe522ca-f2b0-4988-9986-ba0fd523b84f-tls-assets\") pod \"afe522ca-f2b0-4988-9986-ba0fd523b84f\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") "
Jul 29 15:14:26.470708 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.470531    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afe522ca-f2b0-4988-9986-ba0fd523b84f-metrics-client-ca\") pod \"afe522ca-f2b0-4988-9986-ba0fd523b84f\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") "
Jul 29 15:14:26.470708 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.470568    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-config-volume\") pod \"afe522ca-f2b0-4988-9986-ba0fd523b84f\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") "
Jul 29 15:14:26.470708 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.470594    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rp25c\" (UniqueName: \"kubernetes.io/projected/afe522ca-f2b0-4988-9986-ba0fd523b84f-kube-api-access-rp25c\") pod \"afe522ca-f2b0-4988-9986-ba0fd523b84f\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") "
Jul 29 15:14:26.470708 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.470644    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-main-tls\") pod \"afe522ca-f2b0-4988-9986-ba0fd523b84f\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") "
Jul 29 15:14:26.470708 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.470693    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy\") pod \"afe522ca-f2b0-4988-9986-ba0fd523b84f\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") "
Jul 29 15:14:26.470952 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.470714    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afe522ca-f2b0-4988-9986-ba0fd523b84f-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "afe522ca-f2b0-4988-9986-ba0fd523b84f" (UID: "afe522ca-f2b0-4988-9986-ba0fd523b84f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:14:26.470952 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.470720    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afe522ca-f2b0-4988-9986-ba0fd523b84f-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "afe522ca-f2b0-4988-9986-ba0fd523b84f" (UID: "afe522ca-f2b0-4988-9986-ba0fd523b84f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:14:26.470952 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.470910    2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afe522ca-f2b0-4988-9986-ba0fd523b84f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:26.470952 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.470931    2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/afe522ca-f2b0-4988-9986-ba0fd523b84f-alertmanager-main-db\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:26.471155 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.470971    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afe522ca-f2b0-4988-9986-ba0fd523b84f-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "afe522ca-f2b0-4988-9986-ba0fd523b84f" (UID: "afe522ca-f2b0-4988-9986-ba0fd523b84f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:14:26.473598 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.473252    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-config-volume" (OuterVolumeSpecName: "config-volume") pod "afe522ca-f2b0-4988-9986-ba0fd523b84f" (UID: "afe522ca-f2b0-4988-9986-ba0fd523b84f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:14:26.473598 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.473386    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "afe522ca-f2b0-4988-9986-ba0fd523b84f" (UID: "afe522ca-f2b0-4988-9986-ba0fd523b84f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:14:26.473598 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.473471    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "afe522ca-f2b0-4988-9986-ba0fd523b84f" (UID: "afe522ca-f2b0-4988-9986-ba0fd523b84f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:14:26.473598 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.473571    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afe522ca-f2b0-4988-9986-ba0fd523b84f-kube-api-access-rp25c" (OuterVolumeSpecName: "kube-api-access-rp25c") pod "afe522ca-f2b0-4988-9986-ba0fd523b84f" (UID: "afe522ca-f2b0-4988-9986-ba0fd523b84f"). InnerVolumeSpecName "kube-api-access-rp25c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:14:26.473812 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.473656    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afe522ca-f2b0-4988-9986-ba0fd523b84f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "afe522ca-f2b0-4988-9986-ba0fd523b84f" (UID: "afe522ca-f2b0-4988-9986-ba0fd523b84f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:14:26.474497 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.474469    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "afe522ca-f2b0-4988-9986-ba0fd523b84f" (UID: "afe522ca-f2b0-4988-9986-ba0fd523b84f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:14:26.477181 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.477159    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "afe522ca-f2b0-4988-9986-ba0fd523b84f" (UID: "afe522ca-f2b0-4988-9986-ba0fd523b84f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:14:26.572048 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.572008    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy-web\") pod \"afe522ca-f2b0-4988-9986-ba0fd523b84f\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") "
Jul 29 15:14:26.572048 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.572053    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/afe522ca-f2b0-4988-9986-ba0fd523b84f-config-out\") pod \"afe522ca-f2b0-4988-9986-ba0fd523b84f\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") "
Jul 29 15:14:26.572200 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.572079    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-web-config\") pod \"afe522ca-f2b0-4988-9986-ba0fd523b84f\" (UID: \"afe522ca-f2b0-4988-9986-ba0fd523b84f\") "
Jul 29 15:14:26.572358 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.572308    2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/afe522ca-f2b0-4988-9986-ba0fd523b84f-tls-assets\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:26.572358 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.572355    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afe522ca-f2b0-4988-9986-ba0fd523b84f-metrics-client-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:26.572530 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.572369    2575 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:26.572530 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.572384    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rp25c\" (UniqueName: \"kubernetes.io/projected/afe522ca-f2b0-4988-9986-ba0fd523b84f-kube-api-access-rp25c\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:26.572530 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.572396    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-main-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:26.572530 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.572405    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:26.572530 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.572413    2575 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-cluster-tls-config\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:26.572530 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.572421    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:26.573758 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.573737    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afe522ca-f2b0-4988-9986-ba0fd523b84f-config-out" (OuterVolumeSpecName: "config-out") pod "afe522ca-f2b0-4988-9986-ba0fd523b84f" (UID: "afe522ca-f2b0-4988-9986-ba0fd523b84f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:14:26.573935 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.573913    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "afe522ca-f2b0-4988-9986-ba0fd523b84f" (UID: "afe522ca-f2b0-4988-9986-ba0fd523b84f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:14:26.582462 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.582442    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-web-config" (OuterVolumeSpecName: "web-config") pod "afe522ca-f2b0-4988-9986-ba0fd523b84f" (UID: "afe522ca-f2b0-4988-9986-ba0fd523b84f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:14:26.673232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.673120    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:26.673232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.673190    2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/afe522ca-f2b0-4988-9986-ba0fd523b84f-config-out\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:26.673232 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:26.673204    2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/afe522ca-f2b0-4988-9986-ba0fd523b84f-web-config\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:27.145182 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.145149    2575 generic.go:356] "Generic (PLEG): container finished" podID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerID="527a443b09f840860cdd5f77874ca3ea802cfde1733f8641c5200ab7f24ca8f6" exitCode=0
Jul 29 15:14:27.145609 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.145233    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 15:14:27.145609 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.145232    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afe522ca-f2b0-4988-9986-ba0fd523b84f","Type":"ContainerDied","Data":"527a443b09f840860cdd5f77874ca3ea802cfde1733f8641c5200ab7f24ca8f6"}
Jul 29 15:14:27.145609 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.145338    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"afe522ca-f2b0-4988-9986-ba0fd523b84f","Type":"ContainerDied","Data":"11434e89b6134435c67066f3dd1310d3cd034d7a6f88a70d32dca1d59b16ba33"}
Jul 29 15:14:27.145609 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.145390    2575 scope.go:117] "RemoveContainer" containerID="88a5b5030fb75fe575b115df27413e15edcfa62b064d8831a6a65fbf58be89b5"
Jul 29 15:14:27.155675 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.155661    2575 scope.go:117] "RemoveContainer" containerID="527a443b09f840860cdd5f77874ca3ea802cfde1733f8641c5200ab7f24ca8f6"
Jul 29 15:14:27.162409 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.162388    2575 scope.go:117] "RemoveContainer" containerID="e9ff5a7a7c7079801f51d1947bb9c3f9676692f48994453553986f4732734e3c"
Jul 29 15:14:27.168569 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.168549    2575 scope.go:117] "RemoveContainer" containerID="c21ea86d3156f5838d7c71f2f5a684a5e3900404abe91f289385a1db3cc64a57"
Jul 29 15:14:27.173144 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.173121    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 15:14:27.175828 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.175811    2575 scope.go:117] "RemoveContainer" containerID="57267101eaa4d38405101575f2c34435bd0ff5c08bd72a512569b1a63b731b99"
Jul 29 15:14:27.179051 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.179029    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 15:14:27.183056 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.183038    2575 scope.go:117] "RemoveContainer" containerID="54f85c1860eeb21df1c77ea951fbe13d6257969aa450c4d1f6cc4aa680a6ba15"
Jul 29 15:14:27.189578 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.189556    2575 scope.go:117] "RemoveContainer" containerID="51b19e52b616256e5021412bb126b924ad6db1a5608684da7689ed3ccccdf26e"
Jul 29 15:14:27.196579 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.196562    2575 scope.go:117] "RemoveContainer" containerID="88a5b5030fb75fe575b115df27413e15edcfa62b064d8831a6a65fbf58be89b5"
Jul 29 15:14:27.196831 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:14:27.196813    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88a5b5030fb75fe575b115df27413e15edcfa62b064d8831a6a65fbf58be89b5\": container with ID starting with 88a5b5030fb75fe575b115df27413e15edcfa62b064d8831a6a65fbf58be89b5 not found: ID does not exist" containerID="88a5b5030fb75fe575b115df27413e15edcfa62b064d8831a6a65fbf58be89b5"
Jul 29 15:14:27.196869 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.196843    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88a5b5030fb75fe575b115df27413e15edcfa62b064d8831a6a65fbf58be89b5"} err="failed to get container status \"88a5b5030fb75fe575b115df27413e15edcfa62b064d8831a6a65fbf58be89b5\": rpc error: code = NotFound desc = could not find container \"88a5b5030fb75fe575b115df27413e15edcfa62b064d8831a6a65fbf58be89b5\": container with ID starting with 88a5b5030fb75fe575b115df27413e15edcfa62b064d8831a6a65fbf58be89b5 not found: ID does not exist"
Jul 29 15:14:27.196869 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.196864    2575 scope.go:117] "RemoveContainer" containerID="527a443b09f840860cdd5f77874ca3ea802cfde1733f8641c5200ab7f24ca8f6"
Jul 29 15:14:27.197107 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:14:27.197092    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"527a443b09f840860cdd5f77874ca3ea802cfde1733f8641c5200ab7f24ca8f6\": container with ID starting with 527a443b09f840860cdd5f77874ca3ea802cfde1733f8641c5200ab7f24ca8f6 not found: ID does not exist" containerID="527a443b09f840860cdd5f77874ca3ea802cfde1733f8641c5200ab7f24ca8f6"
Jul 29 15:14:27.197150 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.197113    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"527a443b09f840860cdd5f77874ca3ea802cfde1733f8641c5200ab7f24ca8f6"} err="failed to get container status \"527a443b09f840860cdd5f77874ca3ea802cfde1733f8641c5200ab7f24ca8f6\": rpc error: code = NotFound desc = could not find container \"527a443b09f840860cdd5f77874ca3ea802cfde1733f8641c5200ab7f24ca8f6\": container with ID starting with 527a443b09f840860cdd5f77874ca3ea802cfde1733f8641c5200ab7f24ca8f6 not found: ID does not exist"
Jul 29 15:14:27.197150 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.197128    2575 scope.go:117] "RemoveContainer" containerID="e9ff5a7a7c7079801f51d1947bb9c3f9676692f48994453553986f4732734e3c"
Jul 29 15:14:27.197348 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:14:27.197313    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9ff5a7a7c7079801f51d1947bb9c3f9676692f48994453553986f4732734e3c\": container with ID starting with e9ff5a7a7c7079801f51d1947bb9c3f9676692f48994453553986f4732734e3c not found: ID does not exist" containerID="e9ff5a7a7c7079801f51d1947bb9c3f9676692f48994453553986f4732734e3c"
Jul 29 15:14:27.197402 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.197350    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9ff5a7a7c7079801f51d1947bb9c3f9676692f48994453553986f4732734e3c"} err="failed to get container status \"e9ff5a7a7c7079801f51d1947bb9c3f9676692f48994453553986f4732734e3c\": rpc error: code = NotFound desc = could not find container \"e9ff5a7a7c7079801f51d1947bb9c3f9676692f48994453553986f4732734e3c\": container with ID starting with e9ff5a7a7c7079801f51d1947bb9c3f9676692f48994453553986f4732734e3c not found: ID does not exist"
Jul 29 15:14:27.197402 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.197362    2575 scope.go:117] "RemoveContainer" containerID="c21ea86d3156f5838d7c71f2f5a684a5e3900404abe91f289385a1db3cc64a57"
Jul 29 15:14:27.197563 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:14:27.197544    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c21ea86d3156f5838d7c71f2f5a684a5e3900404abe91f289385a1db3cc64a57\": container with ID starting with c21ea86d3156f5838d7c71f2f5a684a5e3900404abe91f289385a1db3cc64a57 not found: ID does not exist" containerID="c21ea86d3156f5838d7c71f2f5a684a5e3900404abe91f289385a1db3cc64a57"
Jul 29 15:14:27.197607 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.197567    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c21ea86d3156f5838d7c71f2f5a684a5e3900404abe91f289385a1db3cc64a57"} err="failed to get container status \"c21ea86d3156f5838d7c71f2f5a684a5e3900404abe91f289385a1db3cc64a57\": rpc error: code = NotFound desc = could not find container \"c21ea86d3156f5838d7c71f2f5a684a5e3900404abe91f289385a1db3cc64a57\": container with ID starting with c21ea86d3156f5838d7c71f2f5a684a5e3900404abe91f289385a1db3cc64a57 not found: ID does not exist"
Jul 29 15:14:27.197607 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.197578    2575 scope.go:117] "RemoveContainer" containerID="57267101eaa4d38405101575f2c34435bd0ff5c08bd72a512569b1a63b731b99"
Jul 29 15:14:27.197783 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:14:27.197766    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57267101eaa4d38405101575f2c34435bd0ff5c08bd72a512569b1a63b731b99\": container with ID starting with 57267101eaa4d38405101575f2c34435bd0ff5c08bd72a512569b1a63b731b99 not found: ID does not exist" containerID="57267101eaa4d38405101575f2c34435bd0ff5c08bd72a512569b1a63b731b99"
Jul 29 15:14:27.197824 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.197787    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57267101eaa4d38405101575f2c34435bd0ff5c08bd72a512569b1a63b731b99"} err="failed to get container status \"57267101eaa4d38405101575f2c34435bd0ff5c08bd72a512569b1a63b731b99\": rpc error: code = NotFound desc = could not find container \"57267101eaa4d38405101575f2c34435bd0ff5c08bd72a512569b1a63b731b99\": container with ID starting with 57267101eaa4d38405101575f2c34435bd0ff5c08bd72a512569b1a63b731b99 not found: ID does not exist"
Jul 29 15:14:27.197824 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.197799    2575 scope.go:117] "RemoveContainer" containerID="54f85c1860eeb21df1c77ea951fbe13d6257969aa450c4d1f6cc4aa680a6ba15"
Jul 29 15:14:27.198037 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:14:27.198024    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54f85c1860eeb21df1c77ea951fbe13d6257969aa450c4d1f6cc4aa680a6ba15\": container with ID starting with 54f85c1860eeb21df1c77ea951fbe13d6257969aa450c4d1f6cc4aa680a6ba15 not found: ID does not exist" containerID="54f85c1860eeb21df1c77ea951fbe13d6257969aa450c4d1f6cc4aa680a6ba15"
Jul 29 15:14:27.198077 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.198039    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54f85c1860eeb21df1c77ea951fbe13d6257969aa450c4d1f6cc4aa680a6ba15"} err="failed to get container status \"54f85c1860eeb21df1c77ea951fbe13d6257969aa450c4d1f6cc4aa680a6ba15\": rpc error: code = NotFound desc = could not find container \"54f85c1860eeb21df1c77ea951fbe13d6257969aa450c4d1f6cc4aa680a6ba15\": container with ID starting with 54f85c1860eeb21df1c77ea951fbe13d6257969aa450c4d1f6cc4aa680a6ba15 not found: ID does not exist"
Jul 29 15:14:27.198077 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.198051    2575 scope.go:117] "RemoveContainer" containerID="51b19e52b616256e5021412bb126b924ad6db1a5608684da7689ed3ccccdf26e"
Jul 29 15:14:27.198237 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:14:27.198223    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51b19e52b616256e5021412bb126b924ad6db1a5608684da7689ed3ccccdf26e\": container with ID starting with 51b19e52b616256e5021412bb126b924ad6db1a5608684da7689ed3ccccdf26e not found: ID does not exist" containerID="51b19e52b616256e5021412bb126b924ad6db1a5608684da7689ed3ccccdf26e"
Jul 29 15:14:27.198270 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.198239    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51b19e52b616256e5021412bb126b924ad6db1a5608684da7689ed3ccccdf26e"} err="failed to get container status \"51b19e52b616256e5021412bb126b924ad6db1a5608684da7689ed3ccccdf26e\": rpc error: code = NotFound desc = could not find container \"51b19e52b616256e5021412bb126b924ad6db1a5608684da7689ed3ccccdf26e\": container with ID starting with 51b19e52b616256e5021412bb126b924ad6db1a5608684da7689ed3ccccdf26e not found: ID does not exist"
Jul 29 15:14:27.297046 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:27.297019    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" path="/var/lib/kubelet/pods/afe522ca-f2b0-4988-9986-ba0fd523b84f/volumes"
Jul 29 15:14:29.262269 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.261742    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 15:14:29.262269 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.262077    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="prometheus" containerID="cri-o://1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7" gracePeriod=600
Jul 29 15:14:29.262828 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.262357    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="kube-rbac-proxy-thanos" containerID="cri-o://ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757" gracePeriod=600
Jul 29 15:14:29.262828 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.262344    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="kube-rbac-proxy" containerID="cri-o://4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04" gracePeriod=600
Jul 29 15:14:29.262828 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.262409    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="config-reloader" containerID="cri-o://e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf" gracePeriod=600
Jul 29 15:14:29.262828 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.262476    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="thanos-sidecar" containerID="cri-o://69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c" gracePeriod=600
Jul 29 15:14:29.262828 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.262491    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="kube-rbac-proxy-web" containerID="cri-o://7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85" gracePeriod=600
Jul 29 15:14:29.498825 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.498802    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:29.591832 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.591756    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-metrics-client-certs\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.591832 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.591787    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-db\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.591832 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.591821    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-2\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592083 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.591846    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592083 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.591878    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-config\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592083 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.591908    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-tls\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592083 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.591939    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcpph\" (UniqueName: \"kubernetes.io/projected/c618a894-ab7d-4d1c-a493-23f63a6cfa55-kube-api-access-mcpph\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592083 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.591961    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c618a894-ab7d-4d1c-a493-23f63a6cfa55-config-out\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592083 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.592000    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-metrics-client-ca\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592083 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.592026    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-1\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592083 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.592057    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-trusted-ca-bundle\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592487 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.592093    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-kubelet-serving-ca-bundle\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592487 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.592118    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-web-config\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592487 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.592146    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-serving-certs-ca-bundle\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592487 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.592178    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592487 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.592243    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-kube-rbac-proxy\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592487 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.592250    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:14:29.592487 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.592276    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-grpc-tls\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592487 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.592301    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c618a894-ab7d-4d1c-a493-23f63a6cfa55-tls-assets\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592487 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.592346    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-0\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592487 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.592387    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-thanos-prometheus-http-client-file\") pod \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\" (UID: \"c618a894-ab7d-4d1c-a493-23f63a6cfa55\") "
Jul 29 15:14:29.592995 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.592769    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:14:29.592995 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.592969    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.593110 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.592990    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:14:29.593834 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.593633    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:14:29.594052 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.594025    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:14:29.594376 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.594296    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:14:29.594585 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.594559    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:14:29.594691 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.594651    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:14:29.595186 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.595158    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:14:29.595291 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.595183    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:14:29.596363 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.596312    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:14:29.596635 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.596601    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:14:29.596711 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.596664    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:14:29.596804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.596779    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-config" (OuterVolumeSpecName: "config") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:14:29.596804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.596793    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:14:29.597346 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.597304    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c618a894-ab7d-4d1c-a493-23f63a6cfa55-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:14:29.597779 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.597759    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c618a894-ab7d-4d1c-a493-23f63a6cfa55-kube-api-access-mcpph" (OuterVolumeSpecName: "kube-api-access-mcpph") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "kube-api-access-mcpph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:14:29.598117 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.598101    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:14:29.598462 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.598441    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c618a894-ab7d-4d1c-a493-23f63a6cfa55-config-out" (OuterVolumeSpecName: "config-out") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:14:29.605172 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.605150    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-web-config" (OuterVolumeSpecName: "web-config") pod "c618a894-ab7d-4d1c-a493-23f63a6cfa55" (UID: "c618a894-ab7d-4d1c-a493-23f63a6cfa55"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:14:29.693917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.693890    2575 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-grpc-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.693917 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.693916    2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c618a894-ab7d-4d1c-a493-23f63a6cfa55-tls-assets\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.694056 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.693924    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.694056 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.693935    2575 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.694056 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.693944    2575 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-metrics-client-certs\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.694056 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.693974    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-db\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.694056 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.693985    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.694056 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.693994    2575 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-config\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.694056 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.694002    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.694056 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.694012    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcpph\" (UniqueName: \"kubernetes.io/projected/c618a894-ab7d-4d1c-a493-23f63a6cfa55-kube-api-access-mcpph\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.694056 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.694020    2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c618a894-ab7d-4d1c-a493-23f63a6cfa55-config-out\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.694056 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.694027    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-metrics-client-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.694056 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.694037    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.694056 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.694046    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.694056 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.694054    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.694056 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.694062    2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-web-config\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.694484 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.694071    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c618a894-ab7d-4d1c-a493-23f63a6cfa55-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.694484 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.694079    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:29.694484 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:29.694089    2575 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c618a894-ab7d-4d1c-a493-23f63a6cfa55-secret-kube-rbac-proxy\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:14:30.159051 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.159017    2575 generic.go:356] "Generic (PLEG): container finished" podID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerID="ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757" exitCode=0
Jul 29 15:14:30.159051 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.159042    2575 generic.go:356] "Generic (PLEG): container finished" podID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerID="4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04" exitCode=0
Jul 29 15:14:30.159051 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.159050    2575 generic.go:356] "Generic (PLEG): container finished" podID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerID="7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85" exitCode=0
Jul 29 15:14:30.159051 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.159057    2575 generic.go:356] "Generic (PLEG): container finished" podID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerID="69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c" exitCode=0
Jul 29 15:14:30.159474 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.159064    2575 generic.go:356] "Generic (PLEG): container finished" podID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerID="e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf" exitCode=0
Jul 29 15:14:30.159474 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.159071    2575 generic.go:356] "Generic (PLEG): container finished" podID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerID="1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7" exitCode=0
Jul 29 15:14:30.159474 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.159110    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.159474 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.159104    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c618a894-ab7d-4d1c-a493-23f63a6cfa55","Type":"ContainerDied","Data":"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757"}
Jul 29 15:14:30.159474 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.159253    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c618a894-ab7d-4d1c-a493-23f63a6cfa55","Type":"ContainerDied","Data":"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04"}
Jul 29 15:14:30.159474 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.159285    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c618a894-ab7d-4d1c-a493-23f63a6cfa55","Type":"ContainerDied","Data":"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85"}
Jul 29 15:14:30.159474 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.159310    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c618a894-ab7d-4d1c-a493-23f63a6cfa55","Type":"ContainerDied","Data":"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c"}
Jul 29 15:14:30.159474 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.159335    2575 scope.go:117] "RemoveContainer" containerID="ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757"
Jul 29 15:14:30.159474 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.159353    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c618a894-ab7d-4d1c-a493-23f63a6cfa55","Type":"ContainerDied","Data":"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf"}
Jul 29 15:14:30.159474 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.159379    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c618a894-ab7d-4d1c-a493-23f63a6cfa55","Type":"ContainerDied","Data":"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7"}
Jul 29 15:14:30.159474 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.159402    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c618a894-ab7d-4d1c-a493-23f63a6cfa55","Type":"ContainerDied","Data":"8ab56a5e3301c7b9372a53ffdcaf212e38b88777049a62c9f31099a16fa27a75"}
Jul 29 15:14:30.166829 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.166798    2575 scope.go:117] "RemoveContainer" containerID="4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04"
Jul 29 15:14:30.173601 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.173581    2575 scope.go:117] "RemoveContainer" containerID="7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85"
Jul 29 15:14:30.179807 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.179789    2575 scope.go:117] "RemoveContainer" containerID="69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c"
Jul 29 15:14:30.183696 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.183671    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 15:14:30.186724 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.186706    2575 scope.go:117] "RemoveContainer" containerID="e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf"
Jul 29 15:14:30.189090 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.189070    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 15:14:30.193422 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.193401    2575 scope.go:117] "RemoveContainer" containerID="1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7"
Jul 29 15:14:30.199819 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.199805    2575 scope.go:117] "RemoveContainer" containerID="18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd"
Jul 29 15:14:30.208044 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.208024    2575 scope.go:117] "RemoveContainer" containerID="ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757"
Jul 29 15:14:30.208439 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:14:30.208420    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757\": container with ID starting with ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757 not found: ID does not exist" containerID="ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757"
Jul 29 15:14:30.208528 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.208450    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757"} err="failed to get container status \"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757\": rpc error: code = NotFound desc = could not find container \"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757\": container with ID starting with ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757 not found: ID does not exist"
Jul 29 15:14:30.208528 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.208474    2575 scope.go:117] "RemoveContainer" containerID="4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04"
Jul 29 15:14:30.208722 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:14:30.208707    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04\": container with ID starting with 4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04 not found: ID does not exist" containerID="4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04"
Jul 29 15:14:30.208782 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.208730    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04"} err="failed to get container status \"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04\": rpc error: code = NotFound desc = could not find container \"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04\": container with ID starting with 4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04 not found: ID does not exist"
Jul 29 15:14:30.208782 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.208752    2575 scope.go:117] "RemoveContainer" containerID="7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85"
Jul 29 15:14:30.209022 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:14:30.209002    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85\": container with ID starting with 7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85 not found: ID does not exist" containerID="7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85"
Jul 29 15:14:30.209104 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.209025    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85"} err="failed to get container status \"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85\": rpc error: code = NotFound desc = could not find container \"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85\": container with ID starting with 7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85 not found: ID does not exist"
Jul 29 15:14:30.209104 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.209040    2575 scope.go:117] "RemoveContainer" containerID="69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c"
Jul 29 15:14:30.209292 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:14:30.209275    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c\": container with ID starting with 69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c not found: ID does not exist" containerID="69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c"
Jul 29 15:14:30.209415 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.209299    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c"} err="failed to get container status \"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c\": rpc error: code = NotFound desc = could not find container \"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c\": container with ID starting with 69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c not found: ID does not exist"
Jul 29 15:14:30.209415 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.209335    2575 scope.go:117] "RemoveContainer" containerID="e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf"
Jul 29 15:14:30.209593 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:14:30.209576    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf\": container with ID starting with e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf not found: ID does not exist" containerID="e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf"
Jul 29 15:14:30.209630 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.209597    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf"} err="failed to get container status \"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf\": rpc error: code = NotFound desc = could not find container \"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf\": container with ID starting with e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf not found: ID does not exist"
Jul 29 15:14:30.209630 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.209610    2575 scope.go:117] "RemoveContainer" containerID="1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7"
Jul 29 15:14:30.209841 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:14:30.209801    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7\": container with ID starting with 1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7 not found: ID does not exist" containerID="1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7"
Jul 29 15:14:30.209881 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.209847    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7"} err="failed to get container status \"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7\": rpc error: code = NotFound desc = could not find container \"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7\": container with ID starting with 1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7 not found: ID does not exist"
Jul 29 15:14:30.209881 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.209862    2575 scope.go:117] "RemoveContainer" containerID="18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd"
Jul 29 15:14:30.210093 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:14:30.210080    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd\": container with ID starting with 18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd not found: ID does not exist" containerID="18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd"
Jul 29 15:14:30.210134 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.210094    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd"} err="failed to get container status \"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd\": rpc error: code = NotFound desc = could not find container \"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd\": container with ID starting with 18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd not found: ID does not exist"
Jul 29 15:14:30.210134 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.210106    2575 scope.go:117] "RemoveContainer" containerID="ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757"
Jul 29 15:14:30.210313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.210297    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757"} err="failed to get container status \"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757\": rpc error: code = NotFound desc = could not find container \"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757\": container with ID starting with ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757 not found: ID does not exist"
Jul 29 15:14:30.210381 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.210313    2575 scope.go:117] "RemoveContainer" containerID="4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04"
Jul 29 15:14:30.210512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.210496    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04"} err="failed to get container status \"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04\": rpc error: code = NotFound desc = could not find container \"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04\": container with ID starting with 4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04 not found: ID does not exist"
Jul 29 15:14:30.210552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.210512    2575 scope.go:117] "RemoveContainer" containerID="7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85"
Jul 29 15:14:30.210697 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.210682    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85"} err="failed to get container status \"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85\": rpc error: code = NotFound desc = could not find container \"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85\": container with ID starting with 7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85 not found: ID does not exist"
Jul 29 15:14:30.210744 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.210698    2575 scope.go:117] "RemoveContainer" containerID="69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c"
Jul 29 15:14:30.210898 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.210880    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c"} err="failed to get container status \"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c\": rpc error: code = NotFound desc = could not find container \"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c\": container with ID starting with 69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c not found: ID does not exist"
Jul 29 15:14:30.210949 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.210899    2575 scope.go:117] "RemoveContainer" containerID="e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf"
Jul 29 15:14:30.211119 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.211099    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf"} err="failed to get container status \"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf\": rpc error: code = NotFound desc = could not find container \"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf\": container with ID starting with e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf not found: ID does not exist"
Jul 29 15:14:30.211166 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.211119    2575 scope.go:117] "RemoveContainer" containerID="1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7"
Jul 29 15:14:30.211372 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.211310    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7"} err="failed to get container status \"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7\": rpc error: code = NotFound desc = could not find container \"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7\": container with ID starting with 1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7 not found: ID does not exist"
Jul 29 15:14:30.211429 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.211373    2575 scope.go:117] "RemoveContainer" containerID="18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd"
Jul 29 15:14:30.211600 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.211585    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd"} err="failed to get container status \"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd\": rpc error: code = NotFound desc = could not find container \"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd\": container with ID starting with 18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd not found: ID does not exist"
Jul 29 15:14:30.211637 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.211601    2575 scope.go:117] "RemoveContainer" containerID="ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757"
Jul 29 15:14:30.211778 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.211763    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757"} err="failed to get container status \"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757\": rpc error: code = NotFound desc = could not find container \"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757\": container with ID starting with ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757 not found: ID does not exist"
Jul 29 15:14:30.211822 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.211778    2575 scope.go:117] "RemoveContainer" containerID="4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04"
Jul 29 15:14:30.211983 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.211967    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04"} err="failed to get container status \"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04\": rpc error: code = NotFound desc = could not find container \"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04\": container with ID starting with 4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04 not found: ID does not exist"
Jul 29 15:14:30.212073 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.211984    2575 scope.go:117] "RemoveContainer" containerID="7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85"
Jul 29 15:14:30.212160 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.212145    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85"} err="failed to get container status \"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85\": rpc error: code = NotFound desc = could not find container \"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85\": container with ID starting with 7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85 not found: ID does not exist"
Jul 29 15:14:30.212203 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.212159    2575 scope.go:117] "RemoveContainer" containerID="69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c"
Jul 29 15:14:30.212343 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.212310    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c"} err="failed to get container status \"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c\": rpc error: code = NotFound desc = could not find container \"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c\": container with ID starting with 69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c not found: ID does not exist"
Jul 29 15:14:30.212417 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.212346    2575 scope.go:117] "RemoveContainer" containerID="e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf"
Jul 29 15:14:30.212556 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.212539    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf"} err="failed to get container status \"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf\": rpc error: code = NotFound desc = could not find container \"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf\": container with ID starting with e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf not found: ID does not exist"
Jul 29 15:14:30.212623 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.212558    2575 scope.go:117] "RemoveContainer" containerID="1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7"
Jul 29 15:14:30.212774 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.212760    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7"} err="failed to get container status \"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7\": rpc error: code = NotFound desc = could not find container \"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7\": container with ID starting with 1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7 not found: ID does not exist"
Jul 29 15:14:30.212816 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.212773    2575 scope.go:117] "RemoveContainer" containerID="18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd"
Jul 29 15:14:30.212955 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.212936    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd"} err="failed to get container status \"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd\": rpc error: code = NotFound desc = could not find container \"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd\": container with ID starting with 18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd not found: ID does not exist"
Jul 29 15:14:30.213019 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.212956    2575 scope.go:117] "RemoveContainer" containerID="ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757"
Jul 29 15:14:30.213157 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.213142    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757"} err="failed to get container status \"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757\": rpc error: code = NotFound desc = could not find container \"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757\": container with ID starting with ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757 not found: ID does not exist"
Jul 29 15:14:30.213202 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.213157    2575 scope.go:117] "RemoveContainer" containerID="4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04"
Jul 29 15:14:30.213333 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.213306    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04"} err="failed to get container status \"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04\": rpc error: code = NotFound desc = could not find container \"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04\": container with ID starting with 4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04 not found: ID does not exist"
Jul 29 15:14:30.213380 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.213335    2575 scope.go:117] "RemoveContainer" containerID="7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85"
Jul 29 15:14:30.213507 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.213491    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85"} err="failed to get container status \"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85\": rpc error: code = NotFound desc = could not find container \"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85\": container with ID starting with 7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85 not found: ID does not exist"
Jul 29 15:14:30.213570 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.213508    2575 scope.go:117] "RemoveContainer" containerID="69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c"
Jul 29 15:14:30.213675 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.213661    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c"} err="failed to get container status \"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c\": rpc error: code = NotFound desc = could not find container \"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c\": container with ID starting with 69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c not found: ID does not exist"
Jul 29 15:14:30.213715 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.213676    2575 scope.go:117] "RemoveContainer" containerID="e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf"
Jul 29 15:14:30.213860 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.213846    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf"} err="failed to get container status \"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf\": rpc error: code = NotFound desc = could not find container \"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf\": container with ID starting with e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf not found: ID does not exist"
Jul 29 15:14:30.213897 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.213861    2575 scope.go:117] "RemoveContainer" containerID="1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7"
Jul 29 15:14:30.214029 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.214013    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7"} err="failed to get container status \"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7\": rpc error: code = NotFound desc = could not find container \"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7\": container with ID starting with 1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7 not found: ID does not exist"
Jul 29 15:14:30.214096 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.214030    2575 scope.go:117] "RemoveContainer" containerID="18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd"
Jul 29 15:14:30.214205 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.214191    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd"} err="failed to get container status \"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd\": rpc error: code = NotFound desc = could not find container \"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd\": container with ID starting with 18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd not found: ID does not exist"
Jul 29 15:14:30.214249 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.214207    2575 scope.go:117] "RemoveContainer" containerID="ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757"
Jul 29 15:14:30.214383 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.214368    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757"} err="failed to get container status \"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757\": rpc error: code = NotFound desc = could not find container \"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757\": container with ID starting with ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757 not found: ID does not exist"
Jul 29 15:14:30.214430 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.214383    2575 scope.go:117] "RemoveContainer" containerID="4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04"
Jul 29 15:14:30.214557 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.214540    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04"} err="failed to get container status \"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04\": rpc error: code = NotFound desc = could not find container \"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04\": container with ID starting with 4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04 not found: ID does not exist"
Jul 29 15:14:30.214623 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.214559    2575 scope.go:117] "RemoveContainer" containerID="7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85"
Jul 29 15:14:30.214736 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.214721    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85"} err="failed to get container status \"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85\": rpc error: code = NotFound desc = could not find container \"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85\": container with ID starting with 7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85 not found: ID does not exist"
Jul 29 15:14:30.214781 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.214736    2575 scope.go:117] "RemoveContainer" containerID="69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c"
Jul 29 15:14:30.214893 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.214879    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c"} err="failed to get container status \"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c\": rpc error: code = NotFound desc = could not find container \"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c\": container with ID starting with 69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c not found: ID does not exist"
Jul 29 15:14:30.214934 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.214893    2575 scope.go:117] "RemoveContainer" containerID="e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf"
Jul 29 15:14:30.215078 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.215061    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf"} err="failed to get container status \"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf\": rpc error: code = NotFound desc = could not find container \"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf\": container with ID starting with e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf not found: ID does not exist"
Jul 29 15:14:30.215145 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.215080    2575 scope.go:117] "RemoveContainer" containerID="1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7"
Jul 29 15:14:30.215282 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.215267    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7"} err="failed to get container status \"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7\": rpc error: code = NotFound desc = could not find container \"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7\": container with ID starting with 1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7 not found: ID does not exist"
Jul 29 15:14:30.215282 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.215282    2575 scope.go:117] "RemoveContainer" containerID="18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd"
Jul 29 15:14:30.215504 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.215484    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd"} err="failed to get container status \"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd\": rpc error: code = NotFound desc = could not find container \"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd\": container with ID starting with 18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd not found: ID does not exist"
Jul 29 15:14:30.215504 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.215502    2575 scope.go:117] "RemoveContainer" containerID="ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757"
Jul 29 15:14:30.215711 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.215693    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757"} err="failed to get container status \"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757\": rpc error: code = NotFound desc = could not find container \"ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757\": container with ID starting with ccc30daf2f2ddf3053afc7e6ebef4bb992f5a82afebc455eac92b64e7214a757 not found: ID does not exist"
Jul 29 15:14:30.215782 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.215713    2575 scope.go:117] "RemoveContainer" containerID="4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04"
Jul 29 15:14:30.215921 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.215904    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04"} err="failed to get container status \"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04\": rpc error: code = NotFound desc = could not find container \"4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04\": container with ID starting with 4a195d474b566a0b734e7d4994c1e4af2aa0d25183cc191559615c7eb5079a04 not found: ID does not exist"
Jul 29 15:14:30.215962 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.215922    2575 scope.go:117] "RemoveContainer" containerID="7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85"
Jul 29 15:14:30.216124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.216105    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85"} err="failed to get container status \"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85\": rpc error: code = NotFound desc = could not find container \"7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85\": container with ID starting with 7736c0705c4b5d615717e8e18031c2f44f01422dbb1d8a2f980d0f98de52ae85 not found: ID does not exist"
Jul 29 15:14:30.216194 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.216124    2575 scope.go:117] "RemoveContainer" containerID="69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c"
Jul 29 15:14:30.216375 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.216349    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c"} err="failed to get container status \"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c\": rpc error: code = NotFound desc = could not find container \"69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c\": container with ID starting with 69a4e546b112edca386187580e9e7319e8361f1da07bdc2cfc3ae61be4618b9c not found: ID does not exist"
Jul 29 15:14:30.216375 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.216372    2575 scope.go:117] "RemoveContainer" containerID="e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf"
Jul 29 15:14:30.216640 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.216611    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf"} err="failed to get container status \"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf\": rpc error: code = NotFound desc = could not find container \"e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf\": container with ID starting with e4bc8835fe3c3a76d4eca8964c6e94eb78c405690b1d4c6d7fbc35993893c9cf not found: ID does not exist"
Jul 29 15:14:30.216640 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.216636    2575 scope.go:117] "RemoveContainer" containerID="1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7"
Jul 29 15:14:30.216916 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.216900    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7"} err="failed to get container status \"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7\": rpc error: code = NotFound desc = could not find container \"1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7\": container with ID starting with 1aae2c5b375071c19b4ce6a0dfb3d244e86ae496cab1f0424e683f4d5b7c9ad7 not found: ID does not exist"
Jul 29 15:14:30.216916 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.216916    2575 scope.go:117] "RemoveContainer" containerID="18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd"
Jul 29 15:14:30.217141 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.217123    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd"} err="failed to get container status \"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd\": rpc error: code = NotFound desc = could not find container \"18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd\": container with ID starting with 18cfd21563ad95849692b54c050faf95c26da9b0dc609b25a95fdf5f093b1afd not found: ID does not exist"
Jul 29 15:14:30.227299 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227278    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 15:14:30.227512 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227500    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="kube-rbac-proxy-metric"
Jul 29 15:14:30.227557 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227514    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="kube-rbac-proxy-metric"
Jul 29 15:14:30.227557 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227522    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="config-reloader"
Jul 29 15:14:30.227557 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227528    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="config-reloader"
Jul 29 15:14:30.227557 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227549    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="config-reloader"
Jul 29 15:14:30.227557 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227554    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="config-reloader"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227562    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="kube-rbac-proxy-thanos"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227567    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="kube-rbac-proxy-thanos"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227574    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="prom-label-proxy"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227579    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="prom-label-proxy"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227586    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="kube-rbac-proxy-web"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227592    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="kube-rbac-proxy-web"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227597    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="alertmanager"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227602    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="alertmanager"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227608    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="kube-rbac-proxy"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227612    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="kube-rbac-proxy"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227618    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="prometheus"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227623    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="prometheus"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227634    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="init-config-reloader"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227640    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="init-config-reloader"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227645    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="kube-rbac-proxy-web"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227651    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="kube-rbac-proxy-web"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227657    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="init-config-reloader"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227661    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="init-config-reloader"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227670    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="thanos-sidecar"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227674    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="thanos-sidecar"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227680    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="kube-rbac-proxy"
Jul 29 15:14:30.227701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227685    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="kube-rbac-proxy"
Jul 29 15:14:30.228289 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227720    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="config-reloader"
Jul 29 15:14:30.228289 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227727    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="kube-rbac-proxy-thanos"
Jul 29 15:14:30.228289 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227733    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="kube-rbac-proxy-metric"
Jul 29 15:14:30.228289 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227739    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="config-reloader"
Jul 29 15:14:30.228289 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227744    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="kube-rbac-proxy-web"
Jul 29 15:14:30.228289 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227749    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="alertmanager"
Jul 29 15:14:30.228289 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227754    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="kube-rbac-proxy"
Jul 29 15:14:30.228289 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227759    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="prom-label-proxy"
Jul 29 15:14:30.228289 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227764    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="prometheus"
Jul 29 15:14:30.228289 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227769    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="thanos-sidecar"
Jul 29 15:14:30.228289 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227774    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="afe522ca-f2b0-4988-9986-ba0fd523b84f" containerName="kube-rbac-proxy-web"
Jul 29 15:14:30.228289 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.227778    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" containerName="kube-rbac-proxy"
Jul 29 15:14:30.232285 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.232270    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.234673 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.234635    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 29 15:14:30.234756 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.234654    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 29 15:14:30.235149 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.235131    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-27d1ht214qgs7\""
Jul 29 15:14:30.236931 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.235595    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 29 15:14:30.236931 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.236314    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 29 15:14:30.236931 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.236506    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 29 15:14:30.236931 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.236510    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5k6nn\""
Jul 29 15:14:30.237359 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.237341    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 29 15:14:30.237871 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.237492    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 29 15:14:30.238732 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.237944    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 29 15:14:30.238732 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.238090    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 29 15:14:30.238732 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.238152    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 29 15:14:30.238732 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.238473    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 29 15:14:30.241445 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.239492    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 29 15:14:30.241445 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.240011    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 29 15:14:30.242802 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.242766    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 29 15:14:30.244631 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.244611    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 29 15:14:30.257956 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.257935    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 15:14:30.298487 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298445    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.298487 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298492    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.298844 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298510    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b55a053e-d8e1-4d15-afbf-76e928fc5d47-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.298844 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298526    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.298844 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298547    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4dsc\" (UniqueName: \"kubernetes.io/projected/b55a053e-d8e1-4d15-afbf-76e928fc5d47-kube-api-access-x4dsc\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.298844 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298568    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-web-config\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.298844 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298586    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b55a053e-d8e1-4d15-afbf-76e928fc5d47-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.298844 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298603    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.298844 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298635    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.298844 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298659    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.298844 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298680    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.298844 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298709    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.298844 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298750    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.298844 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298801    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.298844 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298838    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-config\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.299221 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298853    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.299221 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298869    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.299221 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298886    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.299221 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298913    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.299221 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.298936    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b55a053e-d8e1-4d15-afbf-76e928fc5d47-config-out\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.399899 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.399868    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.399899 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.399901    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400072 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.399922    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400072 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.399943    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b55a053e-d8e1-4d15-afbf-76e928fc5d47-config-out\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400205 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.400061    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400205 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.400101    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400205 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.400124    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b55a053e-d8e1-4d15-afbf-76e928fc5d47-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400205 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.400146    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400205 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.400169    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4dsc\" (UniqueName: \"kubernetes.io/projected/b55a053e-d8e1-4d15-afbf-76e928fc5d47-kube-api-access-x4dsc\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400578 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.400443    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-web-config\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400578 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.400484    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b55a053e-d8e1-4d15-afbf-76e928fc5d47-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400578 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.400512    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400578 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.400541    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400578 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.400567    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.400593    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.400617    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.400652    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.400696    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.400737    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-config\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.400839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.400759    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.401146 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.400881    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b55a053e-d8e1-4d15-afbf-76e928fc5d47-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.401199 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.401146    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.401755 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.401728    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.402269 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.402243    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.403119 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.403075    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b55a053e-d8e1-4d15-afbf-76e928fc5d47-config-out\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.403222 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.403130    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.403474 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.403454    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.403688 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.403660    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.404107 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.403949    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.404107 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.403976    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.404107 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.404087    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.404458 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.404388    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-web-config\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.404701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.404680    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.404788 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.404686    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b55a053e-d8e1-4d15-afbf-76e928fc5d47-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.405417 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.405386    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.406036 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.406010    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b55a053e-d8e1-4d15-afbf-76e928fc5d47-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.406098 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.406086    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.406357 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.406315    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.407006 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.406990    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b55a053e-d8e1-4d15-afbf-76e928fc5d47-config\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.418212 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.418160    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4dsc\" (UniqueName: \"kubernetes.io/projected/b55a053e-d8e1-4d15-afbf-76e928fc5d47-kube-api-access-x4dsc\") pod \"prometheus-k8s-0\" (UID: \"b55a053e-d8e1-4d15-afbf-76e928fc5d47\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.545735 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.545700    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:14:30.692377 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:30.692289    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 15:14:30.695781 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:14:30.695755    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb55a053e_d8e1_4d15_afbf_76e928fc5d47.slice/crio-f8783cc456fb6c6316b086787861c69c7ce28e2df555000c955b4977a48ac4fd WatchSource:0}: Error finding container f8783cc456fb6c6316b086787861c69c7ce28e2df555000c955b4977a48ac4fd: Status 404 returned error can't find the container with id f8783cc456fb6c6316b086787861c69c7ce28e2df555000c955b4977a48ac4fd
Jul 29 15:14:31.164466 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:31.164439    2575 generic.go:356] "Generic (PLEG): container finished" podID="b55a053e-d8e1-4d15-afbf-76e928fc5d47" containerID="a42806d17ae0f1785293beaefbbdfdc526d4b622b574e7ce7d471136064a0be0" exitCode=0
Jul 29 15:14:31.164615 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:31.164510    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b55a053e-d8e1-4d15-afbf-76e928fc5d47","Type":"ContainerDied","Data":"a42806d17ae0f1785293beaefbbdfdc526d4b622b574e7ce7d471136064a0be0"}
Jul 29 15:14:31.164615 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:31.164572    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b55a053e-d8e1-4d15-afbf-76e928fc5d47","Type":"ContainerStarted","Data":"f8783cc456fb6c6316b086787861c69c7ce28e2df555000c955b4977a48ac4fd"}
Jul 29 15:14:31.294727 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:31.294702    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c618a894-ab7d-4d1c-a493-23f63a6cfa55" path="/var/lib/kubelet/pods/c618a894-ab7d-4d1c-a493-23f63a6cfa55/volumes"
Jul 29 15:14:32.173399 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:32.173362    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b55a053e-d8e1-4d15-afbf-76e928fc5d47","Type":"ContainerStarted","Data":"bcc73ab22f1d1199b9e5c52af10476de16baf8bf1ae2eb9eb59d869d62191251"}
Jul 29 15:14:32.173763 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:32.173462    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b55a053e-d8e1-4d15-afbf-76e928fc5d47","Type":"ContainerStarted","Data":"74bb017f60aee9e01767ef717fb2c6069940cb3bc836e598c5822bde842c5d46"}
Jul 29 15:14:32.173763 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:32.173491    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b55a053e-d8e1-4d15-afbf-76e928fc5d47","Type":"ContainerStarted","Data":"0f55a92351335b6f727c7410836506a57ff9fdcebcbe21197260e32a1c23f7a1"}
Jul 29 15:14:32.173763 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:32.173523    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b55a053e-d8e1-4d15-afbf-76e928fc5d47","Type":"ContainerStarted","Data":"57b928cb56a743cfcba7ed802f9b88048e870e628f2f85de1d22f7199d1a23cc"}
Jul 29 15:14:32.173763 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:32.173556    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b55a053e-d8e1-4d15-afbf-76e928fc5d47","Type":"ContainerStarted","Data":"abd154d3f1ed51142748d261c2a9f57539183eadefb2bad0093cdb368ebd60cb"}
Jul 29 15:14:32.173763 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:32.173581    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b55a053e-d8e1-4d15-afbf-76e928fc5d47","Type":"ContainerStarted","Data":"ffb6886f32733c9f9252f30480926993ccbce7a9c54a6a956d4e2c6918f8a8f3"}
Jul 29 15:14:32.210756 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:32.210711    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.189876833 podStartE2EDuration="2.210695892s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.819059ms" isStatefulPod=false podCreationTimestamp="2026-07-29 15:14:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:14:32.209682625 +0000 UTC m=+205.358976525" watchObservedRunningTime="2026-07-29 15:14:32.210695892 +0000 UTC m=+205.359989792"
Jul 29 15:14:35.546473 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:14:35.546434    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:15:30.546215 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:15:30.546118    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:15:30.560956 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:15:30.560931    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:15:31.390988 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:15:31.390961    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 15:16:07.192723 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:07.192692    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:16:07.198869 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:07.198841    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:16:07.199884 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:07.199861    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:16:07.203383 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:07.203363    2575 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 15:16:07.205536 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:07.205514    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:16:20.609848 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:20.609812    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zh44m"]
Jul 29 15:16:20.613107 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:20.613088    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-zh44m"
Jul 29 15:16:20.615386 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:20.615364    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-zm6tm\""
Jul 29 15:16:20.615544 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:20.615364    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 29 15:16:20.615954 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:20.615928    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 29 15:16:20.623058 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:20.623035    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zh44m"]
Jul 29 15:16:20.714925 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:20.714887    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5xsp\" (UniqueName: \"kubernetes.io/projected/710a59d6-0396-422d-84fc-903b15421633-kube-api-access-s5xsp\") pod \"cert-manager-55b4d954d5-zh44m\" (UID: \"710a59d6-0396-422d-84fc-903b15421633\") " pod="cert-manager/cert-manager-55b4d954d5-zh44m"
Jul 29 15:16:20.715104 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:20.714932    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/710a59d6-0396-422d-84fc-903b15421633-bound-sa-token\") pod \"cert-manager-55b4d954d5-zh44m\" (UID: \"710a59d6-0396-422d-84fc-903b15421633\") " pod="cert-manager/cert-manager-55b4d954d5-zh44m"
Jul 29 15:16:20.815582 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:20.815543    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/710a59d6-0396-422d-84fc-903b15421633-bound-sa-token\") pod \"cert-manager-55b4d954d5-zh44m\" (UID: \"710a59d6-0396-422d-84fc-903b15421633\") " pod="cert-manager/cert-manager-55b4d954d5-zh44m"
Jul 29 15:16:20.815803 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:20.815629    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5xsp\" (UniqueName: \"kubernetes.io/projected/710a59d6-0396-422d-84fc-903b15421633-kube-api-access-s5xsp\") pod \"cert-manager-55b4d954d5-zh44m\" (UID: \"710a59d6-0396-422d-84fc-903b15421633\") " pod="cert-manager/cert-manager-55b4d954d5-zh44m"
Jul 29 15:16:20.825655 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:20.825614    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/710a59d6-0396-422d-84fc-903b15421633-bound-sa-token\") pod \"cert-manager-55b4d954d5-zh44m\" (UID: \"710a59d6-0396-422d-84fc-903b15421633\") " pod="cert-manager/cert-manager-55b4d954d5-zh44m"
Jul 29 15:16:20.825798 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:20.825684    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5xsp\" (UniqueName: \"kubernetes.io/projected/710a59d6-0396-422d-84fc-903b15421633-kube-api-access-s5xsp\") pod \"cert-manager-55b4d954d5-zh44m\" (UID: \"710a59d6-0396-422d-84fc-903b15421633\") " pod="cert-manager/cert-manager-55b4d954d5-zh44m"
Jul 29 15:16:20.922927 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:20.922828    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-zh44m"
Jul 29 15:16:21.056258 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:21.056224    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-zh44m"]
Jul 29 15:16:21.059831 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:16:21.059803    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod710a59d6_0396_422d_84fc_903b15421633.slice/crio-b14ed68d7aed8b22974f685b5632e8be71d5aa02badcfc2ef22a3a480f9e2b6e WatchSource:0}: Error finding container b14ed68d7aed8b22974f685b5632e8be71d5aa02badcfc2ef22a3a480f9e2b6e: Status 404 returned error can't find the container with id b14ed68d7aed8b22974f685b5632e8be71d5aa02badcfc2ef22a3a480f9e2b6e
Jul 29 15:16:21.061837 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:21.061820    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 15:16:21.553518 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:21.553483    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-zh44m" event={"ID":"710a59d6-0396-422d-84fc-903b15421633","Type":"ContainerStarted","Data":"b14ed68d7aed8b22974f685b5632e8be71d5aa02badcfc2ef22a3a480f9e2b6e"}
Jul 29 15:16:24.568660 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:24.568621    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-zh44m" event={"ID":"710a59d6-0396-422d-84fc-903b15421633","Type":"ContainerStarted","Data":"0a1bf6ef5d08741768817dd0060429bb73404ec4538a38cc670c7389139ffba6"}
Jul 29 15:16:24.586312 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:24.586256    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-zh44m" podStartSLOduration=1.627534794 podStartE2EDuration="4.586240868s" totalImagesPullingTime="2.958706074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:16:24.585779584 +0000 UTC m=+317.735073485" watchObservedRunningTime="2026-07-29 15:16:24.586240868 +0000 UTC m=+317.735534767"
Jul 29 15:16:34.365991 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.365954    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"]
Jul 29 15:16:34.369091 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.369074    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:34.377445 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.377419    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 15:16:34.377613 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.377547    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 15:16:34.377613 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.377558    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 15:16:34.377744 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.377648    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-djg4x\""
Jul 29 15:16:34.380541 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.380522    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 15:16:34.392759 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.392725    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"]
Jul 29 15:16:34.432990 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.432950    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-rfhp8\" (UID: \"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:34.432990 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.432991    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxx64\" (UniqueName: \"kubernetes.io/projected/539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0-kube-api-access-cxx64\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-rfhp8\" (UID: \"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:34.433201 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.433058    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0-webhook-cert\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-rfhp8\" (UID: \"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:34.433201 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.433093    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0-manifests\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-rfhp8\" (UID: \"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:34.433201 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.433111    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0-tmp\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-rfhp8\" (UID: \"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:34.533784 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.533741    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-rfhp8\" (UID: \"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:34.533784 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.533778    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxx64\" (UniqueName: \"kubernetes.io/projected/539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0-kube-api-access-cxx64\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-rfhp8\" (UID: \"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:34.534045 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.533801    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0-webhook-cert\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-rfhp8\" (UID: \"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:34.534045 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.533913    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0-manifests\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-rfhp8\" (UID: \"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:34.534045 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.533945    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0-tmp\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-rfhp8\" (UID: \"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:34.534344 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.534297    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0-manifests\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-rfhp8\" (UID: \"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:34.536227 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.536206    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0-tmp\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-rfhp8\" (UID: \"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:34.536333 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.536264    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-rfhp8\" (UID: \"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:34.536401 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.536338    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0-webhook-cert\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-rfhp8\" (UID: \"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:34.544604 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.544578    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxx64\" (UniqueName: \"kubernetes.io/projected/539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0-kube-api-access-cxx64\") pod \"opendatahub-operator-controller-manager-85b6fb79c5-rfhp8\" (UID: \"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0\") " pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:34.679497 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.679409    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:34.816864 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:34.816823    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"]
Jul 29 15:16:34.823466 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:16:34.821556    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod539a12a3_ac2e_4ce2_abb6_ebc1d6a1a1d0.slice/crio-e9134a80ee191d4ce207c74eb82f24089ca5bf8304ef0c73415d276f0a037e07 WatchSource:0}: Error finding container e9134a80ee191d4ce207c74eb82f24089ca5bf8304ef0c73415d276f0a037e07: Status 404 returned error can't find the container with id e9134a80ee191d4ce207c74eb82f24089ca5bf8304ef0c73415d276f0a037e07
Jul 29 15:16:35.609401 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:35.609356    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8" event={"ID":"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0","Type":"ContainerStarted","Data":"e9134a80ee191d4ce207c74eb82f24089ca5bf8304ef0c73415d276f0a037e07"}
Jul 29 15:16:38.623303 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:38.623269    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8" event={"ID":"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0","Type":"ContainerStarted","Data":"d1e0d748e97f16d10d9a455b6bc71acf2a66968c8fb613aa13a71dfe3848e91e"}
Jul 29 15:16:39.629275 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:39.629234    2575 generic.go:356] "Generic (PLEG): container finished" podID="539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0" containerID="d1e0d748e97f16d10d9a455b6bc71acf2a66968c8fb613aa13a71dfe3848e91e" exitCode=0
Jul 29 15:16:39.629688 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:39.629335    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8" event={"ID":"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0","Type":"ContainerDied","Data":"d1e0d748e97f16d10d9a455b6bc71acf2a66968c8fb613aa13a71dfe3848e91e"}
Jul 29 15:16:40.635485 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:40.635441    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8" event={"ID":"539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0","Type":"ContainerStarted","Data":"4d0cf9cd45315d2191131b556d927ddad82aade46c8e74f057c0a40634d1955d"}
Jul 29 15:16:40.635958 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:40.635732    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:40.657520 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:40.657460    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8" podStartSLOduration=2.203517243 podStartE2EDuration="6.657441168s" totalImagesPullingTime="3.353161154s" totalInitContainerRuntime="1.100762771s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:16:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:16:40.656407379 +0000 UTC m=+333.805701279" watchObservedRunningTime="2026-07-29 15:16:40.657441168 +0000 UTC m=+333.806735069"
Jul 29 15:16:51.641072 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:51.641040    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-85b6fb79c5-rfhp8"
Jul 29 15:16:55.863564 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:55.863522    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-688955555-v6nlm"]
Jul 29 15:16:55.880954 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:55.880921    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-688955555-v6nlm"
Jul 29 15:16:55.881350 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:55.881307    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-688955555-v6nlm"]
Jul 29 15:16:55.885277 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:55.885257    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 15:16:55.885629 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:55.885612    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 15:16:55.885852 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:55.885834    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 15:16:55.888221 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:55.888192    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 15:16:55.910887 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:55.910854    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4142a983-ca2f-476f-806d-21efb849144e-tmp\") pod \"kube-auth-proxy-688955555-v6nlm\" (UID: \"4142a983-ca2f-476f-806d-21efb849144e\") " pod="openshift-ingress/kube-auth-proxy-688955555-v6nlm"
Jul 29 15:16:55.911059 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:55.910917    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5lp8\" (UniqueName: \"kubernetes.io/projected/4142a983-ca2f-476f-806d-21efb849144e-kube-api-access-v5lp8\") pod \"kube-auth-proxy-688955555-v6nlm\" (UID: \"4142a983-ca2f-476f-806d-21efb849144e\") " pod="openshift-ingress/kube-auth-proxy-688955555-v6nlm"
Jul 29 15:16:55.911059 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:55.911025    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4142a983-ca2f-476f-806d-21efb849144e-tls-certs\") pod \"kube-auth-proxy-688955555-v6nlm\" (UID: \"4142a983-ca2f-476f-806d-21efb849144e\") " pod="openshift-ingress/kube-auth-proxy-688955555-v6nlm"
Jul 29 15:16:56.000267 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.000228    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vspkj"]
Jul 29 15:16:56.003624 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.003601    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-vspkj"
Jul 29 15:16:56.005726 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.005696    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 29 15:16:56.005847 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.005755    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-dgkzp\""
Jul 29 15:16:56.011697 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.011676    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf6pw\" (UniqueName: \"kubernetes.io/projected/146aecc1-dbef-4fd8-9ba9-f9425d7f91c0-kube-api-access-mf6pw\") pod \"odh-model-controller-7998c67dc4-vspkj\" (UID: \"146aecc1-dbef-4fd8-9ba9-f9425d7f91c0\") " pod="opendatahub/odh-model-controller-7998c67dc4-vspkj"
Jul 29 15:16:56.011814 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.011727    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4142a983-ca2f-476f-806d-21efb849144e-tmp\") pod \"kube-auth-proxy-688955555-v6nlm\" (UID: \"4142a983-ca2f-476f-806d-21efb849144e\") " pod="openshift-ingress/kube-auth-proxy-688955555-v6nlm"
Jul 29 15:16:56.011883 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.011811    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5lp8\" (UniqueName: \"kubernetes.io/projected/4142a983-ca2f-476f-806d-21efb849144e-kube-api-access-v5lp8\") pod \"kube-auth-proxy-688955555-v6nlm\" (UID: \"4142a983-ca2f-476f-806d-21efb849144e\") " pod="openshift-ingress/kube-auth-proxy-688955555-v6nlm"
Jul 29 15:16:56.011883 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.011868    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4142a983-ca2f-476f-806d-21efb849144e-tls-certs\") pod \"kube-auth-proxy-688955555-v6nlm\" (UID: \"4142a983-ca2f-476f-806d-21efb849144e\") " pod="openshift-ingress/kube-auth-proxy-688955555-v6nlm"
Jul 29 15:16:56.011990 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.011916    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/146aecc1-dbef-4fd8-9ba9-f9425d7f91c0-cert\") pod \"odh-model-controller-7998c67dc4-vspkj\" (UID: \"146aecc1-dbef-4fd8-9ba9-f9425d7f91c0\") " pod="opendatahub/odh-model-controller-7998c67dc4-vspkj"
Jul 29 15:16:56.013994 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.013968    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4142a983-ca2f-476f-806d-21efb849144e-tmp\") pod \"kube-auth-proxy-688955555-v6nlm\" (UID: \"4142a983-ca2f-476f-806d-21efb849144e\") " pod="openshift-ingress/kube-auth-proxy-688955555-v6nlm"
Jul 29 15:16:56.014231 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.014211    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4142a983-ca2f-476f-806d-21efb849144e-tls-certs\") pod \"kube-auth-proxy-688955555-v6nlm\" (UID: \"4142a983-ca2f-476f-806d-21efb849144e\") " pod="openshift-ingress/kube-auth-proxy-688955555-v6nlm"
Jul 29 15:16:56.018279 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.018256    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vspkj"]
Jul 29 15:16:56.031158 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.031112    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5lp8\" (UniqueName: \"kubernetes.io/projected/4142a983-ca2f-476f-806d-21efb849144e-kube-api-access-v5lp8\") pod \"kube-auth-proxy-688955555-v6nlm\" (UID: \"4142a983-ca2f-476f-806d-21efb849144e\") " pod="openshift-ingress/kube-auth-proxy-688955555-v6nlm"
Jul 29 15:16:56.112740 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.112700    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf6pw\" (UniqueName: \"kubernetes.io/projected/146aecc1-dbef-4fd8-9ba9-f9425d7f91c0-kube-api-access-mf6pw\") pod \"odh-model-controller-7998c67dc4-vspkj\" (UID: \"146aecc1-dbef-4fd8-9ba9-f9425d7f91c0\") " pod="opendatahub/odh-model-controller-7998c67dc4-vspkj"
Jul 29 15:16:56.112915 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.112777    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/146aecc1-dbef-4fd8-9ba9-f9425d7f91c0-cert\") pod \"odh-model-controller-7998c67dc4-vspkj\" (UID: \"146aecc1-dbef-4fd8-9ba9-f9425d7f91c0\") " pod="opendatahub/odh-model-controller-7998c67dc4-vspkj"
Jul 29 15:16:56.115090 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.115024    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/146aecc1-dbef-4fd8-9ba9-f9425d7f91c0-cert\") pod \"odh-model-controller-7998c67dc4-vspkj\" (UID: \"146aecc1-dbef-4fd8-9ba9-f9425d7f91c0\") " pod="opendatahub/odh-model-controller-7998c67dc4-vspkj"
Jul 29 15:16:56.129705 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.129682    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf6pw\" (UniqueName: \"kubernetes.io/projected/146aecc1-dbef-4fd8-9ba9-f9425d7f91c0-kube-api-access-mf6pw\") pod \"odh-model-controller-7998c67dc4-vspkj\" (UID: \"146aecc1-dbef-4fd8-9ba9-f9425d7f91c0\") " pod="opendatahub/odh-model-controller-7998c67dc4-vspkj"
Jul 29 15:16:56.191679 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.191647    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-688955555-v6nlm"
Jul 29 15:16:56.313954 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.313923    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-vspkj"
Jul 29 15:16:56.332142 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.332116    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-688955555-v6nlm"]
Jul 29 15:16:56.337311 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:16:56.337279    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4142a983_ca2f_476f_806d_21efb849144e.slice/crio-1d458f6380d172407b7c86d1037019df012433ff5bbdd5042c3a11a96baa9ccc WatchSource:0}: Error finding container 1d458f6380d172407b7c86d1037019df012433ff5bbdd5042c3a11a96baa9ccc: Status 404 returned error can't find the container with id 1d458f6380d172407b7c86d1037019df012433ff5bbdd5042c3a11a96baa9ccc
Jul 29 15:16:56.456981 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.456954    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-vspkj"]
Jul 29 15:16:56.459581 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:16:56.459558    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod146aecc1_dbef_4fd8_9ba9_f9425d7f91c0.slice/crio-7ad8d145f6825814cec8aac1ecc7702916e326c711863255cdb963fc851bbccf WatchSource:0}: Error finding container 7ad8d145f6825814cec8aac1ecc7702916e326c711863255cdb963fc851bbccf: Status 404 returned error can't find the container with id 7ad8d145f6825814cec8aac1ecc7702916e326c711863255cdb963fc851bbccf
Jul 29 15:16:56.695167 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.695090    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-688955555-v6nlm" event={"ID":"4142a983-ca2f-476f-806d-21efb849144e","Type":"ContainerStarted","Data":"1d458f6380d172407b7c86d1037019df012433ff5bbdd5042c3a11a96baa9ccc"}
Jul 29 15:16:56.696078 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:56.696061    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-vspkj" event={"ID":"146aecc1-dbef-4fd8-9ba9-f9425d7f91c0","Type":"ContainerStarted","Data":"7ad8d145f6825814cec8aac1ecc7702916e326c711863255cdb963fc851bbccf"}
Jul 29 15:16:57.100060 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.099987    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-57wds"]
Jul 29 15:16:57.104182 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.104166    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-57wds"
Jul 29 15:16:57.106467 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.106448    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 29 15:16:57.106596 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.106535    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-7lfxm\""
Jul 29 15:16:57.114107 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.114086    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-57wds"]
Jul 29 15:16:57.118979 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.118952    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghxwp\" (UniqueName: \"kubernetes.io/projected/59fd0b51-1e06-4c85-a9e0-154b7a178bb8-kube-api-access-ghxwp\") pod \"model-serving-api-54d89bd5fc-57wds\" (UID: \"59fd0b51-1e06-4c85-a9e0-154b7a178bb8\") " pod="opendatahub/model-serving-api-54d89bd5fc-57wds"
Jul 29 15:16:57.119090 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.119033    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59fd0b51-1e06-4c85-a9e0-154b7a178bb8-tls-certs\") pod \"model-serving-api-54d89bd5fc-57wds\" (UID: \"59fd0b51-1e06-4c85-a9e0-154b7a178bb8\") " pod="opendatahub/model-serving-api-54d89bd5fc-57wds"
Jul 29 15:16:57.220257 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.220224    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghxwp\" (UniqueName: \"kubernetes.io/projected/59fd0b51-1e06-4c85-a9e0-154b7a178bb8-kube-api-access-ghxwp\") pod \"model-serving-api-54d89bd5fc-57wds\" (UID: \"59fd0b51-1e06-4c85-a9e0-154b7a178bb8\") " pod="opendatahub/model-serving-api-54d89bd5fc-57wds"
Jul 29 15:16:57.220466 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.220301    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59fd0b51-1e06-4c85-a9e0-154b7a178bb8-tls-certs\") pod \"model-serving-api-54d89bd5fc-57wds\" (UID: \"59fd0b51-1e06-4c85-a9e0-154b7a178bb8\") " pod="opendatahub/model-serving-api-54d89bd5fc-57wds"
Jul 29 15:16:57.223180 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.223145    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59fd0b51-1e06-4c85-a9e0-154b7a178bb8-tls-certs\") pod \"model-serving-api-54d89bd5fc-57wds\" (UID: \"59fd0b51-1e06-4c85-a9e0-154b7a178bb8\") " pod="opendatahub/model-serving-api-54d89bd5fc-57wds"
Jul 29 15:16:57.239898 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.239867    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghxwp\" (UniqueName: \"kubernetes.io/projected/59fd0b51-1e06-4c85-a9e0-154b7a178bb8-kube-api-access-ghxwp\") pod \"model-serving-api-54d89bd5fc-57wds\" (UID: \"59fd0b51-1e06-4c85-a9e0-154b7a178bb8\") " pod="opendatahub/model-serving-api-54d89bd5fc-57wds"
Jul 29 15:16:57.335248 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.335214    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"]
Jul 29 15:16:57.338760 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.338740    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.340760 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.340741    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 29 15:16:57.340871 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.340825    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 29 15:16:57.340871 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.340855    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-99s7b\""
Jul 29 15:16:57.340871 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.340867    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 29 15:16:57.341110 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.341094    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 15:16:57.356619 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.354188    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"]
Jul 29 15:16:57.413965 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.413929    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-57wds"
Jul 29 15:16:57.422414 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.422388    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.422541 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.422429    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/84d69788-78ff-4d89-aa54-b940650993fb-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.422541 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.422459    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/84d69788-78ff-4d89-aa54-b940650993fb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.422687 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.422573    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84d69788-78ff-4d89-aa54-b940650993fb-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.422687 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.422609    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd9n6\" (UniqueName: \"kubernetes.io/projected/84d69788-78ff-4d89-aa54-b940650993fb-kube-api-access-pd9n6\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.422687 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.422643    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.422687 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.422678    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.523576 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.523510    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.523576 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.523558    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/84d69788-78ff-4d89-aa54-b940650993fb-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.523841 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.523589    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/84d69788-78ff-4d89-aa54-b940650993fb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.523841 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.523646    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84d69788-78ff-4d89-aa54-b940650993fb-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.523841 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.523671    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd9n6\" (UniqueName: \"kubernetes.io/projected/84d69788-78ff-4d89-aa54-b940650993fb-kube-api-access-pd9n6\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.523841 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.523705    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.523841 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.523740    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.525285 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.525261    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/84d69788-78ff-4d89-aa54-b940650993fb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.527990 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.527966    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.530313 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.530264    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.530892 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.530871    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.532626 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.532552    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/84d69788-78ff-4d89-aa54-b940650993fb-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.534502 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.534435    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84d69788-78ff-4d89-aa54-b940650993fb-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.536951 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.536849    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd9n6\" (UniqueName: \"kubernetes.io/projected/84d69788-78ff-4d89-aa54-b940650993fb-kube-api-access-pd9n6\") pod \"istiod-openshift-gateway-7b7995d5cd-dptjh\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.573173 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.573134    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-57wds"]
Jul 29 15:16:57.576389 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:16:57.576363    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59fd0b51_1e06_4c85_a9e0_154b7a178bb8.slice/crio-2ee31a786eccf94fe7943b89b24bd21df82515c99bcc7229eb4ea0c0b6155c39 WatchSource:0}: Error finding container 2ee31a786eccf94fe7943b89b24bd21df82515c99bcc7229eb4ea0c0b6155c39: Status 404 returned error can't find the container with id 2ee31a786eccf94fe7943b89b24bd21df82515c99bcc7229eb4ea0c0b6155c39
Jul 29 15:16:57.649870 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.649782    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:16:57.702778 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.702746    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-57wds" event={"ID":"59fd0b51-1e06-4c85-a9e0-154b7a178bb8","Type":"ContainerStarted","Data":"2ee31a786eccf94fe7943b89b24bd21df82515c99bcc7229eb4ea0c0b6155c39"}
Jul 29 15:16:57.798447 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:57.798416    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"]
Jul 29 15:16:57.802427 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:16:57.802396    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84d69788_78ff_4d89_aa54_b940650993fb.slice/crio-b588a0aaf059d7d19808badc66c36a14a4a666a8c1563475a01d530e9ba15b53 WatchSource:0}: Error finding container b588a0aaf059d7d19808badc66c36a14a4a666a8c1563475a01d530e9ba15b53: Status 404 returned error can't find the container with id b588a0aaf059d7d19808badc66c36a14a4a666a8c1563475a01d530e9ba15b53
Jul 29 15:16:58.712490 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:16:58.712406    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh" event={"ID":"84d69788-78ff-4d89-aa54-b940650993fb","Type":"ContainerStarted","Data":"b588a0aaf059d7d19808badc66c36a14a4a666a8c1563475a01d530e9ba15b53"}
Jul 29 15:17:02.518133 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:02.518100    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 29 15:17:02.518511 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:02.518162    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 29 15:17:02.737745 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:02.737699    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-688955555-v6nlm" event={"ID":"4142a983-ca2f-476f-806d-21efb849144e","Type":"ContainerStarted","Data":"57178225c4c52876fd435eff92962dc789df75e142cebf4550b0d9414455c4a6"}
Jul 29 15:17:02.739399 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:02.739361    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-vspkj" event={"ID":"146aecc1-dbef-4fd8-9ba9-f9425d7f91c0","Type":"ContainerStarted","Data":"9a4539009424c0a61847e268a98ded09172de8d64b479b637f36af10e0ceaecb"}
Jul 29 15:17:02.739545 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:02.739473    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-vspkj"
Jul 29 15:17:02.741035 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:02.741015    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-57wds" event={"ID":"59fd0b51-1e06-4c85-a9e0-154b7a178bb8","Type":"ContainerStarted","Data":"cb3ba02504e5e50417605a65d608ab3ef0ab42ad5a38fb979e59b23a96bdaf03"}
Jul 29 15:17:02.741161 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:02.741065    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-57wds"
Jul 29 15:17:02.742678 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:02.742648    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh" event={"ID":"84d69788-78ff-4d89-aa54-b940650993fb","Type":"ContainerStarted","Data":"ab578b4c49dfcf644f7ed100805a721a6108517ac51c8159dc73524969216630"}
Jul 29 15:17:02.742772 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:02.742762    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:17:02.755357 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:02.755281    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-688955555-v6nlm" podStartSLOduration=1.586279161 podStartE2EDuration="7.755266085s" totalImagesPullingTime="6.168986924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:16:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:17:02.753946348 +0000 UTC m=+355.903240250" watchObservedRunningTime="2026-07-29 15:17:02.755266085 +0000 UTC m=+355.904559986"
Jul 29 15:17:02.770975 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:02.770883    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-vspkj" podStartSLOduration=1.666759983 podStartE2EDuration="7.770865252s" totalImagesPullingTime="6.104105269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:16:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:17:02.770156781 +0000 UTC m=+355.919450681" watchObservedRunningTime="2026-07-29 15:17:02.770865252 +0000 UTC m=+355.920159154"
Jul 29 15:17:02.789382 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:02.789335    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-57wds" podStartSLOduration=0.859865535 podStartE2EDuration="5.789306503s" totalImagesPullingTime="4.929440968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:17:02.788478351 +0000 UTC m=+355.937772256" watchObservedRunningTime="2026-07-29 15:17:02.789306503 +0000 UTC m=+355.938600404"
Jul 29 15:17:02.809900 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:02.809847    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh" podStartSLOduration=1.096736931 podStartE2EDuration="5.809828078s" totalImagesPullingTime="4.713091147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:17:02.808660245 +0000 UTC m=+355.957954162" watchObservedRunningTime="2026-07-29 15:17:02.809828078 +0000 UTC m=+355.959121980"
Jul 29 15:17:03.748995 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:03.748965    2575 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-dptjh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 15:17:03.749397 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:03.749013    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh" podUID="84d69788-78ff-4d89-aa54-b940650993fb" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 15:17:06.748530 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:06.748502    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:17:07.138374 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.138338    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gjvg9"]
Jul 29 15:17:07.141877 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.141854    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hs8s2"]
Jul 29 15:17:07.141985 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.141958    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gjvg9"
Jul 29 15:17:07.144253 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.144232    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-z86xd\""
Jul 29 15:17:07.144593 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.144577    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 29 15:17:07.145183 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.145165    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hs8s2"
Jul 29 15:17:07.147209 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.147173    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 29 15:17:07.147209 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.147183    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-xxbzc\""
Jul 29 15:17:07.154112 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.154091    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gjvg9"]
Jul 29 15:17:07.159238 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.159218    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hs8s2"]
Jul 29 15:17:07.212482 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.212453    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnr29\" (UniqueName: \"kubernetes.io/projected/54c563b3-de7c-47de-aa91-a909bcad32a4-kube-api-access-xnr29\") pod \"llmisvc-controller-manager-777c6bd7d5-hs8s2\" (UID: \"54c563b3-de7c-47de-aa91-a909bcad32a4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hs8s2"
Jul 29 15:17:07.212650 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.212508    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5cb24fc-8cca-4ff2-bf3c-3269c685b3df-cert\") pod \"kserve-controller-manager-d6f4b65b8-gjvg9\" (UID: \"e5cb24fc-8cca-4ff2-bf3c-3269c685b3df\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gjvg9"
Jul 29 15:17:07.212650 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.212581    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gftlz\" (UniqueName: \"kubernetes.io/projected/e5cb24fc-8cca-4ff2-bf3c-3269c685b3df-kube-api-access-gftlz\") pod \"kserve-controller-manager-d6f4b65b8-gjvg9\" (UID: \"e5cb24fc-8cca-4ff2-bf3c-3269c685b3df\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gjvg9"
Jul 29 15:17:07.212650 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.212616    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54c563b3-de7c-47de-aa91-a909bcad32a4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hs8s2\" (UID: \"54c563b3-de7c-47de-aa91-a909bcad32a4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hs8s2"
Jul 29 15:17:07.313974 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.313949    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5cb24fc-8cca-4ff2-bf3c-3269c685b3df-cert\") pod \"kserve-controller-manager-d6f4b65b8-gjvg9\" (UID: \"e5cb24fc-8cca-4ff2-bf3c-3269c685b3df\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gjvg9"
Jul 29 15:17:07.314116 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.313989    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gftlz\" (UniqueName: \"kubernetes.io/projected/e5cb24fc-8cca-4ff2-bf3c-3269c685b3df-kube-api-access-gftlz\") pod \"kserve-controller-manager-d6f4b65b8-gjvg9\" (UID: \"e5cb24fc-8cca-4ff2-bf3c-3269c685b3df\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gjvg9"
Jul 29 15:17:07.314116 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.314012    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54c563b3-de7c-47de-aa91-a909bcad32a4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hs8s2\" (UID: \"54c563b3-de7c-47de-aa91-a909bcad32a4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hs8s2"
Jul 29 15:17:07.314199 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.314166    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnr29\" (UniqueName: \"kubernetes.io/projected/54c563b3-de7c-47de-aa91-a909bcad32a4-kube-api-access-xnr29\") pod \"llmisvc-controller-manager-777c6bd7d5-hs8s2\" (UID: \"54c563b3-de7c-47de-aa91-a909bcad32a4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hs8s2"
Jul 29 15:17:07.316800 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.316781    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 29 15:17:07.316800 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.316795    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 29 15:17:07.324774 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:17:07.324749    2575 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 29 15:17:07.324888 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.324787    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gftlz\" (UniqueName: \"kubernetes.io/projected/e5cb24fc-8cca-4ff2-bf3c-3269c685b3df-kube-api-access-gftlz\") pod \"kserve-controller-manager-d6f4b65b8-gjvg9\" (UID: \"e5cb24fc-8cca-4ff2-bf3c-3269c685b3df\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gjvg9"
Jul 29 15:17:07.324888 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:17:07.324836    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54c563b3-de7c-47de-aa91-a909bcad32a4-cert podName:54c563b3-de7c-47de-aa91-a909bcad32a4 nodeName:}" failed. No retries permitted until 2026-07-29 15:17:07.824815818 +0000 UTC m=+360.974109701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/54c563b3-de7c-47de-aa91-a909bcad32a4-cert") pod "llmisvc-controller-manager-777c6bd7d5-hs8s2" (UID: "54c563b3-de7c-47de-aa91-a909bcad32a4") : secret "llmisvc-webhook-server-cert" not found
Jul 29 15:17:07.324888 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.324864    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnr29\" (UniqueName: \"kubernetes.io/projected/54c563b3-de7c-47de-aa91-a909bcad32a4-kube-api-access-xnr29\") pod \"llmisvc-controller-manager-777c6bd7d5-hs8s2\" (UID: \"54c563b3-de7c-47de-aa91-a909bcad32a4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hs8s2"
Jul 29 15:17:07.326291 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.326266    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e5cb24fc-8cca-4ff2-bf3c-3269c685b3df-cert\") pod \"kserve-controller-manager-d6f4b65b8-gjvg9\" (UID: \"e5cb24fc-8cca-4ff2-bf3c-3269c685b3df\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gjvg9"
Jul 29 15:17:07.458868 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.458791    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-z86xd\""
Jul 29 15:17:07.466387 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.466370    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gjvg9"
Jul 29 15:17:07.604868 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.604845    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gjvg9"]
Jul 29 15:17:07.606578 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:17:07.606552    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5cb24fc_8cca_4ff2_bf3c_3269c685b3df.slice/crio-6e25a68ac19075bf326612cdc2732615346394fda508068e2d751a4fee2437a6 WatchSource:0}: Error finding container 6e25a68ac19075bf326612cdc2732615346394fda508068e2d751a4fee2437a6: Status 404 returned error can't find the container with id 6e25a68ac19075bf326612cdc2732615346394fda508068e2d751a4fee2437a6
Jul 29 15:17:07.767221 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.767129    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gjvg9" event={"ID":"e5cb24fc-8cca-4ff2-bf3c-3269c685b3df","Type":"ContainerStarted","Data":"6e25a68ac19075bf326612cdc2732615346394fda508068e2d751a4fee2437a6"}
Jul 29 15:17:07.919022 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.918989    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54c563b3-de7c-47de-aa91-a909bcad32a4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hs8s2\" (UID: \"54c563b3-de7c-47de-aa91-a909bcad32a4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hs8s2"
Jul 29 15:17:07.921367 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:07.921346    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/54c563b3-de7c-47de-aa91-a909bcad32a4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hs8s2\" (UID: \"54c563b3-de7c-47de-aa91-a909bcad32a4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hs8s2"
Jul 29 15:17:08.066855 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:08.066781    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-xxbzc\""
Jul 29 15:17:08.074348 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:08.074312    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hs8s2"
Jul 29 15:17:08.225022 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:08.224927    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hs8s2"]
Jul 29 15:17:08.227560 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:17:08.227507    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod54c563b3_de7c_47de_aa91_a909bcad32a4.slice/crio-1cc7dddc107aba988602ea83af8267f364a5950748048a5fda4622bc83364343 WatchSource:0}: Error finding container 1cc7dddc107aba988602ea83af8267f364a5950748048a5fda4622bc83364343: Status 404 returned error can't find the container with id 1cc7dddc107aba988602ea83af8267f364a5950748048a5fda4622bc83364343
Jul 29 15:17:08.773565 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:08.773535    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hs8s2" event={"ID":"54c563b3-de7c-47de-aa91-a909bcad32a4","Type":"ContainerStarted","Data":"1cc7dddc107aba988602ea83af8267f364a5950748048a5fda4622bc83364343"}
Jul 29 15:17:10.096676 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.096583    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"]
Jul 29 15:17:10.097112 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.096817    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh" podUID="84d69788-78ff-4d89-aa54-b940650993fb" containerName="discovery" containerID="cri-o://ab578b4c49dfcf644f7ed100805a721a6108517ac51c8159dc73524969216630" gracePeriod=30
Jul 29 15:17:10.347986 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.347932    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:17:10.440599 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.440564    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd9n6\" (UniqueName: \"kubernetes.io/projected/84d69788-78ff-4d89-aa54-b940650993fb-kube-api-access-pd9n6\") pod \"84d69788-78ff-4d89-aa54-b940650993fb\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") "
Jul 29 15:17:10.440766 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.440609    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84d69788-78ff-4d89-aa54-b940650993fb-istio-token\") pod \"84d69788-78ff-4d89-aa54-b940650993fb\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") "
Jul 29 15:17:10.440766 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.440684    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-istio-csr-dns-cert\") pod \"84d69788-78ff-4d89-aa54-b940650993fb\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") "
Jul 29 15:17:10.440874 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.440805    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/84d69788-78ff-4d89-aa54-b940650993fb-local-certs\") pod \"84d69788-78ff-4d89-aa54-b940650993fb\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") "
Jul 29 15:17:10.440874 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.440836    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-cacerts\") pod \"84d69788-78ff-4d89-aa54-b940650993fb\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") "
Jul 29 15:17:10.440953 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.440873    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/84d69788-78ff-4d89-aa54-b940650993fb-istio-csr-ca-configmap\") pod \"84d69788-78ff-4d89-aa54-b940650993fb\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") "
Jul 29 15:17:10.440953 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.440898    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-istio-kubeconfig\") pod \"84d69788-78ff-4d89-aa54-b940650993fb\" (UID: \"84d69788-78ff-4d89-aa54-b940650993fb\") "
Jul 29 15:17:10.441524 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.441461    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84d69788-78ff-4d89-aa54-b940650993fb-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "84d69788-78ff-4d89-aa54-b940650993fb" (UID: "84d69788-78ff-4d89-aa54-b940650993fb"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:17:10.443611 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.443572    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "84d69788-78ff-4d89-aa54-b940650993fb" (UID: "84d69788-78ff-4d89-aa54-b940650993fb"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:17:10.443611 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.443590    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84d69788-78ff-4d89-aa54-b940650993fb-kube-api-access-pd9n6" (OuterVolumeSpecName: "kube-api-access-pd9n6") pod "84d69788-78ff-4d89-aa54-b940650993fb" (UID: "84d69788-78ff-4d89-aa54-b940650993fb"). InnerVolumeSpecName "kube-api-access-pd9n6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:17:10.443769 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.443624    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-cacerts" (OuterVolumeSpecName: "cacerts") pod "84d69788-78ff-4d89-aa54-b940650993fb" (UID: "84d69788-78ff-4d89-aa54-b940650993fb"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:17:10.444098 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.444068    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84d69788-78ff-4d89-aa54-b940650993fb-local-certs" (OuterVolumeSpecName: "local-certs") pod "84d69788-78ff-4d89-aa54-b940650993fb" (UID: "84d69788-78ff-4d89-aa54-b940650993fb"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:17:10.444226 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.444196    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "84d69788-78ff-4d89-aa54-b940650993fb" (UID: "84d69788-78ff-4d89-aa54-b940650993fb"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:17:10.444597 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.444574    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84d69788-78ff-4d89-aa54-b940650993fb-istio-token" (OuterVolumeSpecName: "istio-token") pod "84d69788-78ff-4d89-aa54-b940650993fb" (UID: "84d69788-78ff-4d89-aa54-b940650993fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:17:10.541832 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.541804    2575 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/84d69788-78ff-4d89-aa54-b940650993fb-local-certs\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:17:10.541832 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.541828    2575 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-cacerts\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:17:10.541832 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.541836    2575 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/84d69788-78ff-4d89-aa54-b940650993fb-istio-csr-ca-configmap\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:17:10.542022 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.541845    2575 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-istio-kubeconfig\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:17:10.542022 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.541853    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd9n6\" (UniqueName: \"kubernetes.io/projected/84d69788-78ff-4d89-aa54-b940650993fb-kube-api-access-pd9n6\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:17:10.542022 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.541861    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84d69788-78ff-4d89-aa54-b940650993fb-istio-token\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:17:10.542022 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.541869    2575 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/84d69788-78ff-4d89-aa54-b940650993fb-istio-csr-dns-cert\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:17:10.786177 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.786136    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gjvg9" event={"ID":"e5cb24fc-8cca-4ff2-bf3c-3269c685b3df","Type":"ContainerStarted","Data":"6292b15468753063c106ee9d27af8124903322923bfaadd601c45daec8a79db1"}
Jul 29 15:17:10.786358 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.786245    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gjvg9"
Jul 29 15:17:10.787526 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.787498    2575 generic.go:356] "Generic (PLEG): container finished" podID="84d69788-78ff-4d89-aa54-b940650993fb" containerID="ab578b4c49dfcf644f7ed100805a721a6108517ac51c8159dc73524969216630" exitCode=0
Jul 29 15:17:10.787655 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.787532    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"
Jul 29 15:17:10.787655 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.787572    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh" event={"ID":"84d69788-78ff-4d89-aa54-b940650993fb","Type":"ContainerDied","Data":"ab578b4c49dfcf644f7ed100805a721a6108517ac51c8159dc73524969216630"}
Jul 29 15:17:10.787655 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.787605    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh" event={"ID":"84d69788-78ff-4d89-aa54-b940650993fb","Type":"ContainerDied","Data":"b588a0aaf059d7d19808badc66c36a14a4a666a8c1563475a01d530e9ba15b53"}
Jul 29 15:17:10.787655 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.787626    2575 scope.go:117] "RemoveContainer" containerID="ab578b4c49dfcf644f7ed100805a721a6108517ac51c8159dc73524969216630"
Jul 29 15:17:10.811199 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.811159    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gjvg9" podStartSLOduration=1.604956289 podStartE2EDuration="3.811139859s" totalImagesPullingTime="2.20618357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:17:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:17:10.80982394 +0000 UTC m=+363.959117834" watchObservedRunningTime="2026-07-29 15:17:10.811139859 +0000 UTC m=+363.960433759"
Jul 29 15:17:10.832688 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.832655    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"]
Jul 29 15:17:10.839547 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.839520    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-dptjh"]
Jul 29 15:17:10.860943 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.860800    2575 scope.go:117] "RemoveContainer" containerID="ab578b4c49dfcf644f7ed100805a721a6108517ac51c8159dc73524969216630"
Jul 29 15:17:10.861188 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:17:10.861168    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab578b4c49dfcf644f7ed100805a721a6108517ac51c8159dc73524969216630\": container with ID starting with ab578b4c49dfcf644f7ed100805a721a6108517ac51c8159dc73524969216630 not found: ID does not exist" containerID="ab578b4c49dfcf644f7ed100805a721a6108517ac51c8159dc73524969216630"
Jul 29 15:17:10.861252 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:10.861199    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab578b4c49dfcf644f7ed100805a721a6108517ac51c8159dc73524969216630"} err="failed to get container status \"ab578b4c49dfcf644f7ed100805a721a6108517ac51c8159dc73524969216630\": rpc error: code = NotFound desc = could not find container \"ab578b4c49dfcf644f7ed100805a721a6108517ac51c8159dc73524969216630\": container with ID starting with ab578b4c49dfcf644f7ed100805a721a6108517ac51c8159dc73524969216630 not found: ID does not exist"
Jul 29 15:17:11.295559 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:11.295522    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84d69788-78ff-4d89-aa54-b940650993fb" path="/var/lib/kubelet/pods/84d69788-78ff-4d89-aa54-b940650993fb/volumes"
Jul 29 15:17:11.793107 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:11.793074    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hs8s2" event={"ID":"54c563b3-de7c-47de-aa91-a909bcad32a4","Type":"ContainerStarted","Data":"44e45bb881ad90fc0c9f75337104aa7195584f6d42ff7e4ff750b3469e4884ce"}
Jul 29 15:17:11.793291 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:11.793188    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hs8s2"
Jul 29 15:17:11.812478 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:11.812432    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hs8s2" podStartSLOduration=1.513360032 podStartE2EDuration="4.812418795s" totalImagesPullingTime="3.299058763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:17:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:17:11.811188558 +0000 UTC m=+364.960482471" watchObservedRunningTime="2026-07-29 15:17:11.812418795 +0000 UTC m=+364.961712695"
Jul 29 15:17:13.749650 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:13.749619    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-vspkj"
Jul 29 15:17:13.751707 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:13.751687    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-57wds"
Jul 29 15:17:41.798929 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:41.798895    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gjvg9"
Jul 29 15:17:42.801388 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:42.801356    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hs8s2"
Jul 29 15:17:57.099491 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.099456    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fc6pw"]
Jul 29 15:17:57.099936 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.099843    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84d69788-78ff-4d89-aa54-b940650993fb" containerName="discovery"
Jul 29 15:17:57.099936 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.099861    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="84d69788-78ff-4d89-aa54-b940650993fb" containerName="discovery"
Jul 29 15:17:57.099936 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.099927    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="84d69788-78ff-4d89-aa54-b940650993fb" containerName="discovery"
Jul 29 15:17:57.111999 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.111977    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fc6pw"
Jul 29 15:17:57.115472 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.115444    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fc6pw"]
Jul 29 15:17:57.115810 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.115794    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 15:17:57.116470 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.116453    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 15:17:57.116577 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.116471    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-hkl5k\""
Jul 29 15:17:57.227132 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.227108    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6txq9\" (UniqueName: \"kubernetes.io/projected/2a95e5e2-cae9-432e-b569-aca1c5afe31a-kube-api-access-6txq9\") pod \"kuadrant-operator-catalog-fc6pw\" (UID: \"2a95e5e2-cae9-432e-b569-aca1c5afe31a\") " pod="kuadrant-system/kuadrant-operator-catalog-fc6pw"
Jul 29 15:17:57.327579 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.327554    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6txq9\" (UniqueName: \"kubernetes.io/projected/2a95e5e2-cae9-432e-b569-aca1c5afe31a-kube-api-access-6txq9\") pod \"kuadrant-operator-catalog-fc6pw\" (UID: \"2a95e5e2-cae9-432e-b569-aca1c5afe31a\") " pod="kuadrant-system/kuadrant-operator-catalog-fc6pw"
Jul 29 15:17:57.338331 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.338297    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6txq9\" (UniqueName: \"kubernetes.io/projected/2a95e5e2-cae9-432e-b569-aca1c5afe31a-kube-api-access-6txq9\") pod \"kuadrant-operator-catalog-fc6pw\" (UID: \"2a95e5e2-cae9-432e-b569-aca1c5afe31a\") " pod="kuadrant-system/kuadrant-operator-catalog-fc6pw"
Jul 29 15:17:57.416333 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.416289    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fc6pw"]
Jul 29 15:17:57.416605 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.416590    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fc6pw"
Jul 29 15:17:57.539755 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.539725    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fc6pw"]
Jul 29 15:17:57.542777 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:17:57.542751    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a95e5e2_cae9_432e_b569_aca1c5afe31a.slice/crio-8a5f7f3a356e611718e65712468d2473fa1ec38db5150dd1e096aae3237b9760 WatchSource:0}: Error finding container 8a5f7f3a356e611718e65712468d2473fa1ec38db5150dd1e096aae3237b9760: Status 404 returned error can't find the container with id 8a5f7f3a356e611718e65712468d2473fa1ec38db5150dd1e096aae3237b9760
Jul 29 15:17:57.627831 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.627801    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5zjdr"]
Jul 29 15:17:57.632058 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.632042    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5zjdr"
Jul 29 15:17:57.645018 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.644997    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5zjdr"]
Jul 29 15:17:57.730681 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.730614    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tcw9\" (UniqueName: \"kubernetes.io/projected/6693fd43-9bf1-41f2-87c6-37f1b8a4981d-kube-api-access-5tcw9\") pod \"kuadrant-operator-catalog-5zjdr\" (UID: \"6693fd43-9bf1-41f2-87c6-37f1b8a4981d\") " pod="kuadrant-system/kuadrant-operator-catalog-5zjdr"
Jul 29 15:17:57.831122 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.831088    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tcw9\" (UniqueName: \"kubernetes.io/projected/6693fd43-9bf1-41f2-87c6-37f1b8a4981d-kube-api-access-5tcw9\") pod \"kuadrant-operator-catalog-5zjdr\" (UID: \"6693fd43-9bf1-41f2-87c6-37f1b8a4981d\") " pod="kuadrant-system/kuadrant-operator-catalog-5zjdr"
Jul 29 15:17:57.841178 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.841154    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tcw9\" (UniqueName: \"kubernetes.io/projected/6693fd43-9bf1-41f2-87c6-37f1b8a4981d-kube-api-access-5tcw9\") pod \"kuadrant-operator-catalog-5zjdr\" (UID: \"6693fd43-9bf1-41f2-87c6-37f1b8a4981d\") " pod="kuadrant-system/kuadrant-operator-catalog-5zjdr"
Jul 29 15:17:57.941393 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.941357    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5zjdr"
Jul 29 15:17:57.975045 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:57.975016    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fc6pw" event={"ID":"2a95e5e2-cae9-432e-b569-aca1c5afe31a","Type":"ContainerStarted","Data":"8a5f7f3a356e611718e65712468d2473fa1ec38db5150dd1e096aae3237b9760"}
Jul 29 15:17:58.069370 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:58.069336    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5zjdr"]
Jul 29 15:17:58.071921 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:17:58.071883    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6693fd43_9bf1_41f2_87c6_37f1b8a4981d.slice/crio-7b99beb1652fad32c8e85ad2cde344537b4bf0644b785501865220bdbdaab45b WatchSource:0}: Error finding container 7b99beb1652fad32c8e85ad2cde344537b4bf0644b785501865220bdbdaab45b: Status 404 returned error can't find the container with id 7b99beb1652fad32c8e85ad2cde344537b4bf0644b785501865220bdbdaab45b
Jul 29 15:17:58.981244 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:58.981209    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5zjdr" event={"ID":"6693fd43-9bf1-41f2-87c6-37f1b8a4981d","Type":"ContainerStarted","Data":"7b99beb1652fad32c8e85ad2cde344537b4bf0644b785501865220bdbdaab45b"}
Jul 29 15:17:59.989519 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:59.989481    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5zjdr" event={"ID":"6693fd43-9bf1-41f2-87c6-37f1b8a4981d","Type":"ContainerStarted","Data":"e92d0ed8c8b455cdc5f03c75320a2894c5aff68c9801b1d14feb43ef3e69fe00"}
Jul 29 15:17:59.990791 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:59.990767    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fc6pw" event={"ID":"2a95e5e2-cae9-432e-b569-aca1c5afe31a","Type":"ContainerStarted","Data":"43b773062e896d11257f640a3f6f05244ee73e4cf902cb3f5949af41fa56f130"}
Jul 29 15:17:59.990900 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:17:59.990845    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-fc6pw" podUID="2a95e5e2-cae9-432e-b569-aca1c5afe31a" containerName="registry-server" containerID="cri-o://43b773062e896d11257f640a3f6f05244ee73e4cf902cb3f5949af41fa56f130" gracePeriod=2
Jul 29 15:18:00.011669 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:00.011633    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5zjdr" podStartSLOduration=1.638372521 podStartE2EDuration="3.011618753s" totalImagesPullingTime="1.373246232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:18:00.009265452 +0000 UTC m=+413.158559352" watchObservedRunningTime="2026-07-29 15:18:00.011618753 +0000 UTC m=+413.160912630"
Jul 29 15:18:00.031835 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:00.031799    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-fc6pw" podStartSLOduration=1.130272883 podStartE2EDuration="3.031787841s" totalImagesPullingTime="1.901514958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:18:00.031500432 +0000 UTC m=+413.180794332" watchObservedRunningTime="2026-07-29 15:18:00.031787841 +0000 UTC m=+413.181081739"
Jul 29 15:18:00.227841 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:00.227820    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fc6pw"
Jul 29 15:18:00.249390 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:00.249337    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6txq9\" (UniqueName: \"kubernetes.io/projected/2a95e5e2-cae9-432e-b569-aca1c5afe31a-kube-api-access-6txq9\") pod \"2a95e5e2-cae9-432e-b569-aca1c5afe31a\" (UID: \"2a95e5e2-cae9-432e-b569-aca1c5afe31a\") "
Jul 29 15:18:00.251633 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:00.251604    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a95e5e2-cae9-432e-b569-aca1c5afe31a-kube-api-access-6txq9" (OuterVolumeSpecName: "kube-api-access-6txq9") pod "2a95e5e2-cae9-432e-b569-aca1c5afe31a" (UID: "2a95e5e2-cae9-432e-b569-aca1c5afe31a"). InnerVolumeSpecName "kube-api-access-6txq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:18:00.350901 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:00.350874    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6txq9\" (UniqueName: \"kubernetes.io/projected/2a95e5e2-cae9-432e-b569-aca1c5afe31a-kube-api-access-6txq9\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:18:00.997049 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:00.997016    2575 generic.go:356] "Generic (PLEG): container finished" podID="2a95e5e2-cae9-432e-b569-aca1c5afe31a" containerID="43b773062e896d11257f640a3f6f05244ee73e4cf902cb3f5949af41fa56f130" exitCode=0
Jul 29 15:18:00.997508 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:00.997056    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fc6pw"
Jul 29 15:18:00.997508 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:00.997104    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fc6pw" event={"ID":"2a95e5e2-cae9-432e-b569-aca1c5afe31a","Type":"ContainerDied","Data":"43b773062e896d11257f640a3f6f05244ee73e4cf902cb3f5949af41fa56f130"}
Jul 29 15:18:00.997508 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:00.997158    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fc6pw" event={"ID":"2a95e5e2-cae9-432e-b569-aca1c5afe31a","Type":"ContainerDied","Data":"8a5f7f3a356e611718e65712468d2473fa1ec38db5150dd1e096aae3237b9760"}
Jul 29 15:18:00.997508 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:00.997187    2575 scope.go:117] "RemoveContainer" containerID="43b773062e896d11257f640a3f6f05244ee73e4cf902cb3f5949af41fa56f130"
Jul 29 15:18:01.006137 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:01.006118    2575 scope.go:117] "RemoveContainer" containerID="43b773062e896d11257f640a3f6f05244ee73e4cf902cb3f5949af41fa56f130"
Jul 29 15:18:01.006394 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:18:01.006375    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43b773062e896d11257f640a3f6f05244ee73e4cf902cb3f5949af41fa56f130\": container with ID starting with 43b773062e896d11257f640a3f6f05244ee73e4cf902cb3f5949af41fa56f130 not found: ID does not exist" containerID="43b773062e896d11257f640a3f6f05244ee73e4cf902cb3f5949af41fa56f130"
Jul 29 15:18:01.006460 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:01.006405    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43b773062e896d11257f640a3f6f05244ee73e4cf902cb3f5949af41fa56f130"} err="failed to get container status \"43b773062e896d11257f640a3f6f05244ee73e4cf902cb3f5949af41fa56f130\": rpc error: code = NotFound desc = could not find container \"43b773062e896d11257f640a3f6f05244ee73e4cf902cb3f5949af41fa56f130\": container with ID starting with 43b773062e896d11257f640a3f6f05244ee73e4cf902cb3f5949af41fa56f130 not found: ID does not exist"
Jul 29 15:18:01.020152 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:01.020132    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fc6pw"]
Jul 29 15:18:01.025181 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:01.025161    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fc6pw"]
Jul 29 15:18:01.293836 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:01.293763    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a95e5e2-cae9-432e-b569-aca1c5afe31a" path="/var/lib/kubelet/pods/2a95e5e2-cae9-432e-b569-aca1c5afe31a/volumes"
Jul 29 15:18:07.941680 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:07.941651    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-5zjdr"
Jul 29 15:18:07.942044 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:07.941706    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-5zjdr"
Jul 29 15:18:07.962631 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:07.962604    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-5zjdr"
Jul 29 15:18:08.045840 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:08.045816    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-5zjdr"
Jul 29 15:18:28.053108 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:28.053077    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"]
Jul 29 15:18:28.053566 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:28.053388    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a95e5e2-cae9-432e-b569-aca1c5afe31a" containerName="registry-server"
Jul 29 15:18:28.053566 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:28.053399    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a95e5e2-cae9-432e-b569-aca1c5afe31a" containerName="registry-server"
Jul 29 15:18:28.053566 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:28.053454    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a95e5e2-cae9-432e-b569-aca1c5afe31a" containerName="registry-server"
Jul 29 15:18:28.057946 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:28.057929    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"
Jul 29 15:18:28.060101 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:28.060085    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-dkv8r\""
Jul 29 15:18:28.069236 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:28.068201    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"]
Jul 29 15:18:28.182339 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:28.182302    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvxb6\" (UniqueName: \"kubernetes.io/projected/35d264c0-bb87-4742-aa2d-bd71b1d8a921-kube-api-access-dvxb6\") pod \"limitador-operator-controller-manager-859d7b78bd-w9l4j\" (UID: \"35d264c0-bb87-4742-aa2d-bd71b1d8a921\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"
Jul 29 15:18:28.283061 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:28.283027    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvxb6\" (UniqueName: \"kubernetes.io/projected/35d264c0-bb87-4742-aa2d-bd71b1d8a921-kube-api-access-dvxb6\") pod \"limitador-operator-controller-manager-859d7b78bd-w9l4j\" (UID: \"35d264c0-bb87-4742-aa2d-bd71b1d8a921\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"
Jul 29 15:18:28.294550 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:28.294520    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvxb6\" (UniqueName: \"kubernetes.io/projected/35d264c0-bb87-4742-aa2d-bd71b1d8a921-kube-api-access-dvxb6\") pod \"limitador-operator-controller-manager-859d7b78bd-w9l4j\" (UID: \"35d264c0-bb87-4742-aa2d-bd71b1d8a921\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"
Jul 29 15:18:28.367974 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:28.367947    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"
Jul 29 15:18:28.502838 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:28.502813    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"]
Jul 29 15:18:28.505434 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:18:28.505403    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35d264c0_bb87_4742_aa2d_bd71b1d8a921.slice/crio-7a5daa98ae13ca66d4b24a78655f53059f8af95a515b3cad30eccb20961d4168 WatchSource:0}: Error finding container 7a5daa98ae13ca66d4b24a78655f53059f8af95a515b3cad30eccb20961d4168: Status 404 returned error can't find the container with id 7a5daa98ae13ca66d4b24a78655f53059f8af95a515b3cad30eccb20961d4168
Jul 29 15:18:29.113833 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:29.113797    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j" event={"ID":"35d264c0-bb87-4742-aa2d-bd71b1d8a921","Type":"ContainerStarted","Data":"7a5daa98ae13ca66d4b24a78655f53059f8af95a515b3cad30eccb20961d4168"}
Jul 29 15:18:31.123628 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:31.123597    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j" event={"ID":"35d264c0-bb87-4742-aa2d-bd71b1d8a921","Type":"ContainerStarted","Data":"8e2e7c162687182003901db1ebaffc0f8ea73fc94697ca7a98b70e9f053cbf9c"}
Jul 29 15:18:31.124006 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:31.123685    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"
Jul 29 15:18:31.148030 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:31.147981    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j" podStartSLOduration=1.316757433 podStartE2EDuration="3.147965067s" totalImagesPullingTime="1.831207634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:18:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:18:31.146828912 +0000 UTC m=+444.296122811" watchObservedRunningTime="2026-07-29 15:18:31.147965067 +0000 UTC m=+444.297258965"
Jul 29 15:18:33.198605 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:33.198574    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"]
Jul 29 15:18:33.201824 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:33.201807    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"
Jul 29 15:18:33.204374 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:33.204349    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rhvk2\""
Jul 29 15:18:33.214861 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:33.214840    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"]
Jul 29 15:18:33.324162 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:33.324127    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5badf3d2-88f6-4e79-b24f-f376722a6fb3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v26k2\" (UID: \"5badf3d2-88f6-4e79-b24f-f376722a6fb3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"
Jul 29 15:18:33.324353 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:33.324197    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fw7z\" (UniqueName: \"kubernetes.io/projected/5badf3d2-88f6-4e79-b24f-f376722a6fb3-kube-api-access-5fw7z\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v26k2\" (UID: \"5badf3d2-88f6-4e79-b24f-f376722a6fb3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"
Jul 29 15:18:33.425509 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:33.425470    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fw7z\" (UniqueName: \"kubernetes.io/projected/5badf3d2-88f6-4e79-b24f-f376722a6fb3-kube-api-access-5fw7z\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v26k2\" (UID: \"5badf3d2-88f6-4e79-b24f-f376722a6fb3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"
Jul 29 15:18:33.425714 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:33.425533    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5badf3d2-88f6-4e79-b24f-f376722a6fb3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v26k2\" (UID: \"5badf3d2-88f6-4e79-b24f-f376722a6fb3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"
Jul 29 15:18:33.425959 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:33.425939    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5badf3d2-88f6-4e79-b24f-f376722a6fb3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v26k2\" (UID: \"5badf3d2-88f6-4e79-b24f-f376722a6fb3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"
Jul 29 15:18:33.434616 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:33.434593    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fw7z\" (UniqueName: \"kubernetes.io/projected/5badf3d2-88f6-4e79-b24f-f376722a6fb3-kube-api-access-5fw7z\") pod \"kuadrant-operator-controller-manager-8684c9ff49-v26k2\" (UID: \"5badf3d2-88f6-4e79-b24f-f376722a6fb3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"
Jul 29 15:18:33.511476 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:33.511414    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"
Jul 29 15:18:33.643293 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:33.643261    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"]
Jul 29 15:18:33.646416 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:18:33.646382    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5badf3d2_88f6_4e79_b24f_f376722a6fb3.slice/crio-d96c8aa432854eee42a285ec22ef53b4767c74e89672b7e0d47ec19825f7dc08 WatchSource:0}: Error finding container d96c8aa432854eee42a285ec22ef53b4767c74e89672b7e0d47ec19825f7dc08: Status 404 returned error can't find the container with id d96c8aa432854eee42a285ec22ef53b4767c74e89672b7e0d47ec19825f7dc08
Jul 29 15:18:34.137970 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:34.137936    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2" event={"ID":"5badf3d2-88f6-4e79-b24f-f376722a6fb3","Type":"ContainerStarted","Data":"d96c8aa432854eee42a285ec22ef53b4767c74e89672b7e0d47ec19825f7dc08"}
Jul 29 15:18:35.760493 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:35.760460    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-27hfh"]
Jul 29 15:18:35.763781 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:35.763759    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-27hfh"
Jul 29 15:18:35.766059 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:35.766036    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bvvnl\""
Jul 29 15:18:35.773455 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:35.773431    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-27hfh"]
Jul 29 15:18:35.846530 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:35.846504    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xss4m\" (UniqueName: \"kubernetes.io/projected/010096a5-22e8-4f05-bdf6-7616da899e6d-kube-api-access-xss4m\") pod \"authorino-operator-75cc8d5cb7-27hfh\" (UID: \"010096a5-22e8-4f05-bdf6-7616da899e6d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-27hfh"
Jul 29 15:18:35.947495 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:35.947450    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xss4m\" (UniqueName: \"kubernetes.io/projected/010096a5-22e8-4f05-bdf6-7616da899e6d-kube-api-access-xss4m\") pod \"authorino-operator-75cc8d5cb7-27hfh\" (UID: \"010096a5-22e8-4f05-bdf6-7616da899e6d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-27hfh"
Jul 29 15:18:35.956275 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:35.956246    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xss4m\" (UniqueName: \"kubernetes.io/projected/010096a5-22e8-4f05-bdf6-7616da899e6d-kube-api-access-xss4m\") pod \"authorino-operator-75cc8d5cb7-27hfh\" (UID: \"010096a5-22e8-4f05-bdf6-7616da899e6d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-27hfh"
Jul 29 15:18:36.076636 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:36.076568    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-27hfh"
Jul 29 15:18:36.205257 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:36.205214    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-27hfh"]
Jul 29 15:18:36.209298 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:18:36.209270    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod010096a5_22e8_4f05_bdf6_7616da899e6d.slice/crio-dc2b66f79602efdc776ebc159637ba6a96737d3893dfb044d0bbde3c964f49a4 WatchSource:0}: Error finding container dc2b66f79602efdc776ebc159637ba6a96737d3893dfb044d0bbde3c964f49a4: Status 404 returned error can't find the container with id dc2b66f79602efdc776ebc159637ba6a96737d3893dfb044d0bbde3c964f49a4
Jul 29 15:18:37.154503 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:37.154462    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-27hfh" event={"ID":"010096a5-22e8-4f05-bdf6-7616da899e6d","Type":"ContainerStarted","Data":"dc2b66f79602efdc776ebc159637ba6a96737d3893dfb044d0bbde3c964f49a4"}
Jul 29 15:18:40.170654 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:40.170613    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2" event={"ID":"5badf3d2-88f6-4e79-b24f-f376722a6fb3","Type":"ContainerStarted","Data":"4ca1fb02c52181436811591599e5ce157dba2b4a7618f0aacfeff77e42d90f59"}
Jul 29 15:18:40.171099 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:40.170694    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"
Jul 29 15:18:40.172123 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:40.172098    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-27hfh" event={"ID":"010096a5-22e8-4f05-bdf6-7616da899e6d","Type":"ContainerStarted","Data":"ba842e810b4bc4790c2eb9b3ddeddad8c60fd193454676248c16bc515f08458c"}
Jul 29 15:18:40.172248 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:40.172143    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-27hfh"
Jul 29 15:18:40.192448 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:40.192405    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2" podStartSLOduration=1.271255534 podStartE2EDuration="7.19239113s" totalImagesPullingTime="5.921135596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:18:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:18:40.190581692 +0000 UTC m=+453.339875591" watchObservedRunningTime="2026-07-29 15:18:40.19239113 +0000 UTC m=+453.341685036"
Jul 29 15:18:40.214650 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:40.214610    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-27hfh" podStartSLOduration=1.858888554 podStartE2EDuration="5.214593706s" totalImagesPullingTime="3.355705152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:18:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:18:40.214291281 +0000 UTC m=+453.363585181" watchObservedRunningTime="2026-07-29 15:18:40.214593706 +0000 UTC m=+453.363887605"
Jul 29 15:18:42.130963 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.130932    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"
Jul 29 15:18:42.226694 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.226665    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-bspg8"]
Jul 29 15:18:42.230377 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.230355    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bspg8"
Jul 29 15:18:42.232596 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.232570    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 29 15:18:42.232704 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.232624    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 29 15:18:42.232704 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.232649    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wt4gx\""
Jul 29 15:18:42.239094 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.239074    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-bspg8"]
Jul 29 15:18:42.403029 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.402927    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/748b38d9-4a8b-42a9-9769-20f041c64d29-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-bspg8\" (UID: \"748b38d9-4a8b-42a9-9769-20f041c64d29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bspg8"
Jul 29 15:18:42.403029 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.403036    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdgr8\" (UniqueName: \"kubernetes.io/projected/748b38d9-4a8b-42a9-9769-20f041c64d29-kube-api-access-wdgr8\") pod \"kuadrant-console-plugin-66b5774985-bspg8\" (UID: \"748b38d9-4a8b-42a9-9769-20f041c64d29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bspg8"
Jul 29 15:18:42.403228 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.403060    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/748b38d9-4a8b-42a9-9769-20f041c64d29-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-bspg8\" (UID: \"748b38d9-4a8b-42a9-9769-20f041c64d29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bspg8"
Jul 29 15:18:42.504329 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.504297    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdgr8\" (UniqueName: \"kubernetes.io/projected/748b38d9-4a8b-42a9-9769-20f041c64d29-kube-api-access-wdgr8\") pod \"kuadrant-console-plugin-66b5774985-bspg8\" (UID: \"748b38d9-4a8b-42a9-9769-20f041c64d29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bspg8"
Jul 29 15:18:42.504490 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.504355    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/748b38d9-4a8b-42a9-9769-20f041c64d29-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-bspg8\" (UID: \"748b38d9-4a8b-42a9-9769-20f041c64d29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bspg8"
Jul 29 15:18:42.504490 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.504395    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/748b38d9-4a8b-42a9-9769-20f041c64d29-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-bspg8\" (UID: \"748b38d9-4a8b-42a9-9769-20f041c64d29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bspg8"
Jul 29 15:18:42.505099 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.505080    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/748b38d9-4a8b-42a9-9769-20f041c64d29-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-bspg8\" (UID: \"748b38d9-4a8b-42a9-9769-20f041c64d29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bspg8"
Jul 29 15:18:42.506758 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.506740    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/748b38d9-4a8b-42a9-9769-20f041c64d29-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-bspg8\" (UID: \"748b38d9-4a8b-42a9-9769-20f041c64d29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bspg8"
Jul 29 15:18:42.516034 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.516011    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdgr8\" (UniqueName: \"kubernetes.io/projected/748b38d9-4a8b-42a9-9769-20f041c64d29-kube-api-access-wdgr8\") pod \"kuadrant-console-plugin-66b5774985-bspg8\" (UID: \"748b38d9-4a8b-42a9-9769-20f041c64d29\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bspg8"
Jul 29 15:18:42.540133 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.540114    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bspg8"
Jul 29 15:18:42.671294 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:42.671198    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-bspg8"]
Jul 29 15:18:42.673867 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:18:42.673843    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod748b38d9_4a8b_42a9_9769_20f041c64d29.slice/crio-aeaa816ae3c0f094cb9e64d0e4840d5cea115fb294d1d21b6e7215a654f5ccd4 WatchSource:0}: Error finding container aeaa816ae3c0f094cb9e64d0e4840d5cea115fb294d1d21b6e7215a654f5ccd4: Status 404 returned error can't find the container with id aeaa816ae3c0f094cb9e64d0e4840d5cea115fb294d1d21b6e7215a654f5ccd4
Jul 29 15:18:43.185887 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:43.185853    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bspg8" event={"ID":"748b38d9-4a8b-42a9-9769-20f041c64d29","Type":"ContainerStarted","Data":"aeaa816ae3c0f094cb9e64d0e4840d5cea115fb294d1d21b6e7215a654f5ccd4"}
Jul 29 15:18:51.178767 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:51.178737    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"
Jul 29 15:18:51.179216 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:51.178809    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-27hfh"
Jul 29 15:18:53.238749 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.238620    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"]
Jul 29 15:18:53.239220 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.238884    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2" podUID="5badf3d2-88f6-4e79-b24f-f376722a6fb3" containerName="manager" containerID="cri-o://4ca1fb02c52181436811591599e5ce157dba2b4a7618f0aacfeff77e42d90f59" gracePeriod=2
Jul 29 15:18:53.253049 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.253001    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"]
Jul 29 15:18:53.260549 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:18:53.260509    2575 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"5badf3d2-88f6-4e79-b24f-f376722a6fb3\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-v26k2\": pods \"kuadrant-operator-controller-manager-8684c9ff49-v26k2\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"
Jul 29 15:18:53.273894 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.273856    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"]
Jul 29 15:18:53.274119 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.274089    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j" podUID="35d264c0-bb87-4742-aa2d-bd71b1d8a921" containerName="manager" containerID="cri-o://8e2e7c162687182003901db1ebaffc0f8ea73fc94697ca7a98b70e9f053cbf9c" gracePeriod=2
Jul 29 15:18:53.277389 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:18:53.277357    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v26k2\" is forbidden: User \"system:node:ip-10-0-135-116.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-116.ec2.internal' and this object" podUID="5badf3d2-88f6-4e79-b24f-f376722a6fb3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"
Jul 29 15:18:53.281379 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.281356    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"]
Jul 29 15:18:53.281811 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.281793    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5badf3d2-88f6-4e79-b24f-f376722a6fb3" containerName="manager"
Jul 29 15:18:53.281913 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.281815    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5badf3d2-88f6-4e79-b24f-f376722a6fb3" containerName="manager"
Jul 29 15:18:53.281913 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.281886    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5badf3d2-88f6-4e79-b24f-f376722a6fb3" containerName="manager"
Jul 29 15:18:53.286078 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.285661    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"
Jul 29 15:18:53.289516 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.289497    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"]
Jul 29 15:18:53.290000 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.289817    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzmgc\" (UniqueName: \"kubernetes.io/projected/6003f970-3ee5-4a8e-8c72-25b0405b0784-kube-api-access-jzmgc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4v9wc\" (UID: \"6003f970-3ee5-4a8e-8c72-25b0405b0784\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"
Jul 29 15:18:53.290000 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.289865    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6003f970-3ee5-4a8e-8c72-25b0405b0784-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4v9wc\" (UID: \"6003f970-3ee5-4a8e-8c72-25b0405b0784\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"
Jul 29 15:18:53.301571 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:18:53.301537    2575 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"35d264c0-bb87-4742-aa2d-bd71b1d8a921\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-w9l4j\": pods \"limitador-operator-controller-manager-859d7b78bd-w9l4j\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"
Jul 29 15:18:53.306225 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.306204    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"]
Jul 29 15:18:53.317016 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.316993    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zhfmp"]
Jul 29 15:18:53.318462 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.317413    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35d264c0-bb87-4742-aa2d-bd71b1d8a921" containerName="manager"
Jul 29 15:18:53.318462 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.317434    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="35d264c0-bb87-4742-aa2d-bd71b1d8a921" containerName="manager"
Jul 29 15:18:53.318462 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.317554    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="35d264c0-bb87-4742-aa2d-bd71b1d8a921" containerName="manager"
Jul 29 15:18:53.322010 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.321991    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zhfmp"
Jul 29 15:18:53.336478 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.336444    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zhfmp"]
Jul 29 15:18:53.391261 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.391231    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzmgc\" (UniqueName: \"kubernetes.io/projected/6003f970-3ee5-4a8e-8c72-25b0405b0784-kube-api-access-jzmgc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4v9wc\" (UID: \"6003f970-3ee5-4a8e-8c72-25b0405b0784\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"
Jul 29 15:18:53.391425 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.391269    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6003f970-3ee5-4a8e-8c72-25b0405b0784-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4v9wc\" (UID: \"6003f970-3ee5-4a8e-8c72-25b0405b0784\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"
Jul 29 15:18:53.391671 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.391651    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6003f970-3ee5-4a8e-8c72-25b0405b0784-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4v9wc\" (UID: \"6003f970-3ee5-4a8e-8c72-25b0405b0784\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"
Jul 29 15:18:53.400567 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.400541    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzmgc\" (UniqueName: \"kubernetes.io/projected/6003f970-3ee5-4a8e-8c72-25b0405b0784-kube-api-access-jzmgc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4v9wc\" (UID: \"6003f970-3ee5-4a8e-8c72-25b0405b0784\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"
Jul 29 15:18:53.492183 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.492108    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzd6t\" (UniqueName: \"kubernetes.io/projected/ff3181c2-66af-4c2c-bc41-63cfcb90f40d-kube-api-access-tzd6t\") pod \"limitador-operator-controller-manager-859d7b78bd-zhfmp\" (UID: \"ff3181c2-66af-4c2c-bc41-63cfcb90f40d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zhfmp"
Jul 29 15:18:53.593113 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.593066    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzd6t\" (UniqueName: \"kubernetes.io/projected/ff3181c2-66af-4c2c-bc41-63cfcb90f40d-kube-api-access-tzd6t\") pod \"limitador-operator-controller-manager-859d7b78bd-zhfmp\" (UID: \"ff3181c2-66af-4c2c-bc41-63cfcb90f40d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zhfmp"
Jul 29 15:18:53.608243 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.608213    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzd6t\" (UniqueName: \"kubernetes.io/projected/ff3181c2-66af-4c2c-bc41-63cfcb90f40d-kube-api-access-tzd6t\") pod \"limitador-operator-controller-manager-859d7b78bd-zhfmp\" (UID: \"ff3181c2-66af-4c2c-bc41-63cfcb90f40d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zhfmp"
Jul 29 15:18:53.662624 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.662579    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"
Jul 29 15:18:53.669772 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:18:53.669747    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zhfmp"
Jul 29 15:19:04.304602 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:04.304565    2575 generic.go:356] "Generic (PLEG): container finished" podID="5badf3d2-88f6-4e79-b24f-f376722a6fb3" containerID="4ca1fb02c52181436811591599e5ce157dba2b4a7618f0aacfeff77e42d90f59" exitCode=0
Jul 29 15:19:04.307431 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:04.307372    2575 generic.go:356] "Generic (PLEG): container finished" podID="35d264c0-bb87-4742-aa2d-bd71b1d8a921" containerID="8e2e7c162687182003901db1ebaffc0f8ea73fc94697ca7a98b70e9f053cbf9c" exitCode=0
Jul 29 15:19:04.461793 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:04.461764    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zhfmp"]
Jul 29 15:19:04.465279 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:19:04.465250    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff3181c2_66af_4c2c_bc41_63cfcb90f40d.slice/crio-d9c6325baf7bc2969e5fcbf01e601221e40c1e66d1867aa9edcf6917da50d77f WatchSource:0}: Error finding container d9c6325baf7bc2969e5fcbf01e601221e40c1e66d1867aa9edcf6917da50d77f: Status 404 returned error can't find the container with id d9c6325baf7bc2969e5fcbf01e601221e40c1e66d1867aa9edcf6917da50d77f
Jul 29 15:19:04.468548 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:04.468488    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"
Jul 29 15:19:04.470442 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:19:04.470417    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-w9l4j\" is forbidden: User \"system:node:ip-10-0-135-116.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-116.ec2.internal' and this object" podUID="35d264c0-bb87-4742-aa2d-bd71b1d8a921" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"
Jul 29 15:19:04.471005 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:04.470987    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"
Jul 29 15:19:04.472899 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:19:04.472869    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-w9l4j\" is forbidden: User \"system:node:ip-10-0-135-116.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-116.ec2.internal' and this object" podUID="35d264c0-bb87-4742-aa2d-bd71b1d8a921" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"
Jul 29 15:19:04.475426 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:19:04.475398    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v26k2\" is forbidden: User \"system:node:ip-10-0-135-116.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-116.ec2.internal' and this object" podUID="5badf3d2-88f6-4e79-b24f-f376722a6fb3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"
Jul 29 15:19:04.489873 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:04.489851    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"]
Jul 29 15:19:04.493423 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:19:04.493399    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6003f970_3ee5_4a8e_8c72_25b0405b0784.slice/crio-7cf42bada009e7de157aa0813ff03fe8e2bf56515977312ada21f3498e6bfc11 WatchSource:0}: Error finding container 7cf42bada009e7de157aa0813ff03fe8e2bf56515977312ada21f3498e6bfc11: Status 404 returned error can't find the container with id 7cf42bada009e7de157aa0813ff03fe8e2bf56515977312ada21f3498e6bfc11
Jul 29 15:19:04.583688 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:04.583659    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvxb6\" (UniqueName: \"kubernetes.io/projected/35d264c0-bb87-4742-aa2d-bd71b1d8a921-kube-api-access-dvxb6\") pod \"35d264c0-bb87-4742-aa2d-bd71b1d8a921\" (UID: \"35d264c0-bb87-4742-aa2d-bd71b1d8a921\") "
Jul 29 15:19:04.583809 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:04.583711    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5badf3d2-88f6-4e79-b24f-f376722a6fb3-extensions-socket-volume\") pod \"5badf3d2-88f6-4e79-b24f-f376722a6fb3\" (UID: \"5badf3d2-88f6-4e79-b24f-f376722a6fb3\") "
Jul 29 15:19:04.583809 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:04.583771    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fw7z\" (UniqueName: \"kubernetes.io/projected/5badf3d2-88f6-4e79-b24f-f376722a6fb3-kube-api-access-5fw7z\") pod \"5badf3d2-88f6-4e79-b24f-f376722a6fb3\" (UID: \"5badf3d2-88f6-4e79-b24f-f376722a6fb3\") "
Jul 29 15:19:04.584178 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:04.584152    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5badf3d2-88f6-4e79-b24f-f376722a6fb3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5badf3d2-88f6-4e79-b24f-f376722a6fb3" (UID: "5badf3d2-88f6-4e79-b24f-f376722a6fb3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:19:04.585443 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:04.585421    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35d264c0-bb87-4742-aa2d-bd71b1d8a921-kube-api-access-dvxb6" (OuterVolumeSpecName: "kube-api-access-dvxb6") pod "35d264c0-bb87-4742-aa2d-bd71b1d8a921" (UID: "35d264c0-bb87-4742-aa2d-bd71b1d8a921"). InnerVolumeSpecName "kube-api-access-dvxb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:19:04.585692 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:04.585671    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5badf3d2-88f6-4e79-b24f-f376722a6fb3-kube-api-access-5fw7z" (OuterVolumeSpecName: "kube-api-access-5fw7z") pod "5badf3d2-88f6-4e79-b24f-f376722a6fb3" (UID: "5badf3d2-88f6-4e79-b24f-f376722a6fb3"). InnerVolumeSpecName "kube-api-access-5fw7z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:19:04.684854 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:04.684826    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvxb6\" (UniqueName: \"kubernetes.io/projected/35d264c0-bb87-4742-aa2d-bd71b1d8a921-kube-api-access-dvxb6\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:19:04.684854 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:04.684851    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5badf3d2-88f6-4e79-b24f-f376722a6fb3-extensions-socket-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:19:04.684854 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:04.684860    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fw7z\" (UniqueName: \"kubernetes.io/projected/5badf3d2-88f6-4e79-b24f-f376722a6fb3-kube-api-access-5fw7z\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:19:05.294963 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:05.294935    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35d264c0-bb87-4742-aa2d-bd71b1d8a921" path="/var/lib/kubelet/pods/35d264c0-bb87-4742-aa2d-bd71b1d8a921/volumes"
Jul 29 15:19:05.295285 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:05.295272    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5badf3d2-88f6-4e79-b24f-f376722a6fb3" path="/var/lib/kubelet/pods/5badf3d2-88f6-4e79-b24f-f376722a6fb3/volumes"
Jul 29 15:19:05.312612 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:05.312590    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zhfmp" event={"ID":"ff3181c2-66af-4c2c-bc41-63cfcb90f40d","Type":"ContainerStarted","Data":"093aba7f5b47e1211ddd80819b8ada00a30c91d8060029edc5aa5426a15b16fc"}
Jul 29 15:19:05.312923 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:05.312633    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zhfmp" event={"ID":"ff3181c2-66af-4c2c-bc41-63cfcb90f40d","Type":"ContainerStarted","Data":"d9c6325baf7bc2969e5fcbf01e601221e40c1e66d1867aa9edcf6917da50d77f"}
Jul 29 15:19:05.312923 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:05.312708    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zhfmp"
Jul 29 15:19:05.314092 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:05.314067    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bspg8" event={"ID":"748b38d9-4a8b-42a9-9769-20f041c64d29","Type":"ContainerStarted","Data":"2c834c0ab9bbfa20cdc8e1e6a9e3ffad417f75360573034422ce87166de1d98b"}
Jul 29 15:19:05.315252 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:05.315239    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"
Jul 29 15:19:05.315355 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:05.315266    2575 scope.go:117] "RemoveContainer" containerID="4ca1fb02c52181436811591599e5ce157dba2b4a7618f0aacfeff77e42d90f59"
Jul 29 15:19:05.316914 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:05.316763    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"
Jul 29 15:19:05.318630 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:05.318599    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc" event={"ID":"6003f970-3ee5-4a8e-8c72-25b0405b0784","Type":"ContainerStarted","Data":"8ad77f06ef4647a25c95bcc04763aaa67a27627e9777aa48c8b6af1819462111"}
Jul 29 15:19:05.318741 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:05.318652    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc" event={"ID":"6003f970-3ee5-4a8e-8c72-25b0405b0784","Type":"ContainerStarted","Data":"7cf42bada009e7de157aa0813ff03fe8e2bf56515977312ada21f3498e6bfc11"}
Jul 29 15:19:05.319123 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:05.319078    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"
Jul 29 15:19:05.325135 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:05.325120    2575 scope.go:117] "RemoveContainer" containerID="8e2e7c162687182003901db1ebaffc0f8ea73fc94697ca7a98b70e9f053cbf9c"
Jul 29 15:19:05.330125 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:05.330083    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zhfmp" podStartSLOduration=12.330070217 podStartE2EDuration="12.330070217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:18:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:19:05.329493333 +0000 UTC m=+478.478787233" watchObservedRunningTime="2026-07-29 15:19:05.330070217 +0000 UTC m=+478.479364115"
Jul 29 15:19:05.346419 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:05.346383    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-bspg8" podStartSLOduration=1.646675187 podStartE2EDuration="23.346370064s" totalImagesPullingTime="21.699694877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:18:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:19:05.345267808 +0000 UTC m=+478.494561707" watchObservedRunningTime="2026-07-29 15:19:05.346370064 +0000 UTC m=+478.495663962"
Jul 29 15:19:05.364499 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:05.364456    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc" podStartSLOduration=12.364443012 podStartE2EDuration="12.364443012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:18:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:19:05.363828886 +0000 UTC m=+478.513122784" watchObservedRunningTime="2026-07-29 15:19:05.364443012 +0000 UTC m=+478.513736911"
Jul 29 15:19:05.365597 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:19:05.365571    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-w9l4j\" is forbidden: User \"system:node:ip-10-0-135-116.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-116.ec2.internal' and this object" podUID="35d264c0-bb87-4742-aa2d-bd71b1d8a921" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"
Jul 29 15:19:05.367557 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:19:05.367529    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v26k2\" is forbidden: User \"system:node:ip-10-0-135-116.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-116.ec2.internal' and this object" podUID="5badf3d2-88f6-4e79-b24f-f376722a6fb3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"
Jul 29 15:19:07.292303 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:19:07.292271    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-w9l4j\" is forbidden: User \"system:node:ip-10-0-135-116.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-116.ec2.internal' and this object" podUID="35d264c0-bb87-4742-aa2d-bd71b1d8a921" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-w9l4j"
Jul 29 15:19:07.293888 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:19:07.293861    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-v26k2\" is forbidden: User \"system:node:ip-10-0-135-116.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-116.ec2.internal' and this object" podUID="5badf3d2-88f6-4e79-b24f-f376722a6fb3" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-v26k2"
Jul 29 15:19:10.421624 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:10.421592    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"]
Jul 29 15:19:10.422176 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:10.421820    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc" podUID="6003f970-3ee5-4a8e-8c72-25b0405b0784" containerName="manager" containerID="cri-o://8ad77f06ef4647a25c95bcc04763aaa67a27627e9777aa48c8b6af1819462111" gracePeriod=10
Jul 29 15:19:10.423450 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:10.423427    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"
Jul 29 15:19:10.765172 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:10.765092    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"]
Jul 29 15:19:10.778455 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:10.778430    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"
Jul 29 15:19:10.781487 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:10.781463    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"]
Jul 29 15:19:10.838023 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:10.837991    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pjm8\" (UniqueName: \"kubernetes.io/projected/75af11f4-880a-4aff-85d2-bb8ec5cf1da4-kube-api-access-9pjm8\") pod \"kuadrant-operator-controller-manager-86878c769f-x4b7j\" (UID: \"75af11f4-880a-4aff-85d2-bb8ec5cf1da4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"
Jul 29 15:19:10.838150 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:10.838043    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/75af11f4-880a-4aff-85d2-bb8ec5cf1da4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x4b7j\" (UID: \"75af11f4-880a-4aff-85d2-bb8ec5cf1da4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"
Jul 29 15:19:10.939032 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:10.939003    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pjm8\" (UniqueName: \"kubernetes.io/projected/75af11f4-880a-4aff-85d2-bb8ec5cf1da4-kube-api-access-9pjm8\") pod \"kuadrant-operator-controller-manager-86878c769f-x4b7j\" (UID: \"75af11f4-880a-4aff-85d2-bb8ec5cf1da4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"
Jul 29 15:19:10.939167 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:10.939040    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/75af11f4-880a-4aff-85d2-bb8ec5cf1da4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x4b7j\" (UID: \"75af11f4-880a-4aff-85d2-bb8ec5cf1da4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"
Jul 29 15:19:10.939384 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:10.939367    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/75af11f4-880a-4aff-85d2-bb8ec5cf1da4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x4b7j\" (UID: \"75af11f4-880a-4aff-85d2-bb8ec5cf1da4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"
Jul 29 15:19:10.947896 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:10.947875    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pjm8\" (UniqueName: \"kubernetes.io/projected/75af11f4-880a-4aff-85d2-bb8ec5cf1da4-kube-api-access-9pjm8\") pod \"kuadrant-operator-controller-manager-86878c769f-x4b7j\" (UID: \"75af11f4-880a-4aff-85d2-bb8ec5cf1da4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"
Jul 29 15:19:11.088975 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.088949    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"
Jul 29 15:19:11.238548 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.238524    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"
Jul 29 15:19:11.247775 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.247752    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"]
Jul 29 15:19:11.251499 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:19:11.251477    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75af11f4_880a_4aff_85d2_bb8ec5cf1da4.slice/crio-6ae5ea2012c981c385ba101bfbeff67733cddeaf32a13dd89559d10cf7607071 WatchSource:0}: Error finding container 6ae5ea2012c981c385ba101bfbeff67733cddeaf32a13dd89559d10cf7607071: Status 404 returned error can't find the container with id 6ae5ea2012c981c385ba101bfbeff67733cddeaf32a13dd89559d10cf7607071
Jul 29 15:19:11.342762 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.342738    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6003f970-3ee5-4a8e-8c72-25b0405b0784-extensions-socket-volume\") pod \"6003f970-3ee5-4a8e-8c72-25b0405b0784\" (UID: \"6003f970-3ee5-4a8e-8c72-25b0405b0784\") "
Jul 29 15:19:11.342879 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.342789    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzmgc\" (UniqueName: \"kubernetes.io/projected/6003f970-3ee5-4a8e-8c72-25b0405b0784-kube-api-access-jzmgc\") pod \"6003f970-3ee5-4a8e-8c72-25b0405b0784\" (UID: \"6003f970-3ee5-4a8e-8c72-25b0405b0784\") "
Jul 29 15:19:11.343083 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.343063    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6003f970-3ee5-4a8e-8c72-25b0405b0784-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6003f970-3ee5-4a8e-8c72-25b0405b0784" (UID: "6003f970-3ee5-4a8e-8c72-25b0405b0784"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:19:11.344941 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.344920    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6003f970-3ee5-4a8e-8c72-25b0405b0784-kube-api-access-jzmgc" (OuterVolumeSpecName: "kube-api-access-jzmgc") pod "6003f970-3ee5-4a8e-8c72-25b0405b0784" (UID: "6003f970-3ee5-4a8e-8c72-25b0405b0784"). InnerVolumeSpecName "kube-api-access-jzmgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:19:11.348874 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.348844    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j" event={"ID":"75af11f4-880a-4aff-85d2-bb8ec5cf1da4","Type":"ContainerStarted","Data":"d0dd74eb3b55f585bdb1649b8cbc9ddfe433cc0df5987aeaa03e13eeee1f601e"}
Jul 29 15:19:11.348968 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.348912    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j" event={"ID":"75af11f4-880a-4aff-85d2-bb8ec5cf1da4","Type":"ContainerStarted","Data":"6ae5ea2012c981c385ba101bfbeff67733cddeaf32a13dd89559d10cf7607071"}
Jul 29 15:19:11.350243 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.350220    2575 generic.go:356] "Generic (PLEG): container finished" podID="6003f970-3ee5-4a8e-8c72-25b0405b0784" containerID="8ad77f06ef4647a25c95bcc04763aaa67a27627e9777aa48c8b6af1819462111" exitCode=0
Jul 29 15:19:11.350370 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.350271    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"
Jul 29 15:19:11.350370 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.350275    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc" event={"ID":"6003f970-3ee5-4a8e-8c72-25b0405b0784","Type":"ContainerDied","Data":"8ad77f06ef4647a25c95bcc04763aaa67a27627e9777aa48c8b6af1819462111"}
Jul 29 15:19:11.350476 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.350390    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc" event={"ID":"6003f970-3ee5-4a8e-8c72-25b0405b0784","Type":"ContainerDied","Data":"7cf42bada009e7de157aa0813ff03fe8e2bf56515977312ada21f3498e6bfc11"}
Jul 29 15:19:11.350476 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.350419    2575 scope.go:117] "RemoveContainer" containerID="8ad77f06ef4647a25c95bcc04763aaa67a27627e9777aa48c8b6af1819462111"
Jul 29 15:19:11.358144 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.358130    2575 scope.go:117] "RemoveContainer" containerID="8ad77f06ef4647a25c95bcc04763aaa67a27627e9777aa48c8b6af1819462111"
Jul 29 15:19:11.358421 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:19:11.358405    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ad77f06ef4647a25c95bcc04763aaa67a27627e9777aa48c8b6af1819462111\": container with ID starting with 8ad77f06ef4647a25c95bcc04763aaa67a27627e9777aa48c8b6af1819462111 not found: ID does not exist" containerID="8ad77f06ef4647a25c95bcc04763aaa67a27627e9777aa48c8b6af1819462111"
Jul 29 15:19:11.358493 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.358434    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ad77f06ef4647a25c95bcc04763aaa67a27627e9777aa48c8b6af1819462111"} err="failed to get container status \"8ad77f06ef4647a25c95bcc04763aaa67a27627e9777aa48c8b6af1819462111\": rpc error: code = NotFound desc = could not find container \"8ad77f06ef4647a25c95bcc04763aaa67a27627e9777aa48c8b6af1819462111\": container with ID starting with 8ad77f06ef4647a25c95bcc04763aaa67a27627e9777aa48c8b6af1819462111 not found: ID does not exist"
Jul 29 15:19:11.385080 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.385052    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"]
Jul 29 15:19:11.398133 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.398108    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4v9wc"]
Jul 29 15:19:11.443916 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.443888    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6003f970-3ee5-4a8e-8c72-25b0405b0784-extensions-socket-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:19:11.443916 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:11.443916    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzmgc\" (UniqueName: \"kubernetes.io/projected/6003f970-3ee5-4a8e-8c72-25b0405b0784-kube-api-access-jzmgc\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:19:12.355451 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:12.355417    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"
Jul 29 15:19:12.377044 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:12.376997    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j" podStartSLOduration=2.376983841 podStartE2EDuration="2.376983841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:19:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:19:12.374859411 +0000 UTC m=+485.524153309" watchObservedRunningTime="2026-07-29 15:19:12.376983841 +0000 UTC m=+485.526277738"
Jul 29 15:19:13.294083 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:13.294051    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6003f970-3ee5-4a8e-8c72-25b0405b0784" path="/var/lib/kubelet/pods/6003f970-3ee5-4a8e-8c72-25b0405b0784/volumes"
Jul 29 15:19:16.328023 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:16.327997    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zhfmp"
Jul 29 15:19:23.362371 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:23.362338    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"
Jul 29 15:19:40.463069 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:40.463031    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"]
Jul 29 15:19:40.463493 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:40.463419    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6003f970-3ee5-4a8e-8c72-25b0405b0784" containerName="manager"
Jul 29 15:19:40.463493 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:40.463431    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6003f970-3ee5-4a8e-8c72-25b0405b0784" containerName="manager"
Jul 29 15:19:40.463493 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:40.463494    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6003f970-3ee5-4a8e-8c72-25b0405b0784" containerName="manager"
Jul 29 15:19:40.498362 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:40.498335    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"]
Jul 29 15:19:40.498509 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:40.498448    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"
Jul 29 15:19:40.500646 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:40.500622    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 29 15:19:40.567958 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:40.567924    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"]
Jul 29 15:19:40.601639 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:40.601610    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d8sg\" (UniqueName: \"kubernetes.io/projected/164ae7c1-c134-4f61-b7a7-e26fd53b0df2-kube-api-access-5d8sg\") pod \"limitador-limitador-64c94fd64f-wf2zh\" (UID: \"164ae7c1-c134-4f61-b7a7-e26fd53b0df2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"
Jul 29 15:19:40.601785 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:40.601645    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/164ae7c1-c134-4f61-b7a7-e26fd53b0df2-config-file\") pod \"limitador-limitador-64c94fd64f-wf2zh\" (UID: \"164ae7c1-c134-4f61-b7a7-e26fd53b0df2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"
Jul 29 15:19:40.702559 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:40.702526    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d8sg\" (UniqueName: \"kubernetes.io/projected/164ae7c1-c134-4f61-b7a7-e26fd53b0df2-kube-api-access-5d8sg\") pod \"limitador-limitador-64c94fd64f-wf2zh\" (UID: \"164ae7c1-c134-4f61-b7a7-e26fd53b0df2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"
Jul 29 15:19:40.702743 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:40.702568    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/164ae7c1-c134-4f61-b7a7-e26fd53b0df2-config-file\") pod \"limitador-limitador-64c94fd64f-wf2zh\" (UID: \"164ae7c1-c134-4f61-b7a7-e26fd53b0df2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"
Jul 29 15:19:40.703163 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:40.703146    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/164ae7c1-c134-4f61-b7a7-e26fd53b0df2-config-file\") pod \"limitador-limitador-64c94fd64f-wf2zh\" (UID: \"164ae7c1-c134-4f61-b7a7-e26fd53b0df2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"
Jul 29 15:19:40.711593 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:40.711565    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d8sg\" (UniqueName: \"kubernetes.io/projected/164ae7c1-c134-4f61-b7a7-e26fd53b0df2-kube-api-access-5d8sg\") pod \"limitador-limitador-64c94fd64f-wf2zh\" (UID: \"164ae7c1-c134-4f61-b7a7-e26fd53b0df2\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"
Jul 29 15:19:40.808085 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:40.808009    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"
Jul 29 15:19:40.936528 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:40.936499    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"]
Jul 29 15:19:40.940062 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:19:40.940038    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod164ae7c1_c134_4f61_b7a7_e26fd53b0df2.slice/crio-2319eeadf59c6191755c69f85a33a2530341b38bdc4da99200a9235059ba4bf1 WatchSource:0}: Error finding container 2319eeadf59c6191755c69f85a33a2530341b38bdc4da99200a9235059ba4bf1: Status 404 returned error can't find the container with id 2319eeadf59c6191755c69f85a33a2530341b38bdc4da99200a9235059ba4bf1
Jul 29 15:19:41.487236 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:41.487204    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh" event={"ID":"164ae7c1-c134-4f61-b7a7-e26fd53b0df2","Type":"ContainerStarted","Data":"2319eeadf59c6191755c69f85a33a2530341b38bdc4da99200a9235059ba4bf1"}
Jul 29 15:19:44.506276 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:44.506246    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh" event={"ID":"164ae7c1-c134-4f61-b7a7-e26fd53b0df2","Type":"ContainerStarted","Data":"f00b743bb438f8c6e6b250ea9519f58cfeaadde9044ffa084d5a407e42830de6"}
Jul 29 15:19:44.506650 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:44.506380    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"
Jul 29 15:19:44.525647 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:44.525605    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh" podStartSLOduration=1.913001733 podStartE2EDuration="4.525591907s" totalImagesPullingTime="2.612590174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:19:44.52365935 +0000 UTC m=+517.672953243" watchObservedRunningTime="2026-07-29 15:19:44.525591907 +0000 UTC m=+517.674885806"
Jul 29 15:19:55.511613 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:55.511587    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"
Jul 29 15:19:55.672177 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:55.672143    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"]
Jul 29 15:19:55.672352 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:55.672316    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh" podUID="164ae7c1-c134-4f61-b7a7-e26fd53b0df2" containerName="limitador" containerID="cri-o://f00b743bb438f8c6e6b250ea9519f58cfeaadde9044ffa084d5a407e42830de6" gracePeriod=30
Jul 29 15:19:56.216951 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:56.216922    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"
Jul 29 15:19:56.324108 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:56.324027    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/164ae7c1-c134-4f61-b7a7-e26fd53b0df2-config-file\") pod \"164ae7c1-c134-4f61-b7a7-e26fd53b0df2\" (UID: \"164ae7c1-c134-4f61-b7a7-e26fd53b0df2\") "
Jul 29 15:19:56.324245 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:56.324119    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5d8sg\" (UniqueName: \"kubernetes.io/projected/164ae7c1-c134-4f61-b7a7-e26fd53b0df2-kube-api-access-5d8sg\") pod \"164ae7c1-c134-4f61-b7a7-e26fd53b0df2\" (UID: \"164ae7c1-c134-4f61-b7a7-e26fd53b0df2\") "
Jul 29 15:19:56.324347 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:56.324308    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/164ae7c1-c134-4f61-b7a7-e26fd53b0df2-config-file" (OuterVolumeSpecName: "config-file") pod "164ae7c1-c134-4f61-b7a7-e26fd53b0df2" (UID: "164ae7c1-c134-4f61-b7a7-e26fd53b0df2"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:19:56.326375 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:56.326352    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/164ae7c1-c134-4f61-b7a7-e26fd53b0df2-kube-api-access-5d8sg" (OuterVolumeSpecName: "kube-api-access-5d8sg") pod "164ae7c1-c134-4f61-b7a7-e26fd53b0df2" (UID: "164ae7c1-c134-4f61-b7a7-e26fd53b0df2"). InnerVolumeSpecName "kube-api-access-5d8sg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:19:56.425179 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:56.425149    2575 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/164ae7c1-c134-4f61-b7a7-e26fd53b0df2-config-file\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:19:56.425179 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:56.425178    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5d8sg\" (UniqueName: \"kubernetes.io/projected/164ae7c1-c134-4f61-b7a7-e26fd53b0df2-kube-api-access-5d8sg\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:19:56.562331 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:56.562298    2575 generic.go:356] "Generic (PLEG): container finished" podID="164ae7c1-c134-4f61-b7a7-e26fd53b0df2" containerID="f00b743bb438f8c6e6b250ea9519f58cfeaadde9044ffa084d5a407e42830de6" exitCode=0
Jul 29 15:19:56.562714 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:56.562350    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh" event={"ID":"164ae7c1-c134-4f61-b7a7-e26fd53b0df2","Type":"ContainerDied","Data":"f00b743bb438f8c6e6b250ea9519f58cfeaadde9044ffa084d5a407e42830de6"}
Jul 29 15:19:56.562714 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:56.562372    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"
Jul 29 15:19:56.562714 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:56.562393    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wf2zh" event={"ID":"164ae7c1-c134-4f61-b7a7-e26fd53b0df2","Type":"ContainerDied","Data":"2319eeadf59c6191755c69f85a33a2530341b38bdc4da99200a9235059ba4bf1"}
Jul 29 15:19:56.562714 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:56.562418    2575 scope.go:117] "RemoveContainer" containerID="f00b743bb438f8c6e6b250ea9519f58cfeaadde9044ffa084d5a407e42830de6"
Jul 29 15:19:56.570791 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:56.570774    2575 scope.go:117] "RemoveContainer" containerID="f00b743bb438f8c6e6b250ea9519f58cfeaadde9044ffa084d5a407e42830de6"
Jul 29 15:19:56.571054 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:19:56.571036    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f00b743bb438f8c6e6b250ea9519f58cfeaadde9044ffa084d5a407e42830de6\": container with ID starting with f00b743bb438f8c6e6b250ea9519f58cfeaadde9044ffa084d5a407e42830de6 not found: ID does not exist" containerID="f00b743bb438f8c6e6b250ea9519f58cfeaadde9044ffa084d5a407e42830de6"
Jul 29 15:19:56.571099 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:56.571062    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f00b743bb438f8c6e6b250ea9519f58cfeaadde9044ffa084d5a407e42830de6"} err="failed to get container status \"f00b743bb438f8c6e6b250ea9519f58cfeaadde9044ffa084d5a407e42830de6\": rpc error: code = NotFound desc = could not find container \"f00b743bb438f8c6e6b250ea9519f58cfeaadde9044ffa084d5a407e42830de6\": container with ID starting with f00b743bb438f8c6e6b250ea9519f58cfeaadde9044ffa084d5a407e42830de6 not found: ID does not exist"
Jul 29 15:19:56.585101 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:56.585054    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"]
Jul 29 15:19:56.587977 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:56.587958    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wf2zh"]
Jul 29 15:19:57.298811 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:19:57.298773    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="164ae7c1-c134-4f61-b7a7-e26fd53b0df2" path="/var/lib/kubelet/pods/164ae7c1-c134-4f61-b7a7-e26fd53b0df2/volumes"
Jul 29 15:20:00.874149 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:00.874063    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-z4vnp"]
Jul 29 15:20:00.874519 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:00.874464    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="164ae7c1-c134-4f61-b7a7-e26fd53b0df2" containerName="limitador"
Jul 29 15:20:00.874519 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:00.874479    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="164ae7c1-c134-4f61-b7a7-e26fd53b0df2" containerName="limitador"
Jul 29 15:20:00.874601 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:00.874555    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="164ae7c1-c134-4f61-b7a7-e26fd53b0df2" containerName="limitador"
Jul 29 15:20:00.877489 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:00.877470    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z4vnp"
Jul 29 15:20:00.879884 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:00.879868    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 29 15:20:00.880033 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:00.880015    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-wqlkf\""
Jul 29 15:20:00.880226 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:00.880214    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 15:20:00.880552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:00.880539    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 15:20:00.895010 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:00.894989    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-z4vnp"]
Jul 29 15:20:00.965816 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:00.965787    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6c8373a2-3b2a-4b90-abfc-a063ab89ce84-data\") pod \"postgres-69fb4594cf-z4vnp\" (UID: \"6c8373a2-3b2a-4b90-abfc-a063ab89ce84\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z4vnp"
Jul 29 15:20:00.965952 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:00.965835    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b99bq\" (UniqueName: \"kubernetes.io/projected/6c8373a2-3b2a-4b90-abfc-a063ab89ce84-kube-api-access-b99bq\") pod \"postgres-69fb4594cf-z4vnp\" (UID: \"6c8373a2-3b2a-4b90-abfc-a063ab89ce84\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z4vnp"
Jul 29 15:20:01.067134 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:01.067109    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b99bq\" (UniqueName: \"kubernetes.io/projected/6c8373a2-3b2a-4b90-abfc-a063ab89ce84-kube-api-access-b99bq\") pod \"postgres-69fb4594cf-z4vnp\" (UID: \"6c8373a2-3b2a-4b90-abfc-a063ab89ce84\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z4vnp"
Jul 29 15:20:01.067292 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:01.067165    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6c8373a2-3b2a-4b90-abfc-a063ab89ce84-data\") pod \"postgres-69fb4594cf-z4vnp\" (UID: \"6c8373a2-3b2a-4b90-abfc-a063ab89ce84\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z4vnp"
Jul 29 15:20:01.067490 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:01.067471    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6c8373a2-3b2a-4b90-abfc-a063ab89ce84-data\") pod \"postgres-69fb4594cf-z4vnp\" (UID: \"6c8373a2-3b2a-4b90-abfc-a063ab89ce84\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z4vnp"
Jul 29 15:20:01.079634 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:01.079610    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b99bq\" (UniqueName: \"kubernetes.io/projected/6c8373a2-3b2a-4b90-abfc-a063ab89ce84-kube-api-access-b99bq\") pod \"postgres-69fb4594cf-z4vnp\" (UID: \"6c8373a2-3b2a-4b90-abfc-a063ab89ce84\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-z4vnp"
Jul 29 15:20:01.186700 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:01.186649    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z4vnp"
Jul 29 15:20:01.315429 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:01.315401    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-z4vnp"]
Jul 29 15:20:01.317480 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:20:01.317453    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c8373a2_3b2a_4b90_abfc_a063ab89ce84.slice/crio-a9c04ee3ba9642e8e644d3b66b0bf9a24d7f16f82b68c10af736bd91563bc169 WatchSource:0}: Error finding container a9c04ee3ba9642e8e644d3b66b0bf9a24d7f16f82b68c10af736bd91563bc169: Status 404 returned error can't find the container with id a9c04ee3ba9642e8e644d3b66b0bf9a24d7f16f82b68c10af736bd91563bc169
Jul 29 15:20:01.586283 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:01.586197    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z4vnp" event={"ID":"6c8373a2-3b2a-4b90-abfc-a063ab89ce84","Type":"ContainerStarted","Data":"a9c04ee3ba9642e8e644d3b66b0bf9a24d7f16f82b68c10af736bd91563bc169"}
Jul 29 15:20:07.618256 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:07.618219    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z4vnp" event={"ID":"6c8373a2-3b2a-4b90-abfc-a063ab89ce84","Type":"ContainerStarted","Data":"5d06fe90247f201c5cdb062cbcfa661c975db4591170bc0e63126c1a01e6f23e"}
Jul 29 15:20:07.618610 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:07.618302    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z4vnp"
Jul 29 15:20:07.637080 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:07.637035    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z4vnp" podStartSLOduration=2.009659452 podStartE2EDuration="7.637019759s" totalImagesPullingTime="5.627360307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:20:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:20:07.635601829 +0000 UTC m=+540.784895725" watchObservedRunningTime="2026-07-29 15:20:07.637019759 +0000 UTC m=+540.786313656"
Jul 29 15:20:13.650433 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:13.650398    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-z4vnp"
Jul 29 15:20:21.088055 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.088023    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f686dc7cf-64s56"]
Jul 29 15:20:21.116720 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.116692    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f686dc7cf-64s56"]
Jul 29 15:20:21.116882 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.116790    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f686dc7cf-64s56"
Jul 29 15:20:21.119453 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.119429    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 15:20:21.119596 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.119512    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-96jgf\""
Jul 29 15:20:21.119596 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.119534    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 15:20:21.243495 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.243462    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a33ac860-b71b-48de-b95e-1575e64e6f23-webhook-cert\") pod \"maas-controller-5f686dc7cf-64s56\" (UID: \"a33ac860-b71b-48de-b95e-1575e64e6f23\") " pod="opendatahub/maas-controller-5f686dc7cf-64s56"
Jul 29 15:20:21.243677 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.243514    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k84vl\" (UniqueName: \"kubernetes.io/projected/a33ac860-b71b-48de-b95e-1575e64e6f23-kube-api-access-k84vl\") pod \"maas-controller-5f686dc7cf-64s56\" (UID: \"a33ac860-b71b-48de-b95e-1575e64e6f23\") " pod="opendatahub/maas-controller-5f686dc7cf-64s56"
Jul 29 15:20:21.280379 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.280345    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8497979899-44mhv"]
Jul 29 15:20:21.283886 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.283865    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8497979899-44mhv"
Jul 29 15:20:21.297881 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.297817    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8497979899-44mhv"]
Jul 29 15:20:21.344789 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.344723    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a33ac860-b71b-48de-b95e-1575e64e6f23-webhook-cert\") pod \"maas-controller-5f686dc7cf-64s56\" (UID: \"a33ac860-b71b-48de-b95e-1575e64e6f23\") " pod="opendatahub/maas-controller-5f686dc7cf-64s56"
Jul 29 15:20:21.344789 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.344774    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k84vl\" (UniqueName: \"kubernetes.io/projected/a33ac860-b71b-48de-b95e-1575e64e6f23-kube-api-access-k84vl\") pod \"maas-controller-5f686dc7cf-64s56\" (UID: \"a33ac860-b71b-48de-b95e-1575e64e6f23\") " pod="opendatahub/maas-controller-5f686dc7cf-64s56"
Jul 29 15:20:21.345022 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:20:21.344871    2575 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 29 15:20:21.345022 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:20:21.344947    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a33ac860-b71b-48de-b95e-1575e64e6f23-webhook-cert podName:a33ac860-b71b-48de-b95e-1575e64e6f23 nodeName:}" failed. No retries permitted until 2026-07-29 15:20:21.844915976 +0000 UTC m=+554.994209862 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/a33ac860-b71b-48de-b95e-1575e64e6f23-webhook-cert") pod "maas-controller-5f686dc7cf-64s56" (UID: "a33ac860-b71b-48de-b95e-1575e64e6f23") : secret "maas-controller-webhook-cert" not found
Jul 29 15:20:21.363535 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.363511    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k84vl\" (UniqueName: \"kubernetes.io/projected/a33ac860-b71b-48de-b95e-1575e64e6f23-kube-api-access-k84vl\") pod \"maas-controller-5f686dc7cf-64s56\" (UID: \"a33ac860-b71b-48de-b95e-1575e64e6f23\") " pod="opendatahub/maas-controller-5f686dc7cf-64s56"
Jul 29 15:20:21.445465 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.445431    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/950ff7b9-ba8a-442a-b2da-2f92f09ce235-webhook-cert\") pod \"maas-controller-8497979899-44mhv\" (UID: \"950ff7b9-ba8a-442a-b2da-2f92f09ce235\") " pod="opendatahub/maas-controller-8497979899-44mhv"
Jul 29 15:20:21.445660 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.445568    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsknl\" (UniqueName: \"kubernetes.io/projected/950ff7b9-ba8a-442a-b2da-2f92f09ce235-kube-api-access-rsknl\") pod \"maas-controller-8497979899-44mhv\" (UID: \"950ff7b9-ba8a-442a-b2da-2f92f09ce235\") " pod="opendatahub/maas-controller-8497979899-44mhv"
Jul 29 15:20:21.546910 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.546879    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/950ff7b9-ba8a-442a-b2da-2f92f09ce235-webhook-cert\") pod \"maas-controller-8497979899-44mhv\" (UID: \"950ff7b9-ba8a-442a-b2da-2f92f09ce235\") " pod="opendatahub/maas-controller-8497979899-44mhv"
Jul 29 15:20:21.547082 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.546924    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsknl\" (UniqueName: \"kubernetes.io/projected/950ff7b9-ba8a-442a-b2da-2f92f09ce235-kube-api-access-rsknl\") pod \"maas-controller-8497979899-44mhv\" (UID: \"950ff7b9-ba8a-442a-b2da-2f92f09ce235\") " pod="opendatahub/maas-controller-8497979899-44mhv"
Jul 29 15:20:21.547082 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:20:21.547020    2575 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 29 15:20:21.547192 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:20:21.547083    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/950ff7b9-ba8a-442a-b2da-2f92f09ce235-webhook-cert podName:950ff7b9-ba8a-442a-b2da-2f92f09ce235 nodeName:}" failed. No retries permitted until 2026-07-29 15:20:22.047065389 +0000 UTC m=+555.196359281 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/950ff7b9-ba8a-442a-b2da-2f92f09ce235-webhook-cert") pod "maas-controller-8497979899-44mhv" (UID: "950ff7b9-ba8a-442a-b2da-2f92f09ce235") : secret "maas-controller-webhook-cert" not found
Jul 29 15:20:21.562880 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.562851    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsknl\" (UniqueName: \"kubernetes.io/projected/950ff7b9-ba8a-442a-b2da-2f92f09ce235-kube-api-access-rsknl\") pod \"maas-controller-8497979899-44mhv\" (UID: \"950ff7b9-ba8a-442a-b2da-2f92f09ce235\") " pod="opendatahub/maas-controller-8497979899-44mhv"
Jul 29 15:20:21.849614 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.849579    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a33ac860-b71b-48de-b95e-1575e64e6f23-webhook-cert\") pod \"maas-controller-5f686dc7cf-64s56\" (UID: \"a33ac860-b71b-48de-b95e-1575e64e6f23\") " pod="opendatahub/maas-controller-5f686dc7cf-64s56"
Jul 29 15:20:21.851926 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:21.851906    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a33ac860-b71b-48de-b95e-1575e64e6f23-webhook-cert\") pod \"maas-controller-5f686dc7cf-64s56\" (UID: \"a33ac860-b71b-48de-b95e-1575e64e6f23\") " pod="opendatahub/maas-controller-5f686dc7cf-64s56"
Jul 29 15:20:22.027391 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:22.027348    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f686dc7cf-64s56"
Jul 29 15:20:22.051407 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:22.051380    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/950ff7b9-ba8a-442a-b2da-2f92f09ce235-webhook-cert\") pod \"maas-controller-8497979899-44mhv\" (UID: \"950ff7b9-ba8a-442a-b2da-2f92f09ce235\") " pod="opendatahub/maas-controller-8497979899-44mhv"
Jul 29 15:20:22.053560 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:22.053537    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/950ff7b9-ba8a-442a-b2da-2f92f09ce235-webhook-cert\") pod \"maas-controller-8497979899-44mhv\" (UID: \"950ff7b9-ba8a-442a-b2da-2f92f09ce235\") " pod="opendatahub/maas-controller-8497979899-44mhv"
Jul 29 15:20:22.153489 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:22.153463    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f686dc7cf-64s56"]
Jul 29 15:20:22.155739 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:20:22.155712    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda33ac860_b71b_48de_b95e_1575e64e6f23.slice/crio-c3821cb82992518936b8c4e4119006a97e0f397733d7ddc11f548b4c33fb64f6 WatchSource:0}: Error finding container c3821cb82992518936b8c4e4119006a97e0f397733d7ddc11f548b4c33fb64f6: Status 404 returned error can't find the container with id c3821cb82992518936b8c4e4119006a97e0f397733d7ddc11f548b4c33fb64f6
Jul 29 15:20:22.194576 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:22.194556    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8497979899-44mhv"
Jul 29 15:20:22.317563 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:22.317541    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8497979899-44mhv"]
Jul 29 15:20:22.320203 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:20:22.320177    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod950ff7b9_ba8a_442a_b2da_2f92f09ce235.slice/crio-840cd46f72a66d4f8ef44e21f27da4ad45916a3642e445b5a78f26b610e437bc WatchSource:0}: Error finding container 840cd46f72a66d4f8ef44e21f27da4ad45916a3642e445b5a78f26b610e437bc: Status 404 returned error can't find the container with id 840cd46f72a66d4f8ef44e21f27da4ad45916a3642e445b5a78f26b610e437bc
Jul 29 15:20:22.682606 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:22.682570    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8497979899-44mhv" event={"ID":"950ff7b9-ba8a-442a-b2da-2f92f09ce235","Type":"ContainerStarted","Data":"840cd46f72a66d4f8ef44e21f27da4ad45916a3642e445b5a78f26b610e437bc"}
Jul 29 15:20:22.683620 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:22.683596    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f686dc7cf-64s56" event={"ID":"a33ac860-b71b-48de-b95e-1575e64e6f23","Type":"ContainerStarted","Data":"c3821cb82992518936b8c4e4119006a97e0f397733d7ddc11f548b4c33fb64f6"}
Jul 29 15:20:25.702053 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:25.702013    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8497979899-44mhv" event={"ID":"950ff7b9-ba8a-442a-b2da-2f92f09ce235","Type":"ContainerStarted","Data":"d24da9e5e8fdab2c6c3240ce3594e67ba8b22891f28dbcb7ee532bb718c6fe80"}
Jul 29 15:20:25.702524 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:25.702081    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8497979899-44mhv"
Jul 29 15:20:25.703606 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:25.703578    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f686dc7cf-64s56" event={"ID":"a33ac860-b71b-48de-b95e-1575e64e6f23","Type":"ContainerStarted","Data":"d63b07e98fc3c5e4faf0bfde5748dbfe4d51172ddf884ca2fe2d8c16c5b0d949"}
Jul 29 15:20:25.703721 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:25.703649    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f686dc7cf-64s56"
Jul 29 15:20:25.720800 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:25.720753    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8497979899-44mhv" podStartSLOduration=2.2749453170000002 podStartE2EDuration="4.720735425s" totalImagesPullingTime="2.445790108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:20:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:20:25.719655778 +0000 UTC m=+558.868949677" watchObservedRunningTime="2026-07-29 15:20:25.720735425 +0000 UTC m=+558.870029325"
Jul 29 15:20:25.737295 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:25.737251    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f686dc7cf-64s56" podStartSLOduration=2.133362884 podStartE2EDuration="4.737235952s" totalImagesPullingTime="2.603873068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:20:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:20:25.736309802 +0000 UTC m=+558.885603703" watchObservedRunningTime="2026-07-29 15:20:25.737235952 +0000 UTC m=+558.886529882"
Jul 29 15:20:36.710065 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:36.710035    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8497979899-44mhv"
Jul 29 15:20:36.710535 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:36.710111    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f686dc7cf-64s56"
Jul 29 15:20:36.778727 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:36.778698    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f686dc7cf-64s56"]
Jul 29 15:20:36.779086 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:36.778860    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5f686dc7cf-64s56" podUID="a33ac860-b71b-48de-b95e-1575e64e6f23" containerName="manager" containerID="cri-o://d63b07e98fc3c5e4faf0bfde5748dbfe4d51172ddf884ca2fe2d8c16c5b0d949" gracePeriod=30
Jul 29 15:20:37.021862 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:37.021837    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f686dc7cf-64s56"
Jul 29 15:20:37.082491 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:37.082466    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a33ac860-b71b-48de-b95e-1575e64e6f23-webhook-cert\") pod \"a33ac860-b71b-48de-b95e-1575e64e6f23\" (UID: \"a33ac860-b71b-48de-b95e-1575e64e6f23\") "
Jul 29 15:20:37.082635 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:37.082553    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k84vl\" (UniqueName: \"kubernetes.io/projected/a33ac860-b71b-48de-b95e-1575e64e6f23-kube-api-access-k84vl\") pod \"a33ac860-b71b-48de-b95e-1575e64e6f23\" (UID: \"a33ac860-b71b-48de-b95e-1575e64e6f23\") "
Jul 29 15:20:37.084489 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:37.084464    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a33ac860-b71b-48de-b95e-1575e64e6f23-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a33ac860-b71b-48de-b95e-1575e64e6f23" (UID: "a33ac860-b71b-48de-b95e-1575e64e6f23"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:20:37.084601 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:37.084488    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a33ac860-b71b-48de-b95e-1575e64e6f23-kube-api-access-k84vl" (OuterVolumeSpecName: "kube-api-access-k84vl") pod "a33ac860-b71b-48de-b95e-1575e64e6f23" (UID: "a33ac860-b71b-48de-b95e-1575e64e6f23"). InnerVolumeSpecName "kube-api-access-k84vl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:20:37.183206 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:37.183176    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k84vl\" (UniqueName: \"kubernetes.io/projected/a33ac860-b71b-48de-b95e-1575e64e6f23-kube-api-access-k84vl\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:20:37.183206 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:37.183204    2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a33ac860-b71b-48de-b95e-1575e64e6f23-webhook-cert\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:20:37.758396 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:37.758360    2575 generic.go:356] "Generic (PLEG): container finished" podID="a33ac860-b71b-48de-b95e-1575e64e6f23" containerID="d63b07e98fc3c5e4faf0bfde5748dbfe4d51172ddf884ca2fe2d8c16c5b0d949" exitCode=0
Jul 29 15:20:37.758784 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:37.758409    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f686dc7cf-64s56"
Jul 29 15:20:37.758784 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:37.758420    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f686dc7cf-64s56" event={"ID":"a33ac860-b71b-48de-b95e-1575e64e6f23","Type":"ContainerDied","Data":"d63b07e98fc3c5e4faf0bfde5748dbfe4d51172ddf884ca2fe2d8c16c5b0d949"}
Jul 29 15:20:37.758784 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:37.758471    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f686dc7cf-64s56" event={"ID":"a33ac860-b71b-48de-b95e-1575e64e6f23","Type":"ContainerDied","Data":"c3821cb82992518936b8c4e4119006a97e0f397733d7ddc11f548b4c33fb64f6"}
Jul 29 15:20:37.758784 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:37.758500    2575 scope.go:117] "RemoveContainer" containerID="d63b07e98fc3c5e4faf0bfde5748dbfe4d51172ddf884ca2fe2d8c16c5b0d949"
Jul 29 15:20:37.766408 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:37.766374    2575 scope.go:117] "RemoveContainer" containerID="d63b07e98fc3c5e4faf0bfde5748dbfe4d51172ddf884ca2fe2d8c16c5b0d949"
Jul 29 15:20:37.766662 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:20:37.766642    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d63b07e98fc3c5e4faf0bfde5748dbfe4d51172ddf884ca2fe2d8c16c5b0d949\": container with ID starting with d63b07e98fc3c5e4faf0bfde5748dbfe4d51172ddf884ca2fe2d8c16c5b0d949 not found: ID does not exist" containerID="d63b07e98fc3c5e4faf0bfde5748dbfe4d51172ddf884ca2fe2d8c16c5b0d949"
Jul 29 15:20:37.766709 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:37.766671    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d63b07e98fc3c5e4faf0bfde5748dbfe4d51172ddf884ca2fe2d8c16c5b0d949"} err="failed to get container status \"d63b07e98fc3c5e4faf0bfde5748dbfe4d51172ddf884ca2fe2d8c16c5b0d949\": rpc error: code = NotFound desc = could not find container \"d63b07e98fc3c5e4faf0bfde5748dbfe4d51172ddf884ca2fe2d8c16c5b0d949\": container with ID starting with d63b07e98fc3c5e4faf0bfde5748dbfe4d51172ddf884ca2fe2d8c16c5b0d949 not found: ID does not exist"
Jul 29 15:20:37.782156 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:37.782132    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f686dc7cf-64s56"]
Jul 29 15:20:37.785829 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:37.785809    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f686dc7cf-64s56"]
Jul 29 15:20:39.299334 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:39.299300    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a33ac860-b71b-48de-b95e-1575e64e6f23" path="/var/lib/kubelet/pods/a33ac860-b71b-48de-b95e-1575e64e6f23/volumes"
Jul 29 15:20:56.862246 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:56.862214    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-847fd45f6b-52k77"]
Jul 29 15:20:56.862704 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:56.862546    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a33ac860-b71b-48de-b95e-1575e64e6f23" containerName="manager"
Jul 29 15:20:56.862704 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:56.862556    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a33ac860-b71b-48de-b95e-1575e64e6f23" containerName="manager"
Jul 29 15:20:56.862704 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:56.862616    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a33ac860-b71b-48de-b95e-1575e64e6f23" containerName="manager"
Jul 29 15:20:56.866790 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:56.866774    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-847fd45f6b-52k77"
Jul 29 15:20:56.869577 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:56.869556    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 29 15:20:56.870585 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:56.870568    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-2jzb8\""
Jul 29 15:20:56.878444 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:56.878417    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-847fd45f6b-52k77"]
Jul 29 15:20:56.946674 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:56.946647    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8b103fd-382c-45cb-bce6-71f59a5f33ef-maas-api-tls\") pod \"maas-api-847fd45f6b-52k77\" (UID: \"d8b103fd-382c-45cb-bce6-71f59a5f33ef\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-52k77"
Jul 29 15:20:56.946827 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:56.946688    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf42f\" (UniqueName: \"kubernetes.io/projected/d8b103fd-382c-45cb-bce6-71f59a5f33ef-kube-api-access-kf42f\") pod \"maas-api-847fd45f6b-52k77\" (UID: \"d8b103fd-382c-45cb-bce6-71f59a5f33ef\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-52k77"
Jul 29 15:20:56.946827 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:56.946706    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b103fd-382c-45cb-bce6-71f59a5f33ef-openshift-service-ca\") pod \"maas-api-847fd45f6b-52k77\" (UID: \"d8b103fd-382c-45cb-bce6-71f59a5f33ef\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-52k77"
Jul 29 15:20:57.047709 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.047681    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8b103fd-382c-45cb-bce6-71f59a5f33ef-maas-api-tls\") pod \"maas-api-847fd45f6b-52k77\" (UID: \"d8b103fd-382c-45cb-bce6-71f59a5f33ef\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-52k77"
Jul 29 15:20:57.047864 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.047723    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf42f\" (UniqueName: \"kubernetes.io/projected/d8b103fd-382c-45cb-bce6-71f59a5f33ef-kube-api-access-kf42f\") pod \"maas-api-847fd45f6b-52k77\" (UID: \"d8b103fd-382c-45cb-bce6-71f59a5f33ef\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-52k77"
Jul 29 15:20:57.047864 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.047744    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b103fd-382c-45cb-bce6-71f59a5f33ef-openshift-service-ca\") pod \"maas-api-847fd45f6b-52k77\" (UID: \"d8b103fd-382c-45cb-bce6-71f59a5f33ef\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-52k77"
Jul 29 15:20:57.048306 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.048288    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b103fd-382c-45cb-bce6-71f59a5f33ef-openshift-service-ca\") pod \"maas-api-847fd45f6b-52k77\" (UID: \"d8b103fd-382c-45cb-bce6-71f59a5f33ef\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-52k77"
Jul 29 15:20:57.049838 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.049819    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8b103fd-382c-45cb-bce6-71f59a5f33ef-maas-api-tls\") pod \"maas-api-847fd45f6b-52k77\" (UID: \"d8b103fd-382c-45cb-bce6-71f59a5f33ef\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-52k77"
Jul 29 15:20:57.058733 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.058710    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf42f\" (UniqueName: \"kubernetes.io/projected/d8b103fd-382c-45cb-bce6-71f59a5f33ef-kube-api-access-kf42f\") pod \"maas-api-847fd45f6b-52k77\" (UID: \"d8b103fd-382c-45cb-bce6-71f59a5f33ef\") " pod="odh-ai-gateway-infra/maas-api-847fd45f6b-52k77"
Jul 29 15:20:57.176544 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.176456    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-847fd45f6b-52k77"
Jul 29 15:20:57.312155 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.312131    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-847fd45f6b-52k77"]
Jul 29 15:20:57.314516 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:20:57.314488    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8b103fd_382c_45cb_bce6_71f59a5f33ef.slice/crio-35c88b59dab7402ea6a98c5c5fa9398ddd5ddb5f5ca555dd33813e1e1c87cd0c WatchSource:0}: Error finding container 35c88b59dab7402ea6a98c5c5fa9398ddd5ddb5f5ca555dd33813e1e1c87cd0c: Status 404 returned error can't find the container with id 35c88b59dab7402ea6a98c5c5fa9398ddd5ddb5f5ca555dd33813e1e1c87cd0c
Jul 29 15:20:57.561522 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.561450    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-n8z9v"]
Jul 29 15:20:57.565929 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.565908    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-n8z9v"
Jul 29 15:20:57.569033 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.569012    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 29 15:20:57.569381 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.569366    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-5s52c\""
Jul 29 15:20:57.574743 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.574719    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-knc5p"]
Jul 29 15:20:57.578622 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.578604    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-n8z9v"]
Jul 29 15:20:57.578717 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.578682    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-knc5p"
Jul 29 15:20:57.594579 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.594553    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-knc5p"]
Jul 29 15:20:57.652424 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.652386    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/244016a2-b3e4-4f13-a387-6197b07e8be2-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-n8z9v\" (UID: \"244016a2-b3e4-4f13-a387-6197b07e8be2\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-n8z9v"
Jul 29 15:20:57.652619 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.652435    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc6dl\" (UniqueName: \"kubernetes.io/projected/244016a2-b3e4-4f13-a387-6197b07e8be2-kube-api-access-qc6dl\") pod \"payload-pre-processing-687dbb6cf6-n8z9v\" (UID: \"244016a2-b3e4-4f13-a387-6197b07e8be2\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-n8z9v"
Jul 29 15:20:57.652835 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.652790    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwzbq\" (UniqueName: \"kubernetes.io/projected/6fb56df1-ee67-48c1-be21-5b9faa36351e-kube-api-access-kwzbq\") pod \"payload-processing-5cc6cd444b-knc5p\" (UID: \"6fb56df1-ee67-48c1-be21-5b9faa36351e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-knc5p"
Jul 29 15:20:57.653056 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.653036    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fb56df1-ee67-48c1-be21-5b9faa36351e-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-knc5p\" (UID: \"6fb56df1-ee67-48c1-be21-5b9faa36351e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-knc5p"
Jul 29 15:20:57.754476 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.754440    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/244016a2-b3e4-4f13-a387-6197b07e8be2-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-n8z9v\" (UID: \"244016a2-b3e4-4f13-a387-6197b07e8be2\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-n8z9v"
Jul 29 15:20:57.754476 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.754474    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc6dl\" (UniqueName: \"kubernetes.io/projected/244016a2-b3e4-4f13-a387-6197b07e8be2-kube-api-access-qc6dl\") pod \"payload-pre-processing-687dbb6cf6-n8z9v\" (UID: \"244016a2-b3e4-4f13-a387-6197b07e8be2\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-n8z9v"
Jul 29 15:20:57.754686 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.754517    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwzbq\" (UniqueName: \"kubernetes.io/projected/6fb56df1-ee67-48c1-be21-5b9faa36351e-kube-api-access-kwzbq\") pod \"payload-processing-5cc6cd444b-knc5p\" (UID: \"6fb56df1-ee67-48c1-be21-5b9faa36351e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-knc5p"
Jul 29 15:20:57.754686 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.754551    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fb56df1-ee67-48c1-be21-5b9faa36351e-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-knc5p\" (UID: \"6fb56df1-ee67-48c1-be21-5b9faa36351e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-knc5p"
Jul 29 15:20:57.755291 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.755271    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fb56df1-ee67-48c1-be21-5b9faa36351e-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-knc5p\" (UID: \"6fb56df1-ee67-48c1-be21-5b9faa36351e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-knc5p"
Jul 29 15:20:57.755392 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.755367    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/244016a2-b3e4-4f13-a387-6197b07e8be2-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-n8z9v\" (UID: \"244016a2-b3e4-4f13-a387-6197b07e8be2\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-n8z9v"
Jul 29 15:20:57.767943 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.767904    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwzbq\" (UniqueName: \"kubernetes.io/projected/6fb56df1-ee67-48c1-be21-5b9faa36351e-kube-api-access-kwzbq\") pod \"payload-processing-5cc6cd444b-knc5p\" (UID: \"6fb56df1-ee67-48c1-be21-5b9faa36351e\") " pod="openshift-ingress/payload-processing-5cc6cd444b-knc5p"
Jul 29 15:20:57.768102 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.767948    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc6dl\" (UniqueName: \"kubernetes.io/projected/244016a2-b3e4-4f13-a387-6197b07e8be2-kube-api-access-qc6dl\") pod \"payload-pre-processing-687dbb6cf6-n8z9v\" (UID: \"244016a2-b3e4-4f13-a387-6197b07e8be2\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-n8z9v"
Jul 29 15:20:57.848726 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.848639    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-847fd45f6b-52k77" event={"ID":"d8b103fd-382c-45cb-bce6-71f59a5f33ef","Type":"ContainerStarted","Data":"35c88b59dab7402ea6a98c5c5fa9398ddd5ddb5f5ca555dd33813e1e1c87cd0c"}
Jul 29 15:20:57.878003 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.877973    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-n8z9v"
Jul 29 15:20:57.888191 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:57.888168    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-knc5p"
Jul 29 15:20:58.050475 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:58.050390    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-n8z9v"]
Jul 29 15:20:58.052124 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:20:58.052097    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod244016a2_b3e4_4f13_a387_6197b07e8be2.slice/crio-f335c9d595b855f22d906f18c4a688ccd109caae2f557b12a1eead988162eb78 WatchSource:0}: Error finding container f335c9d595b855f22d906f18c4a688ccd109caae2f557b12a1eead988162eb78: Status 404 returned error can't find the container with id f335c9d595b855f22d906f18c4a688ccd109caae2f557b12a1eead988162eb78
Jul 29 15:20:58.073376 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:58.073356    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-knc5p"]
Jul 29 15:20:58.075622 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:20:58.075599    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb56df1_ee67_48c1_be21_5b9faa36351e.slice/crio-b87b047d5413005fab0a6973435c780d0028e64d48b8f04f2a9469b5844143ab WatchSource:0}: Error finding container b87b047d5413005fab0a6973435c780d0028e64d48b8f04f2a9469b5844143ab: Status 404 returned error can't find the container with id b87b047d5413005fab0a6973435c780d0028e64d48b8f04f2a9469b5844143ab
Jul 29 15:20:58.859124 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:58.859068    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-knc5p" event={"ID":"6fb56df1-ee67-48c1-be21-5b9faa36351e","Type":"ContainerStarted","Data":"b87b047d5413005fab0a6973435c780d0028e64d48b8f04f2a9469b5844143ab"}
Jul 29 15:20:58.861494 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:58.861461    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-n8z9v" event={"ID":"244016a2-b3e4-4f13-a387-6197b07e8be2","Type":"ContainerStarted","Data":"f335c9d595b855f22d906f18c4a688ccd109caae2f557b12a1eead988162eb78"}
Jul 29 15:20:59.868774 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:59.868728    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-847fd45f6b-52k77" event={"ID":"d8b103fd-382c-45cb-bce6-71f59a5f33ef","Type":"ContainerStarted","Data":"943ca7b920b733c9ed90e15866738741c2d28ebe734d89bda365ef8a830b8488"}
Jul 29 15:20:59.869230 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:59.868901    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-847fd45f6b-52k77"
Jul 29 15:20:59.886605 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:20:59.886545    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-847fd45f6b-52k77" podStartSLOduration=1.862733736 podStartE2EDuration="3.886524558s" totalImagesPullingTime="2.023790822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:20:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:20:59.8853754 +0000 UTC m=+593.034669299" watchObservedRunningTime="2026-07-29 15:20:59.886524558 +0000 UTC m=+593.035818460"
Jul 29 15:21:00.874702 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:00.874655    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-knc5p" event={"ID":"6fb56df1-ee67-48c1-be21-5b9faa36351e","Type":"ContainerStarted","Data":"c7488a8630d5ebfff5dddb79706e9734c3522faf9dd5077c5ae6331e2e33eea2"}
Jul 29 15:21:00.875139 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:00.874716    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-knc5p"
Jul 29 15:21:00.876165 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:00.876137    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-n8z9v" event={"ID":"244016a2-b3e4-4f13-a387-6197b07e8be2","Type":"ContainerStarted","Data":"d3567b4c3af01899e0fc5775963753ca66655877de3ed29c32fc581575e3e3fa"}
Jul 29 15:21:00.876350 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:00.876334    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-n8z9v"
Jul 29 15:21:00.892059 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:00.892007    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-knc5p" podStartSLOduration=1.855912751 podStartE2EDuration="3.891989459s" totalImagesPullingTime="2.036076708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:20:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:21:00.891363616 +0000 UTC m=+594.040657515" watchObservedRunningTime="2026-07-29 15:21:00.891989459 +0000 UTC m=+594.041283360"
Jul 29 15:21:00.908700 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:00.908661    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-n8z9v" podStartSLOduration=1.857883696 podStartE2EDuration="3.908644904s" totalImagesPullingTime="2.050761208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:20:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:21:00.907629605 +0000 UTC m=+594.056923504" watchObservedRunningTime="2026-07-29 15:21:00.908644904 +0000 UTC m=+594.057938804"
Jul 29 15:21:05.881549 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:05.881515    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-847fd45f6b-52k77"
Jul 29 15:21:07.224781 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:07.224746    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:21:07.227101 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:07.227081    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:21:07.231528 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:07.231511    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:21:07.233533 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:07.233518    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:21:11.882515 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:11.882487    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-n8z9v"
Jul 29 15:21:11.882938 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:11.882704    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-knc5p"
Jul 29 15:21:25.788379 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.788333    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"]
Jul 29 15:21:25.791731 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.791715    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:25.794867 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.794844    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lc787\""
Jul 29 15:21:25.794993 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.794866    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 15:21:25.794993 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.794866    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 15:21:25.794993 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.794846    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 29 15:21:25.802377 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.802350    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"]
Jul 29 15:21:25.892889 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.892855    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4bxc\" (UniqueName: \"kubernetes.io/projected/fd94d310-9226-4c39-adf9-3c5d8a11e129-kube-api-access-g4bxc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:25.893089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.892896    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd94d310-9226-4c39-adf9-3c5d8a11e129-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:25.893089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.892936    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd94d310-9226-4c39-adf9-3c5d8a11e129-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:25.893089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.892978    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd94d310-9226-4c39-adf9-3c5d8a11e129-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:25.893089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.893043    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd94d310-9226-4c39-adf9-3c5d8a11e129-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:25.893089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.893078    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd94d310-9226-4c39-adf9-3c5d8a11e129-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:25.994043 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.994012    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4bxc\" (UniqueName: \"kubernetes.io/projected/fd94d310-9226-4c39-adf9-3c5d8a11e129-kube-api-access-g4bxc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:25.994043 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.994049    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd94d310-9226-4c39-adf9-3c5d8a11e129-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:25.994300 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.994093    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd94d310-9226-4c39-adf9-3c5d8a11e129-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:25.994300 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.994118    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd94d310-9226-4c39-adf9-3c5d8a11e129-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:25.994300 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.994249    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd94d310-9226-4c39-adf9-3c5d8a11e129-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:25.994506 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.994299    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd94d310-9226-4c39-adf9-3c5d8a11e129-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:25.994571 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.994553    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fd94d310-9226-4c39-adf9-3c5d8a11e129-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:25.994631 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.994574    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fd94d310-9226-4c39-adf9-3c5d8a11e129-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:25.994686 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.994649    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd94d310-9226-4c39-adf9-3c5d8a11e129-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:25.996474 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.996446    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fd94d310-9226-4c39-adf9-3c5d8a11e129-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:25.996713 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:25.996695    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd94d310-9226-4c39-adf9-3c5d8a11e129-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:26.003535 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:26.003512    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4bxc\" (UniqueName: \"kubernetes.io/projected/fd94d310-9226-4c39-adf9-3c5d8a11e129-kube-api-access-g4bxc\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m\" (UID: \"fd94d310-9226-4c39-adf9-3c5d8a11e129\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:26.103616 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:26.103525    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:26.244508 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:26.244462    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"]
Jul 29 15:21:26.247423 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:21:26.247245    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd94d310_9226_4c39_adf9_3c5d8a11e129.slice/crio-216ce5407b08afb3f7f1f3005859d10f9fe788660ffa5e3169e391fbf9388209 WatchSource:0}: Error finding container 216ce5407b08afb3f7f1f3005859d10f9fe788660ffa5e3169e391fbf9388209: Status 404 returned error can't find the container with id 216ce5407b08afb3f7f1f3005859d10f9fe788660ffa5e3169e391fbf9388209
Jul 29 15:21:26.252886 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:26.252859    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 15:21:27.002426 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:27.002388    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m" event={"ID":"fd94d310-9226-4c39-adf9-3c5d8a11e129","Type":"ContainerStarted","Data":"216ce5407b08afb3f7f1f3005859d10f9fe788660ffa5e3169e391fbf9388209"}
Jul 29 15:21:28.008286 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:28.008250    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m" event={"ID":"fd94d310-9226-4c39-adf9-3c5d8a11e129","Type":"ContainerStarted","Data":"0c3b9d26877caa1c4b00030228ea662e2628ad16388c62154842eb659e5e03de"}
Jul 29 15:21:28.033561 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:28.033502    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m" podStartSLOduration=2.028368182 podStartE2EDuration="3.03348125s" totalImagesPullingTime="1.005113068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:21:28.032842089 +0000 UTC m=+621.182135990" watchObservedRunningTime="2026-07-29 15:21:28.03348125 +0000 UTC m=+621.182775155"
Jul 29 15:21:36.104035 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:36.103950    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:36.104035 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:36.104025    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:36.116789 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:36.116764    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:21:37.060352 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:21:37.060314    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m"
Jul 29 15:22:02.485303 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.485269    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"]
Jul 29 15:22:02.490038 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.490021    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.492725 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.492705    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 29 15:22:02.500459 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.500436    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"]
Jul 29 15:22:02.606654 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.606617    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.606822 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.606694    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.606822 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.606719    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.606822 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.606740    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfj5h\" (UniqueName: \"kubernetes.io/projected/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-kube-api-access-dfj5h\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.606822 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.606772    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.606960 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.606864    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.707782 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.707751    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.707782 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.707788    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.708036 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.707845    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.708036 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.707870    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.708036 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.707894    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfj5h\" (UniqueName: \"kubernetes.io/projected/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-kube-api-access-dfj5h\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.708036 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.707994    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.708303 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.708279    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.708416 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.708307    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.708416 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.708371    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.710133 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.710104    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.710403 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.710383    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.717968 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.717943    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfj5h\" (UniqueName: \"kubernetes.io/projected/14d20cc8-0ed5-4f5b-a3b9-f95d7022e336-kube-api-access-dfj5h\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc\" (UID: \"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.800312 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.800221    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:02.939514 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:02.939484    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"]
Jul 29 15:22:02.942226 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:22:02.942196    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14d20cc8_0ed5_4f5b_a3b9_f95d7022e336.slice/crio-2221f2054a6677857ec6e15f80e2dfd3d013427f4b8ce956de4f3741d8acd201 WatchSource:0}: Error finding container 2221f2054a6677857ec6e15f80e2dfd3d013427f4b8ce956de4f3741d8acd201: Status 404 returned error can't find the container with id 2221f2054a6677857ec6e15f80e2dfd3d013427f4b8ce956de4f3741d8acd201
Jul 29 15:22:03.171458 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:03.171424    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc" event={"ID":"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336","Type":"ContainerStarted","Data":"2221f2054a6677857ec6e15f80e2dfd3d013427f4b8ce956de4f3741d8acd201"}
Jul 29 15:22:04.178000 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:04.177967    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc" event={"ID":"14d20cc8-0ed5-4f5b-a3b9-f95d7022e336","Type":"ContainerStarted","Data":"d3ec1fefc724358dc6aabf4103847e5ecbe212d128f9e6ab186ba89feacc1954"}
Jul 29 15:22:04.200530 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:04.200470    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc" podStartSLOduration=1.9602718220000002 podStartE2EDuration="2.200451853s" totalImagesPullingTime="240.180031ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:22:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:22:04.199749587 +0000 UTC m=+657.349043486" watchObservedRunningTime="2026-07-29 15:22:04.200451853 +0000 UTC m=+657.349745758"
Jul 29 15:22:12.800628 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:12.800595    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:12.801017 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:12.800652    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:12.812935 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:12.812910    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:22:13.234581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:22:13.234553    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc"
Jul 29 15:25:12.784890 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:12.784852    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8497979899-44mhv"]
Jul 29 15:25:12.785508 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:12.785107    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8497979899-44mhv" podUID="950ff7b9-ba8a-442a-b2da-2f92f09ce235" containerName="manager" containerID="cri-o://d24da9e5e8fdab2c6c3240ce3594e67ba8b22891f28dbcb7ee532bb718c6fe80" gracePeriod=30
Jul 29 15:25:13.035190 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:13.035135    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8497979899-44mhv"
Jul 29 15:25:13.065781 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:13.065755    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/950ff7b9-ba8a-442a-b2da-2f92f09ce235-webhook-cert\") pod \"950ff7b9-ba8a-442a-b2da-2f92f09ce235\" (UID: \"950ff7b9-ba8a-442a-b2da-2f92f09ce235\") "
Jul 29 15:25:13.065928 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:13.065798    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsknl\" (UniqueName: \"kubernetes.io/projected/950ff7b9-ba8a-442a-b2da-2f92f09ce235-kube-api-access-rsknl\") pod \"950ff7b9-ba8a-442a-b2da-2f92f09ce235\" (UID: \"950ff7b9-ba8a-442a-b2da-2f92f09ce235\") "
Jul 29 15:25:13.067888 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:13.067851    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/950ff7b9-ba8a-442a-b2da-2f92f09ce235-kube-api-access-rsknl" (OuterVolumeSpecName: "kube-api-access-rsknl") pod "950ff7b9-ba8a-442a-b2da-2f92f09ce235" (UID: "950ff7b9-ba8a-442a-b2da-2f92f09ce235"). InnerVolumeSpecName "kube-api-access-rsknl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:25:13.067983 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:13.067919    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/950ff7b9-ba8a-442a-b2da-2f92f09ce235-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "950ff7b9-ba8a-442a-b2da-2f92f09ce235" (UID: "950ff7b9-ba8a-442a-b2da-2f92f09ce235"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:25:13.078852 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:13.078825    2575 generic.go:356] "Generic (PLEG): container finished" podID="950ff7b9-ba8a-442a-b2da-2f92f09ce235" containerID="d24da9e5e8fdab2c6c3240ce3594e67ba8b22891f28dbcb7ee532bb718c6fe80" exitCode=0
Jul 29 15:25:13.078976 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:13.078886    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8497979899-44mhv"
Jul 29 15:25:13.078976 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:13.078912    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8497979899-44mhv" event={"ID":"950ff7b9-ba8a-442a-b2da-2f92f09ce235","Type":"ContainerDied","Data":"d24da9e5e8fdab2c6c3240ce3594e67ba8b22891f28dbcb7ee532bb718c6fe80"}
Jul 29 15:25:13.079091 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:13.078974    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8497979899-44mhv" event={"ID":"950ff7b9-ba8a-442a-b2da-2f92f09ce235","Type":"ContainerDied","Data":"840cd46f72a66d4f8ef44e21f27da4ad45916a3642e445b5a78f26b610e437bc"}
Jul 29 15:25:13.079091 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:13.079008    2575 scope.go:117] "RemoveContainer" containerID="d24da9e5e8fdab2c6c3240ce3594e67ba8b22891f28dbcb7ee532bb718c6fe80"
Jul 29 15:25:13.089526 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:13.089506    2575 scope.go:117] "RemoveContainer" containerID="d24da9e5e8fdab2c6c3240ce3594e67ba8b22891f28dbcb7ee532bb718c6fe80"
Jul 29 15:25:13.089807 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:25:13.089789    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d24da9e5e8fdab2c6c3240ce3594e67ba8b22891f28dbcb7ee532bb718c6fe80\": container with ID starting with d24da9e5e8fdab2c6c3240ce3594e67ba8b22891f28dbcb7ee532bb718c6fe80 not found: ID does not exist" containerID="d24da9e5e8fdab2c6c3240ce3594e67ba8b22891f28dbcb7ee532bb718c6fe80"
Jul 29 15:25:13.089877 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:13.089814    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d24da9e5e8fdab2c6c3240ce3594e67ba8b22891f28dbcb7ee532bb718c6fe80"} err="failed to get container status \"d24da9e5e8fdab2c6c3240ce3594e67ba8b22891f28dbcb7ee532bb718c6fe80\": rpc error: code = NotFound desc = could not find container \"d24da9e5e8fdab2c6c3240ce3594e67ba8b22891f28dbcb7ee532bb718c6fe80\": container with ID starting with d24da9e5e8fdab2c6c3240ce3594e67ba8b22891f28dbcb7ee532bb718c6fe80 not found: ID does not exist"
Jul 29 15:25:13.102804 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:13.102781    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8497979899-44mhv"]
Jul 29 15:25:13.107962 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:13.107939    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8497979899-44mhv"]
Jul 29 15:25:13.166848 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:13.166818    2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/950ff7b9-ba8a-442a-b2da-2f92f09ce235-webhook-cert\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:25:13.166848 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:13.166845    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsknl\" (UniqueName: \"kubernetes.io/projected/950ff7b9-ba8a-442a-b2da-2f92f09ce235-kube-api-access-rsknl\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:25:13.299796 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:25:13.299703    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="950ff7b9-ba8a-442a-b2da-2f92f09ce235" path="/var/lib/kubelet/pods/950ff7b9-ba8a-442a-b2da-2f92f09ce235/volumes"
Jul 29 15:26:07.254817 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:07.254732    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:26:07.255613 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:07.255579    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:26:07.262111 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:07.262089    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:26:07.262784 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:07.262766    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:26:13.994480 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:13.994441    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8497979899-9n4d7"]
Jul 29 15:26:13.994877 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:13.994812    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="950ff7b9-ba8a-442a-b2da-2f92f09ce235" containerName="manager"
Jul 29 15:26:13.994877 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:13.994823    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="950ff7b9-ba8a-442a-b2da-2f92f09ce235" containerName="manager"
Jul 29 15:26:13.994955 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:13.994909    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="950ff7b9-ba8a-442a-b2da-2f92f09ce235" containerName="manager"
Jul 29 15:26:13.999110 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:13.999094    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8497979899-9n4d7"
Jul 29 15:26:14.002310 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:14.002289    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 15:26:14.002450 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:14.002291    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-96jgf\""
Jul 29 15:26:14.002450 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:14.002349    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 15:26:14.009799 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:14.009776    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8497979899-9n4d7"]
Jul 29 15:26:14.093660 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:14.093614    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9c6b31a-cc33-472d-ba6c-8433eddcb8e5-webhook-cert\") pod \"maas-controller-8497979899-9n4d7\" (UID: \"b9c6b31a-cc33-472d-ba6c-8433eddcb8e5\") " pod="opendatahub/maas-controller-8497979899-9n4d7"
Jul 29 15:26:14.093861 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:14.093700    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jvpk\" (UniqueName: \"kubernetes.io/projected/b9c6b31a-cc33-472d-ba6c-8433eddcb8e5-kube-api-access-6jvpk\") pod \"maas-controller-8497979899-9n4d7\" (UID: \"b9c6b31a-cc33-472d-ba6c-8433eddcb8e5\") " pod="opendatahub/maas-controller-8497979899-9n4d7"
Jul 29 15:26:14.194585 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:14.194540    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9c6b31a-cc33-472d-ba6c-8433eddcb8e5-webhook-cert\") pod \"maas-controller-8497979899-9n4d7\" (UID: \"b9c6b31a-cc33-472d-ba6c-8433eddcb8e5\") " pod="opendatahub/maas-controller-8497979899-9n4d7"
Jul 29 15:26:14.194795 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:14.194608    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jvpk\" (UniqueName: \"kubernetes.io/projected/b9c6b31a-cc33-472d-ba6c-8433eddcb8e5-kube-api-access-6jvpk\") pod \"maas-controller-8497979899-9n4d7\" (UID: \"b9c6b31a-cc33-472d-ba6c-8433eddcb8e5\") " pod="opendatahub/maas-controller-8497979899-9n4d7"
Jul 29 15:26:14.197020 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:14.196986    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9c6b31a-cc33-472d-ba6c-8433eddcb8e5-webhook-cert\") pod \"maas-controller-8497979899-9n4d7\" (UID: \"b9c6b31a-cc33-472d-ba6c-8433eddcb8e5\") " pod="opendatahub/maas-controller-8497979899-9n4d7"
Jul 29 15:26:14.205000 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:14.204977    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jvpk\" (UniqueName: \"kubernetes.io/projected/b9c6b31a-cc33-472d-ba6c-8433eddcb8e5-kube-api-access-6jvpk\") pod \"maas-controller-8497979899-9n4d7\" (UID: \"b9c6b31a-cc33-472d-ba6c-8433eddcb8e5\") " pod="opendatahub/maas-controller-8497979899-9n4d7"
Jul 29 15:26:14.310003 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:14.309908    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8497979899-9n4d7"
Jul 29 15:26:14.449989 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:14.449967    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8497979899-9n4d7"]
Jul 29 15:26:14.452822 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:26:14.452790    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9c6b31a_cc33_472d_ba6c_8433eddcb8e5.slice/crio-04823fba708a3de37fda28f50171193d847bfd5d5c76736884f53ecc09adb45a WatchSource:0}: Error finding container 04823fba708a3de37fda28f50171193d847bfd5d5c76736884f53ecc09adb45a: Status 404 returned error can't find the container with id 04823fba708a3de37fda28f50171193d847bfd5d5c76736884f53ecc09adb45a
Jul 29 15:26:15.372816 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:15.372785    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8497979899-9n4d7" event={"ID":"b9c6b31a-cc33-472d-ba6c-8433eddcb8e5","Type":"ContainerStarted","Data":"3c9b0ecd4f78d6cea2db38e3e400fdb7b7fedba8733aede23a3bda5ecb4871be"}
Jul 29 15:26:15.373188 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:15.372830    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8497979899-9n4d7" event={"ID":"b9c6b31a-cc33-472d-ba6c-8433eddcb8e5","Type":"ContainerStarted","Data":"04823fba708a3de37fda28f50171193d847bfd5d5c76736884f53ecc09adb45a"}
Jul 29 15:26:15.373188 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:15.372921    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8497979899-9n4d7"
Jul 29 15:26:15.404060 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:15.404009    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8497979899-9n4d7" podStartSLOduration=1.684348461 podStartE2EDuration="2.403992443s" totalImagesPullingTime="719.643982ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:26:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:26:15.40334928 +0000 UTC m=+908.552643178" watchObservedRunningTime="2026-07-29 15:26:15.403992443 +0000 UTC m=+908.553286344"
Jul 29 15:26:26.380370 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:26.380340    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8497979899-9n4d7"
Jul 29 15:26:31.104661 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.104628    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8497979899-9n4d7"]
Jul 29 15:26:31.105088 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.104829    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8497979899-9n4d7" podUID="b9c6b31a-cc33-472d-ba6c-8433eddcb8e5" containerName="manager" containerID="cri-o://3c9b0ecd4f78d6cea2db38e3e400fdb7b7fedba8733aede23a3bda5ecb4871be" gracePeriod=30
Jul 29 15:26:31.345605 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.345580    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8497979899-9n4d7"
Jul 29 15:26:31.445423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.445393    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9c6b31a-cc33-472d-ba6c-8433eddcb8e5-webhook-cert\") pod \"b9c6b31a-cc33-472d-ba6c-8433eddcb8e5\" (UID: \"b9c6b31a-cc33-472d-ba6c-8433eddcb8e5\") "
Jul 29 15:26:31.445615 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.445435    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jvpk\" (UniqueName: \"kubernetes.io/projected/b9c6b31a-cc33-472d-ba6c-8433eddcb8e5-kube-api-access-6jvpk\") pod \"b9c6b31a-cc33-472d-ba6c-8433eddcb8e5\" (UID: \"b9c6b31a-cc33-472d-ba6c-8433eddcb8e5\") "
Jul 29 15:26:31.447549 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.447517    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9c6b31a-cc33-472d-ba6c-8433eddcb8e5-kube-api-access-6jvpk" (OuterVolumeSpecName: "kube-api-access-6jvpk") pod "b9c6b31a-cc33-472d-ba6c-8433eddcb8e5" (UID: "b9c6b31a-cc33-472d-ba6c-8433eddcb8e5"). InnerVolumeSpecName "kube-api-access-6jvpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:26:31.447717 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.447698    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9c6b31a-cc33-472d-ba6c-8433eddcb8e5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b9c6b31a-cc33-472d-ba6c-8433eddcb8e5" (UID: "b9c6b31a-cc33-472d-ba6c-8433eddcb8e5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:26:31.450945 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.450919    2575 generic.go:356] "Generic (PLEG): container finished" podID="b9c6b31a-cc33-472d-ba6c-8433eddcb8e5" containerID="3c9b0ecd4f78d6cea2db38e3e400fdb7b7fedba8733aede23a3bda5ecb4871be" exitCode=0
Jul 29 15:26:31.451075 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.450967    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8497979899-9n4d7"
Jul 29 15:26:31.451075 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.451010    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8497979899-9n4d7" event={"ID":"b9c6b31a-cc33-472d-ba6c-8433eddcb8e5","Type":"ContainerDied","Data":"3c9b0ecd4f78d6cea2db38e3e400fdb7b7fedba8733aede23a3bda5ecb4871be"}
Jul 29 15:26:31.451190 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.451087    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8497979899-9n4d7" event={"ID":"b9c6b31a-cc33-472d-ba6c-8433eddcb8e5","Type":"ContainerDied","Data":"04823fba708a3de37fda28f50171193d847bfd5d5c76736884f53ecc09adb45a"}
Jul 29 15:26:31.451190 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.451124    2575 scope.go:117] "RemoveContainer" containerID="3c9b0ecd4f78d6cea2db38e3e400fdb7b7fedba8733aede23a3bda5ecb4871be"
Jul 29 15:26:31.466367 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.466344    2575 scope.go:117] "RemoveContainer" containerID="3c9b0ecd4f78d6cea2db38e3e400fdb7b7fedba8733aede23a3bda5ecb4871be"
Jul 29 15:26:31.466658 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:26:31.466638    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c9b0ecd4f78d6cea2db38e3e400fdb7b7fedba8733aede23a3bda5ecb4871be\": container with ID starting with 3c9b0ecd4f78d6cea2db38e3e400fdb7b7fedba8733aede23a3bda5ecb4871be not found: ID does not exist" containerID="3c9b0ecd4f78d6cea2db38e3e400fdb7b7fedba8733aede23a3bda5ecb4871be"
Jul 29 15:26:31.466735 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.466666    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c9b0ecd4f78d6cea2db38e3e400fdb7b7fedba8733aede23a3bda5ecb4871be"} err="failed to get container status \"3c9b0ecd4f78d6cea2db38e3e400fdb7b7fedba8733aede23a3bda5ecb4871be\": rpc error: code = NotFound desc = could not find container \"3c9b0ecd4f78d6cea2db38e3e400fdb7b7fedba8733aede23a3bda5ecb4871be\": container with ID starting with 3c9b0ecd4f78d6cea2db38e3e400fdb7b7fedba8733aede23a3bda5ecb4871be not found: ID does not exist"
Jul 29 15:26:31.477873 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.477853    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8497979899-9n4d7"]
Jul 29 15:26:31.482186 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.482165    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8497979899-9n4d7"]
Jul 29 15:26:31.546575 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.546553    2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9c6b31a-cc33-472d-ba6c-8433eddcb8e5-webhook-cert\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:26:31.546575 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:31.546574    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jvpk\" (UniqueName: \"kubernetes.io/projected/b9c6b31a-cc33-472d-ba6c-8433eddcb8e5-kube-api-access-6jvpk\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:26:32.351161 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:32.351127    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8497979899-wgmwr"]
Jul 29 15:26:32.351569 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:32.351536    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9c6b31a-cc33-472d-ba6c-8433eddcb8e5" containerName="manager"
Jul 29 15:26:32.351569 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:32.351549    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9c6b31a-cc33-472d-ba6c-8433eddcb8e5" containerName="manager"
Jul 29 15:26:32.351645 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:32.351612    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9c6b31a-cc33-472d-ba6c-8433eddcb8e5" containerName="manager"
Jul 29 15:26:32.356127 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:32.356105    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8497979899-wgmwr"
Jul 29 15:26:32.358361 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:32.358338    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 29 15:26:32.358975 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:32.358960    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 29 15:26:32.359074 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:32.358980    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-96jgf\""
Jul 29 15:26:32.364732 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:32.364707    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8497979899-wgmwr"]
Jul 29 15:26:32.454885 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:32.454858    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdk5z\" (UniqueName: \"kubernetes.io/projected/19b9c0bc-a3b2-4694-afad-3e5c35d074d8-kube-api-access-gdk5z\") pod \"maas-controller-8497979899-wgmwr\" (UID: \"19b9c0bc-a3b2-4694-afad-3e5c35d074d8\") " pod="opendatahub/maas-controller-8497979899-wgmwr"
Jul 29 15:26:32.455044 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:32.454906    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/19b9c0bc-a3b2-4694-afad-3e5c35d074d8-webhook-cert\") pod \"maas-controller-8497979899-wgmwr\" (UID: \"19b9c0bc-a3b2-4694-afad-3e5c35d074d8\") " pod="opendatahub/maas-controller-8497979899-wgmwr"
Jul 29 15:26:32.556162 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:32.556133    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdk5z\" (UniqueName: \"kubernetes.io/projected/19b9c0bc-a3b2-4694-afad-3e5c35d074d8-kube-api-access-gdk5z\") pod \"maas-controller-8497979899-wgmwr\" (UID: \"19b9c0bc-a3b2-4694-afad-3e5c35d074d8\") " pod="opendatahub/maas-controller-8497979899-wgmwr"
Jul 29 15:26:32.556307 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:32.556176    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/19b9c0bc-a3b2-4694-afad-3e5c35d074d8-webhook-cert\") pod \"maas-controller-8497979899-wgmwr\" (UID: \"19b9c0bc-a3b2-4694-afad-3e5c35d074d8\") " pod="opendatahub/maas-controller-8497979899-wgmwr"
Jul 29 15:26:32.558530 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:32.558503    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/19b9c0bc-a3b2-4694-afad-3e5c35d074d8-webhook-cert\") pod \"maas-controller-8497979899-wgmwr\" (UID: \"19b9c0bc-a3b2-4694-afad-3e5c35d074d8\") " pod="opendatahub/maas-controller-8497979899-wgmwr"
Jul 29 15:26:32.565603 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:32.565586    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdk5z\" (UniqueName: \"kubernetes.io/projected/19b9c0bc-a3b2-4694-afad-3e5c35d074d8-kube-api-access-gdk5z\") pod \"maas-controller-8497979899-wgmwr\" (UID: \"19b9c0bc-a3b2-4694-afad-3e5c35d074d8\") " pod="opendatahub/maas-controller-8497979899-wgmwr"
Jul 29 15:26:32.667000 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:32.666971    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8497979899-wgmwr"
Jul 29 15:26:33.006774 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:33.006665    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8497979899-wgmwr"]
Jul 29 15:26:33.009310 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:26:33.009281    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19b9c0bc_a3b2_4694_afad_3e5c35d074d8.slice/crio-972c2fb223a554869b9be6ed0665b3797536003edfcece071d47c201714c842c WatchSource:0}: Error finding container 972c2fb223a554869b9be6ed0665b3797536003edfcece071d47c201714c842c: Status 404 returned error can't find the container with id 972c2fb223a554869b9be6ed0665b3797536003edfcece071d47c201714c842c
Jul 29 15:26:33.010795 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:33.010773    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 15:26:33.295544 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:33.295472    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9c6b31a-cc33-472d-ba6c-8433eddcb8e5" path="/var/lib/kubelet/pods/b9c6b31a-cc33-472d-ba6c-8433eddcb8e5/volumes"
Jul 29 15:26:33.463096 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:33.463068    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8497979899-wgmwr" event={"ID":"19b9c0bc-a3b2-4694-afad-3e5c35d074d8","Type":"ContainerStarted","Data":"e189c109141760d1f869b4fdc580ba35d33f266800f046527b3f7d05d40be343"}
Jul 29 15:26:33.463475 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:33.463116    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8497979899-wgmwr" event={"ID":"19b9c0bc-a3b2-4694-afad-3e5c35d074d8","Type":"ContainerStarted","Data":"972c2fb223a554869b9be6ed0665b3797536003edfcece071d47c201714c842c"}
Jul 29 15:26:33.463475 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:33.463167    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8497979899-wgmwr"
Jul 29 15:26:33.481415 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:33.481369    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8497979899-wgmwr" podStartSLOduration=1.109427707 podStartE2EDuration="1.48135391s" totalImagesPullingTime="371.926203ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:26:33.479353132 +0000 UTC m=+926.628647025" watchObservedRunningTime="2026-07-29 15:26:33.48135391 +0000 UTC m=+926.630647809"
Jul 29 15:26:44.470589 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:26:44.470557    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8497979899-wgmwr"
Jul 29 15:31:07.285701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:31:07.285674    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:31:07.286223 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:31:07.286160    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:31:07.293051 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:31:07.293031    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:31:07.293180 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:31:07.293031    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:33:49.627447 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:49.627363    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"]
Jul 29 15:33:49.629972 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:49.627633    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j" podUID="75af11f4-880a-4aff-85d2-bb8ec5cf1da4" containerName="manager" containerID="cri-o://d0dd74eb3b55f585bdb1649b8cbc9ddfe433cc0df5987aeaa03e13eeee1f601e" gracePeriod=10
Jul 29 15:33:49.869714 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:49.869689    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"
Jul 29 15:33:49.955312 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:49.955223    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pjm8\" (UniqueName: \"kubernetes.io/projected/75af11f4-880a-4aff-85d2-bb8ec5cf1da4-kube-api-access-9pjm8\") pod \"75af11f4-880a-4aff-85d2-bb8ec5cf1da4\" (UID: \"75af11f4-880a-4aff-85d2-bb8ec5cf1da4\") "
Jul 29 15:33:49.955312 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:49.955287    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/75af11f4-880a-4aff-85d2-bb8ec5cf1da4-extensions-socket-volume\") pod \"75af11f4-880a-4aff-85d2-bb8ec5cf1da4\" (UID: \"75af11f4-880a-4aff-85d2-bb8ec5cf1da4\") "
Jul 29 15:33:49.955712 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:49.955686    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75af11f4-880a-4aff-85d2-bb8ec5cf1da4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "75af11f4-880a-4aff-85d2-bb8ec5cf1da4" (UID: "75af11f4-880a-4aff-85d2-bb8ec5cf1da4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:33:49.957204 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:49.957184    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75af11f4-880a-4aff-85d2-bb8ec5cf1da4-kube-api-access-9pjm8" (OuterVolumeSpecName: "kube-api-access-9pjm8") pod "75af11f4-880a-4aff-85d2-bb8ec5cf1da4" (UID: "75af11f4-880a-4aff-85d2-bb8ec5cf1da4"). InnerVolumeSpecName "kube-api-access-9pjm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:33:50.056038 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:50.056003    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/75af11f4-880a-4aff-85d2-bb8ec5cf1da4-extensions-socket-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:33:50.056038 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:50.056031    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pjm8\" (UniqueName: \"kubernetes.io/projected/75af11f4-880a-4aff-85d2-bb8ec5cf1da4-kube-api-access-9pjm8\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:33:50.551770 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:50.551733    2575 generic.go:356] "Generic (PLEG): container finished" podID="75af11f4-880a-4aff-85d2-bb8ec5cf1da4" containerID="d0dd74eb3b55f585bdb1649b8cbc9ddfe433cc0df5987aeaa03e13eeee1f601e" exitCode=0
Jul 29 15:33:50.551977 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:50.551790    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"
Jul 29 15:33:50.551977 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:50.551817    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j" event={"ID":"75af11f4-880a-4aff-85d2-bb8ec5cf1da4","Type":"ContainerDied","Data":"d0dd74eb3b55f585bdb1649b8cbc9ddfe433cc0df5987aeaa03e13eeee1f601e"}
Jul 29 15:33:50.551977 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:50.551876    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j" event={"ID":"75af11f4-880a-4aff-85d2-bb8ec5cf1da4","Type":"ContainerDied","Data":"6ae5ea2012c981c385ba101bfbeff67733cddeaf32a13dd89559d10cf7607071"}
Jul 29 15:33:50.551977 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:50.551904    2575 scope.go:117] "RemoveContainer" containerID="d0dd74eb3b55f585bdb1649b8cbc9ddfe433cc0df5987aeaa03e13eeee1f601e"
Jul 29 15:33:50.562152 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:50.562135    2575 scope.go:117] "RemoveContainer" containerID="d0dd74eb3b55f585bdb1649b8cbc9ddfe433cc0df5987aeaa03e13eeee1f601e"
Jul 29 15:33:50.562478 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:33:50.562455    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0dd74eb3b55f585bdb1649b8cbc9ddfe433cc0df5987aeaa03e13eeee1f601e\": container with ID starting with d0dd74eb3b55f585bdb1649b8cbc9ddfe433cc0df5987aeaa03e13eeee1f601e not found: ID does not exist" containerID="d0dd74eb3b55f585bdb1649b8cbc9ddfe433cc0df5987aeaa03e13eeee1f601e"
Jul 29 15:33:50.562551 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:50.562490    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0dd74eb3b55f585bdb1649b8cbc9ddfe433cc0df5987aeaa03e13eeee1f601e"} err="failed to get container status \"d0dd74eb3b55f585bdb1649b8cbc9ddfe433cc0df5987aeaa03e13eeee1f601e\": rpc error: code = NotFound desc = could not find container \"d0dd74eb3b55f585bdb1649b8cbc9ddfe433cc0df5987aeaa03e13eeee1f601e\": container with ID starting with d0dd74eb3b55f585bdb1649b8cbc9ddfe433cc0df5987aeaa03e13eeee1f601e not found: ID does not exist"
Jul 29 15:33:50.578376 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:50.578353    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"]
Jul 29 15:33:50.585417 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:50.585393    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4b7j"]
Jul 29 15:33:51.295299 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:33:51.295267    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75af11f4-880a-4aff-85d2-bb8ec5cf1da4" path="/var/lib/kubelet/pods/75af11f4-880a-4aff-85d2-bb8ec5cf1da4/volumes"
Jul 29 15:34:50.444191 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:50.444158    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"]
Jul 29 15:34:50.444669 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:50.444532    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75af11f4-880a-4aff-85d2-bb8ec5cf1da4" containerName="manager"
Jul 29 15:34:50.444669 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:50.444543    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="75af11f4-880a-4aff-85d2-bb8ec5cf1da4" containerName="manager"
Jul 29 15:34:50.444669 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:50.444605    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="75af11f4-880a-4aff-85d2-bb8ec5cf1da4" containerName="manager"
Jul 29 15:34:50.447541 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:50.447524    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"
Jul 29 15:34:50.450639 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:50.450624    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rhvk2\""
Jul 29 15:34:50.465508 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:50.465486    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"]
Jul 29 15:34:50.570803 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:50.570774    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04fcbf32-55b8-4dee-9078-8d29e6119ad7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fchzp\" (UID: \"04fcbf32-55b8-4dee-9078-8d29e6119ad7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"
Jul 29 15:34:50.570966 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:50.570867    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzsrz\" (UniqueName: \"kubernetes.io/projected/04fcbf32-55b8-4dee-9078-8d29e6119ad7-kube-api-access-wzsrz\") pod \"kuadrant-operator-controller-manager-86878c769f-fchzp\" (UID: \"04fcbf32-55b8-4dee-9078-8d29e6119ad7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"
Jul 29 15:34:50.671942 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:50.671910    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04fcbf32-55b8-4dee-9078-8d29e6119ad7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fchzp\" (UID: \"04fcbf32-55b8-4dee-9078-8d29e6119ad7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"
Jul 29 15:34:50.672107 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:50.671976    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzsrz\" (UniqueName: \"kubernetes.io/projected/04fcbf32-55b8-4dee-9078-8d29e6119ad7-kube-api-access-wzsrz\") pod \"kuadrant-operator-controller-manager-86878c769f-fchzp\" (UID: \"04fcbf32-55b8-4dee-9078-8d29e6119ad7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"
Jul 29 15:34:50.672265 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:50.672246    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04fcbf32-55b8-4dee-9078-8d29e6119ad7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fchzp\" (UID: \"04fcbf32-55b8-4dee-9078-8d29e6119ad7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"
Jul 29 15:34:50.684560 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:50.684537    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzsrz\" (UniqueName: \"kubernetes.io/projected/04fcbf32-55b8-4dee-9078-8d29e6119ad7-kube-api-access-wzsrz\") pod \"kuadrant-operator-controller-manager-86878c769f-fchzp\" (UID: \"04fcbf32-55b8-4dee-9078-8d29e6119ad7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"
Jul 29 15:34:50.756982 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:50.756915    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"
Jul 29 15:34:50.900541 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:50.900509    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"]
Jul 29 15:34:50.903686 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:34:50.903660    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04fcbf32_55b8_4dee_9078_8d29e6119ad7.slice/crio-24613d58298fb2a6263c98cf82bef52f0bdb8ec60a197a3330a95633ce3792da WatchSource:0}: Error finding container 24613d58298fb2a6263c98cf82bef52f0bdb8ec60a197a3330a95633ce3792da: Status 404 returned error can't find the container with id 24613d58298fb2a6263c98cf82bef52f0bdb8ec60a197a3330a95633ce3792da
Jul 29 15:34:51.832696 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:51.832659    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp" event={"ID":"04fcbf32-55b8-4dee-9078-8d29e6119ad7","Type":"ContainerStarted","Data":"7ce7294ba9d9579da040079bcbdfb32ff96d135bd75600411bf024d300f82840"}
Jul 29 15:34:51.833081 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:51.832716    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp" event={"ID":"04fcbf32-55b8-4dee-9078-8d29e6119ad7","Type":"ContainerStarted","Data":"24613d58298fb2a6263c98cf82bef52f0bdb8ec60a197a3330a95633ce3792da"}
Jul 29 15:34:51.833081 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:51.832748    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"
Jul 29 15:34:51.862719 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:34:51.862671    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp" podStartSLOduration=1.862655205 podStartE2EDuration="1.862655205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:34:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:34:51.859346946 +0000 UTC m=+1425.008640842" watchObservedRunningTime="2026-07-29 15:34:51.862655205 +0000 UTC m=+1425.011949103"
Jul 29 15:35:02.840050 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:35:02.840021    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"
Jul 29 15:36:07.316667 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:36:07.316640    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:36:07.319722 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:36:07.319700    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:36:07.323721 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:36:07.323703    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:36:07.326821 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:36:07.326804    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:41:07.350808 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:41:07.350777    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:41:07.354823 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:41:07.354800    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:41:07.357499 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:41:07.357479    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:41:07.361363 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:41:07.361346    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:43:53.076932 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.076898    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"]
Jul 29 15:43:53.080879 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.080853    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"
Jul 29 15:43:53.084854 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.084833    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-93e16c15-serving-cert\""
Jul 29 15:43:53.096585 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.096562    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"]
Jul 29 15:43:53.210701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.210661    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krwgq\" (UniqueName: \"kubernetes.io/projected/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-kube-api-access-krwgq\") pod \"maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d\" (UID: \"da9acb43-d54b-4d91-9018-2d7f5e6b22d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"
Jul 29 15:43:53.210881 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.210709    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-openshift-service-ca\") pod \"maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d\" (UID: \"da9acb43-d54b-4d91-9018-2d7f5e6b22d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"
Jul 29 15:43:53.210881 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.210787    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-maas-api-tls\") pod \"maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d\" (UID: \"da9acb43-d54b-4d91-9018-2d7f5e6b22d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"
Jul 29 15:43:53.312014 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.311979    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-maas-api-tls\") pod \"maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d\" (UID: \"da9acb43-d54b-4d91-9018-2d7f5e6b22d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"
Jul 29 15:43:53.312222 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.312068    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krwgq\" (UniqueName: \"kubernetes.io/projected/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-kube-api-access-krwgq\") pod \"maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d\" (UID: \"da9acb43-d54b-4d91-9018-2d7f5e6b22d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"
Jul 29 15:43:53.312222 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.312096    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-openshift-service-ca\") pod \"maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d\" (UID: \"da9acb43-d54b-4d91-9018-2d7f5e6b22d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"
Jul 29 15:43:53.312222 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:43:53.312149    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-serving-cert: secret "maas-api-e2e-ait-93e16c15-serving-cert" not found
Jul 29 15:43:53.312437 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:43:53.312238    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-maas-api-tls podName:da9acb43-d54b-4d91-9018-2d7f5e6b22d4 nodeName:}" failed. No retries permitted until 2026-07-29 15:43:53.812215481 +0000 UTC m=+1966.961509360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-maas-api-tls") pod "maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d" (UID: "da9acb43-d54b-4d91-9018-2d7f5e6b22d4") : secret "maas-api-e2e-ait-93e16c15-serving-cert" not found
Jul 29 15:43:53.312716 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.312700    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-openshift-service-ca\") pod \"maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d\" (UID: \"da9acb43-d54b-4d91-9018-2d7f5e6b22d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"
Jul 29 15:43:53.322766 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.322742    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krwgq\" (UniqueName: \"kubernetes.io/projected/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-kube-api-access-krwgq\") pod \"maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d\" (UID: \"da9acb43-d54b-4d91-9018-2d7f5e6b22d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"
Jul 29 15:43:53.815640 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.815608    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-maas-api-tls\") pod \"maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d\" (UID: \"da9acb43-d54b-4d91-9018-2d7f5e6b22d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"
Jul 29 15:43:53.817928 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.817906    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-maas-api-tls\") pod \"maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d\" (UID: \"da9acb43-d54b-4d91-9018-2d7f5e6b22d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"
Jul 29 15:43:53.846570 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.846543    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"]
Jul 29 15:43:53.850158 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.850142    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"
Jul 29 15:43:53.852266 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.852247    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-93e16c15\""
Jul 29 15:43:53.858788 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.858765    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"]
Jul 29 15:43:53.872603 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.872569    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"]
Jul 29 15:43:53.876673 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.876652    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"
Jul 29 15:43:53.879111 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.879094    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-93e16c15-dockercfg-cmm4b\""
Jul 29 15:43:53.886745 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.886721    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"]
Jul 29 15:43:53.916681 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.916657    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w47j\" (UniqueName: \"kubernetes.io/projected/ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc-kube-api-access-7w47j\") pod \"payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4\" (UID: \"ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc\") " pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"
Jul 29 15:43:53.916792 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.916688    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc-plugins-config-volume\") pod \"payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4\" (UID: \"ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc\") " pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"
Jul 29 15:43:53.916792 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.916753    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3b9f156-a83c-40b2-a3c7-0ea8786d3762-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9\" (UID: \"a3b9f156-a83c-40b2-a3c7-0ea8786d3762\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"
Jul 29 15:43:53.916873 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.916800    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjmd5\" (UniqueName: \"kubernetes.io/projected/a3b9f156-a83c-40b2-a3c7-0ea8786d3762-kube-api-access-pjmd5\") pod \"payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9\" (UID: \"a3b9f156-a83c-40b2-a3c7-0ea8786d3762\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"
Jul 29 15:43:53.994397 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:53.994363    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"
Jul 29 15:43:54.017812 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:54.017777    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w47j\" (UniqueName: \"kubernetes.io/projected/ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc-kube-api-access-7w47j\") pod \"payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4\" (UID: \"ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc\") " pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"
Jul 29 15:43:54.017951 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:54.017817    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc-plugins-config-volume\") pod \"payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4\" (UID: \"ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc\") " pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"
Jul 29 15:43:54.017951 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:54.017851    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3b9f156-a83c-40b2-a3c7-0ea8786d3762-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9\" (UID: \"a3b9f156-a83c-40b2-a3c7-0ea8786d3762\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"
Jul 29 15:43:54.017951 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:54.017887    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjmd5\" (UniqueName: \"kubernetes.io/projected/a3b9f156-a83c-40b2-a3c7-0ea8786d3762-kube-api-access-pjmd5\") pod \"payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9\" (UID: \"a3b9f156-a83c-40b2-a3c7-0ea8786d3762\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"
Jul 29 15:43:54.018575 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:54.018554    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc-plugins-config-volume\") pod \"payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4\" (UID: \"ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc\") " pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"
Jul 29 15:43:54.018868 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:54.018848    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3b9f156-a83c-40b2-a3c7-0ea8786d3762-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9\" (UID: \"a3b9f156-a83c-40b2-a3c7-0ea8786d3762\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"
Jul 29 15:43:54.030765 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:54.030739    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w47j\" (UniqueName: \"kubernetes.io/projected/ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc-kube-api-access-7w47j\") pod \"payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4\" (UID: \"ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc\") " pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"
Jul 29 15:43:54.031303 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:54.031287    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjmd5\" (UniqueName: \"kubernetes.io/projected/a3b9f156-a83c-40b2-a3c7-0ea8786d3762-kube-api-access-pjmd5\") pod \"payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9\" (UID: \"a3b9f156-a83c-40b2-a3c7-0ea8786d3762\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"
Jul 29 15:43:54.155307 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:54.155272    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"]
Jul 29 15:43:54.156145 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:43:54.156120    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda9acb43_d54b_4d91_9018_2d7f5e6b22d4.slice/crio-e8b05d4bf769dd4b03e53cee8592d541e16cf9ceddce981623a618d21eab5ae1 WatchSource:0}: Error finding container e8b05d4bf769dd4b03e53cee8592d541e16cf9ceddce981623a618d21eab5ae1: Status 404 returned error can't find the container with id e8b05d4bf769dd4b03e53cee8592d541e16cf9ceddce981623a618d21eab5ae1
Jul 29 15:43:54.157447 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:54.157427    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 15:43:54.162110 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:54.162086    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"
Jul 29 15:43:54.187403 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:54.187376    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"
Jul 29 15:43:54.326023 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:54.325794    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"]
Jul 29 15:43:54.329336 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:43:54.329294    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3b9f156_a83c_40b2_a3c7_0ea8786d3762.slice/crio-b352329ffb0f2728f38b676e084eb01cca5a0d11ebe63132551ac81776435ed5 WatchSource:0}: Error finding container b352329ffb0f2728f38b676e084eb01cca5a0d11ebe63132551ac81776435ed5: Status 404 returned error can't find the container with id b352329ffb0f2728f38b676e084eb01cca5a0d11ebe63132551ac81776435ed5
Jul 29 15:43:54.341545 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:54.341507    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9" event={"ID":"a3b9f156-a83c-40b2-a3c7-0ea8786d3762","Type":"ContainerStarted","Data":"b352329ffb0f2728f38b676e084eb01cca5a0d11ebe63132551ac81776435ed5"}
Jul 29 15:43:54.344489 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:54.344446    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d" event={"ID":"da9acb43-d54b-4d91-9018-2d7f5e6b22d4","Type":"ContainerStarted","Data":"e8b05d4bf769dd4b03e53cee8592d541e16cf9ceddce981623a618d21eab5ae1"}
Jul 29 15:43:54.362182 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:54.362155    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"]
Jul 29 15:43:54.374939 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:43:54.374751    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded8e4bc4_c70e_4681_bcc6_bd5c224ceccc.slice/crio-5fa00778ed9bb0eaf40b1ebc32d0ce050d98765ecf5f19f2e2e8174c79cb12be WatchSource:0}: Error finding container 5fa00778ed9bb0eaf40b1ebc32d0ce050d98765ecf5f19f2e2e8174c79cb12be: Status 404 returned error can't find the container with id 5fa00778ed9bb0eaf40b1ebc32d0ce050d98765ecf5f19f2e2e8174c79cb12be
Jul 29 15:43:55.351726 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:55.351635    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9" event={"ID":"a3b9f156-a83c-40b2-a3c7-0ea8786d3762","Type":"ContainerStarted","Data":"37f61f9da35e1ef3994c5496e5228ed974028ef25dd8f443b26087686ef0da5c"}
Jul 29 15:43:55.351726 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:55.351711    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"
Jul 29 15:43:55.353278 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:55.353252    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4" event={"ID":"ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc","Type":"ContainerStarted","Data":"b9e86b49dcdf6421d45c0d23d8d919d131b4aad319c4b3965f6dc2e3629f993c"}
Jul 29 15:43:55.353449 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:55.353287    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4" event={"ID":"ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc","Type":"ContainerStarted","Data":"5fa00778ed9bb0eaf40b1ebc32d0ce050d98765ecf5f19f2e2e8174c79cb12be"}
Jul 29 15:43:55.353449 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:55.353313    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"
Jul 29 15:43:55.354645 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:55.354624    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d" event={"ID":"da9acb43-d54b-4d91-9018-2d7f5e6b22d4","Type":"ContainerStarted","Data":"719f7d4ec1cc0e5542417f388737e8d57d074a4520c55da5707e3d0e2e7fdea1"}
Jul 29 15:43:55.354749 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:55.354677    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"
Jul 29 15:43:55.370142 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:55.370095    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9" podStartSLOduration=2.3700812239999998 podStartE2EDuration="2.370081224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:43:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:43:55.368436053 +0000 UTC m=+1968.517729966" watchObservedRunningTime="2026-07-29 15:43:55.370081224 +0000 UTC m=+1968.519375124"
Jul 29 15:43:55.384862 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:55.384805    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4" podStartSLOduration=2.384787216 podStartE2EDuration="2.384787216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:43:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:43:55.384245495 +0000 UTC m=+1968.533539394" watchObservedRunningTime="2026-07-29 15:43:55.384787216 +0000 UTC m=+1968.534081116"
Jul 29 15:43:55.402924 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:43:55.402786    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d" podStartSLOduration=1.494342579 podStartE2EDuration="2.402766514s" totalImagesPullingTime="908.423935ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:43:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:43:55.402299649 +0000 UTC m=+1968.551593546" watchObservedRunningTime="2026-07-29 15:43:55.402766514 +0000 UTC m=+1968.552060411"
Jul 29 15:44:01.366227 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:01.366198    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"
Jul 29 15:44:06.362611 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:06.362557    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9" podUID="a3b9f156-a83c-40b2-a3c7-0ea8786d3762" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.47:9004: connect: connection refused"
Jul 29 15:44:06.363085 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:06.362947    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"
Jul 29 15:44:16.362022 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:16.361982    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"
Jul 29 15:44:33.786197 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:33.786161    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"]
Jul 29 15:44:33.788786 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:33.786465    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d" podUID="da9acb43-d54b-4d91-9018-2d7f5e6b22d4" containerName="maas-api" containerID="cri-o://719f7d4ec1cc0e5542417f388737e8d57d074a4520c55da5707e3d0e2e7fdea1" gracePeriod=30
Jul 29 15:44:33.858790 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:33.858755    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"]
Jul 29 15:44:33.859090 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:33.859018    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4" podUID="ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc" containerName="payload-processing" containerID="cri-o://b9e86b49dcdf6421d45c0d23d8d919d131b4aad319c4b3965f6dc2e3629f993c" gracePeriod=30
Jul 29 15:44:33.867839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:33.867812    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"]
Jul 29 15:44:33.868160 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:33.868127    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9" podUID="a3b9f156-a83c-40b2-a3c7-0ea8786d3762" containerName="payload-pre-processing" containerID="cri-o://37f61f9da35e1ef3994c5496e5228ed974028ef25dd8f443b26087686ef0da5c" gracePeriod=30
Jul 29 15:44:34.147962 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.147925    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"
Jul 29 15:44:34.162195 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.162170    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"
Jul 29 15:44:34.175032 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.175008    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"
Jul 29 15:44:34.273957 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.273923    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3b9f156-a83c-40b2-a3c7-0ea8786d3762-plugins-config-volume\") pod \"a3b9f156-a83c-40b2-a3c7-0ea8786d3762\" (UID: \"a3b9f156-a83c-40b2-a3c7-0ea8786d3762\") "
Jul 29 15:44:34.274136 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.273961    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-maas-api-tls\") pod \"da9acb43-d54b-4d91-9018-2d7f5e6b22d4\" (UID: \"da9acb43-d54b-4d91-9018-2d7f5e6b22d4\") "
Jul 29 15:44:34.274136 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.273999    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjmd5\" (UniqueName: \"kubernetes.io/projected/a3b9f156-a83c-40b2-a3c7-0ea8786d3762-kube-api-access-pjmd5\") pod \"a3b9f156-a83c-40b2-a3c7-0ea8786d3762\" (UID: \"a3b9f156-a83c-40b2-a3c7-0ea8786d3762\") "
Jul 29 15:44:34.274136 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.274033    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krwgq\" (UniqueName: \"kubernetes.io/projected/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-kube-api-access-krwgq\") pod \"da9acb43-d54b-4d91-9018-2d7f5e6b22d4\" (UID: \"da9acb43-d54b-4d91-9018-2d7f5e6b22d4\") "
Jul 29 15:44:34.274136 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.274062    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w47j\" (UniqueName: \"kubernetes.io/projected/ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc-kube-api-access-7w47j\") pod \"ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc\" (UID: \"ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc\") "
Jul 29 15:44:34.274136 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.274091    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc-plugins-config-volume\") pod \"ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc\" (UID: \"ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc\") "
Jul 29 15:44:34.274136 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.274116    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-openshift-service-ca\") pod \"da9acb43-d54b-4d91-9018-2d7f5e6b22d4\" (UID: \"da9acb43-d54b-4d91-9018-2d7f5e6b22d4\") "
Jul 29 15:44:34.274507 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.274381    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3b9f156-a83c-40b2-a3c7-0ea8786d3762-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a3b9f156-a83c-40b2-a3c7-0ea8786d3762" (UID: "a3b9f156-a83c-40b2-a3c7-0ea8786d3762"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:44:34.274658 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.274632    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "da9acb43-d54b-4d91-9018-2d7f5e6b22d4" (UID: "da9acb43-d54b-4d91-9018-2d7f5e6b22d4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:44:34.274735 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.274703    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc" (UID: "ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:44:34.276431 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.276404    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-kube-api-access-krwgq" (OuterVolumeSpecName: "kube-api-access-krwgq") pod "da9acb43-d54b-4d91-9018-2d7f5e6b22d4" (UID: "da9acb43-d54b-4d91-9018-2d7f5e6b22d4"). InnerVolumeSpecName "kube-api-access-krwgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:44:34.276547 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.276492    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3b9f156-a83c-40b2-a3c7-0ea8786d3762-kube-api-access-pjmd5" (OuterVolumeSpecName: "kube-api-access-pjmd5") pod "a3b9f156-a83c-40b2-a3c7-0ea8786d3762" (UID: "a3b9f156-a83c-40b2-a3c7-0ea8786d3762"). InnerVolumeSpecName "kube-api-access-pjmd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:44:34.276547 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.276532    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc-kube-api-access-7w47j" (OuterVolumeSpecName: "kube-api-access-7w47j") pod "ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc" (UID: "ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc"). InnerVolumeSpecName "kube-api-access-7w47j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:44:34.276626 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.276536    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "da9acb43-d54b-4d91-9018-2d7f5e6b22d4" (UID: "da9acb43-d54b-4d91-9018-2d7f5e6b22d4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:44:34.374857 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.374830    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a3b9f156-a83c-40b2-a3c7-0ea8786d3762-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:44:34.374857 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.374855    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-maas-api-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:44:34.375031 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.374867    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjmd5\" (UniqueName: \"kubernetes.io/projected/a3b9f156-a83c-40b2-a3c7-0ea8786d3762-kube-api-access-pjmd5\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:44:34.375031 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.374875    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krwgq\" (UniqueName: \"kubernetes.io/projected/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-kube-api-access-krwgq\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:44:34.375031 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.374884    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w47j\" (UniqueName: \"kubernetes.io/projected/ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc-kube-api-access-7w47j\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:44:34.375031 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.374893    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:44:34.375031 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.374901    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da9acb43-d54b-4d91-9018-2d7f5e6b22d4-openshift-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:44:34.547034 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.546999    2575 generic.go:356] "Generic (PLEG): container finished" podID="a3b9f156-a83c-40b2-a3c7-0ea8786d3762" containerID="37f61f9da35e1ef3994c5496e5228ed974028ef25dd8f443b26087686ef0da5c" exitCode=0
Jul 29 15:44:34.547221 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.547043    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"
Jul 29 15:44:34.547221 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.547090    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9" event={"ID":"a3b9f156-a83c-40b2-a3c7-0ea8786d3762","Type":"ContainerDied","Data":"37f61f9da35e1ef3994c5496e5228ed974028ef25dd8f443b26087686ef0da5c"}
Jul 29 15:44:34.547221 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.547143    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9" event={"ID":"a3b9f156-a83c-40b2-a3c7-0ea8786d3762","Type":"ContainerDied","Data":"b352329ffb0f2728f38b676e084eb01cca5a0d11ebe63132551ac81776435ed5"}
Jul 29 15:44:34.547221 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.547181    2575 scope.go:117] "RemoveContainer" containerID="37f61f9da35e1ef3994c5496e5228ed974028ef25dd8f443b26087686ef0da5c"
Jul 29 15:44:34.548474 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.548455    2575 generic.go:356] "Generic (PLEG): container finished" podID="ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc" containerID="b9e86b49dcdf6421d45c0d23d8d919d131b4aad319c4b3965f6dc2e3629f993c" exitCode=0
Jul 29 15:44:34.548570 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.548493    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4" event={"ID":"ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc","Type":"ContainerDied","Data":"b9e86b49dcdf6421d45c0d23d8d919d131b4aad319c4b3965f6dc2e3629f993c"}
Jul 29 15:44:34.548570 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.548507    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"
Jul 29 15:44:34.548570 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.548540    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4" event={"ID":"ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc","Type":"ContainerDied","Data":"5fa00778ed9bb0eaf40b1ebc32d0ce050d98765ecf5f19f2e2e8174c79cb12be"}
Jul 29 15:44:34.550117 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.550092    2575 generic.go:356] "Generic (PLEG): container finished" podID="da9acb43-d54b-4d91-9018-2d7f5e6b22d4" containerID="719f7d4ec1cc0e5542417f388737e8d57d074a4520c55da5707e3d0e2e7fdea1" exitCode=0
Jul 29 15:44:34.550231 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.550142    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"
Jul 29 15:44:34.550231 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.550156    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d" event={"ID":"da9acb43-d54b-4d91-9018-2d7f5e6b22d4","Type":"ContainerDied","Data":"719f7d4ec1cc0e5542417f388737e8d57d074a4520c55da5707e3d0e2e7fdea1"}
Jul 29 15:44:34.550231 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.550197    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d" event={"ID":"da9acb43-d54b-4d91-9018-2d7f5e6b22d4","Type":"ContainerDied","Data":"e8b05d4bf769dd4b03e53cee8592d541e16cf9ceddce981623a618d21eab5ae1"}
Jul 29 15:44:34.558878 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.558721    2575 scope.go:117] "RemoveContainer" containerID="37f61f9da35e1ef3994c5496e5228ed974028ef25dd8f443b26087686ef0da5c"
Jul 29 15:44:34.559232 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:44:34.559202    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37f61f9da35e1ef3994c5496e5228ed974028ef25dd8f443b26087686ef0da5c\": container with ID starting with 37f61f9da35e1ef3994c5496e5228ed974028ef25dd8f443b26087686ef0da5c not found: ID does not exist" containerID="37f61f9da35e1ef3994c5496e5228ed974028ef25dd8f443b26087686ef0da5c"
Jul 29 15:44:34.559375 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.559241    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37f61f9da35e1ef3994c5496e5228ed974028ef25dd8f443b26087686ef0da5c"} err="failed to get container status \"37f61f9da35e1ef3994c5496e5228ed974028ef25dd8f443b26087686ef0da5c\": rpc error: code = NotFound desc = could not find container \"37f61f9da35e1ef3994c5496e5228ed974028ef25dd8f443b26087686ef0da5c\": container with ID starting with 37f61f9da35e1ef3994c5496e5228ed974028ef25dd8f443b26087686ef0da5c not found: ID does not exist"
Jul 29 15:44:34.559375 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.559259    2575 scope.go:117] "RemoveContainer" containerID="b9e86b49dcdf6421d45c0d23d8d919d131b4aad319c4b3965f6dc2e3629f993c"
Jul 29 15:44:34.567474 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.567460    2575 scope.go:117] "RemoveContainer" containerID="b9e86b49dcdf6421d45c0d23d8d919d131b4aad319c4b3965f6dc2e3629f993c"
Jul 29 15:44:34.567710 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:44:34.567688    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9e86b49dcdf6421d45c0d23d8d919d131b4aad319c4b3965f6dc2e3629f993c\": container with ID starting with b9e86b49dcdf6421d45c0d23d8d919d131b4aad319c4b3965f6dc2e3629f993c not found: ID does not exist" containerID="b9e86b49dcdf6421d45c0d23d8d919d131b4aad319c4b3965f6dc2e3629f993c"
Jul 29 15:44:34.567792 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.567714    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9e86b49dcdf6421d45c0d23d8d919d131b4aad319c4b3965f6dc2e3629f993c"} err="failed to get container status \"b9e86b49dcdf6421d45c0d23d8d919d131b4aad319c4b3965f6dc2e3629f993c\": rpc error: code = NotFound desc = could not find container \"b9e86b49dcdf6421d45c0d23d8d919d131b4aad319c4b3965f6dc2e3629f993c\": container with ID starting with b9e86b49dcdf6421d45c0d23d8d919d131b4aad319c4b3965f6dc2e3629f993c not found: ID does not exist"
Jul 29 15:44:34.567792 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.567729    2575 scope.go:117] "RemoveContainer" containerID="719f7d4ec1cc0e5542417f388737e8d57d074a4520c55da5707e3d0e2e7fdea1"
Jul 29 15:44:34.575294 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.575279    2575 scope.go:117] "RemoveContainer" containerID="719f7d4ec1cc0e5542417f388737e8d57d074a4520c55da5707e3d0e2e7fdea1"
Jul 29 15:44:34.575513 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.575494    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"]
Jul 29 15:44:34.575572 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:44:34.575544    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"719f7d4ec1cc0e5542417f388737e8d57d074a4520c55da5707e3d0e2e7fdea1\": container with ID starting with 719f7d4ec1cc0e5542417f388737e8d57d074a4520c55da5707e3d0e2e7fdea1 not found: ID does not exist" containerID="719f7d4ec1cc0e5542417f388737e8d57d074a4520c55da5707e3d0e2e7fdea1"
Jul 29 15:44:34.575616 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.575565    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"719f7d4ec1cc0e5542417f388737e8d57d074a4520c55da5707e3d0e2e7fdea1"} err="failed to get container status \"719f7d4ec1cc0e5542417f388737e8d57d074a4520c55da5707e3d0e2e7fdea1\": rpc error: code = NotFound desc = could not find container \"719f7d4ec1cc0e5542417f388737e8d57d074a4520c55da5707e3d0e2e7fdea1\": container with ID starting with 719f7d4ec1cc0e5542417f388737e8d57d074a4520c55da5707e3d0e2e7fdea1 not found: ID does not exist"
Jul 29 15:44:34.584487 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.584467    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-93e16c15-5f57db65b-7l6w9"]
Jul 29 15:44:34.599545 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.599522    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"]
Jul 29 15:44:34.604205 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.604187    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-93e16c15-7fd775d666-kzck4"]
Jul 29 15:44:34.620882 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.620860    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"]
Jul 29 15:44:34.626881 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:34.626833    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-93e16c15-7b944dd466-4nl6d"]
Jul 29 15:44:35.295384 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:35.295347    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3b9f156-a83c-40b2-a3c7-0ea8786d3762" path="/var/lib/kubelet/pods/a3b9f156-a83c-40b2-a3c7-0ea8786d3762/volumes"
Jul 29 15:44:35.295910 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:35.295890    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da9acb43-d54b-4d91-9018-2d7f5e6b22d4" path="/var/lib/kubelet/pods/da9acb43-d54b-4d91-9018-2d7f5e6b22d4/volumes"
Jul 29 15:44:35.296270 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:35.296251    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc" path="/var/lib/kubelet/pods/ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc/volumes"
Jul 29 15:44:38.285455 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.285419    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"]
Jul 29 15:44:38.285995 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.285975    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc" containerName="payload-processing"
Jul 29 15:44:38.286075 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.285998    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc" containerName="payload-processing"
Jul 29 15:44:38.286075 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.286022    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3b9f156-a83c-40b2-a3c7-0ea8786d3762" containerName="payload-pre-processing"
Jul 29 15:44:38.286075 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.286031    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3b9f156-a83c-40b2-a3c7-0ea8786d3762" containerName="payload-pre-processing"
Jul 29 15:44:38.286075 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.286042    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da9acb43-d54b-4d91-9018-2d7f5e6b22d4" containerName="maas-api"
Jul 29 15:44:38.286075 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.286051    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="da9acb43-d54b-4d91-9018-2d7f5e6b22d4" containerName="maas-api"
Jul 29 15:44:38.286360 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.286157    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed8e4bc4-c70e-4681-bcc6-bd5c224ceccc" containerName="payload-processing"
Jul 29 15:44:38.286360 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.286175    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="da9acb43-d54b-4d91-9018-2d7f5e6b22d4" containerName="maas-api"
Jul 29 15:44:38.286360 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.286188    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3b9f156-a83c-40b2-a3c7-0ea8786d3762" containerName="payload-pre-processing"
Jul 29 15:44:38.291386 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.291314    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"
Jul 29 15:44:38.293604 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.293578    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-adc0e91c-serving-cert\""
Jul 29 15:44:38.301004 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.300978    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"]
Jul 29 15:44:38.410940 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.410898    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/611b4648-c680-4ccf-99e1-c192895ccb73-maas-api-tls\") pod \"maas-api-e2e-migrate-adc0e91c-647795c996-k29bp\" (UID: \"611b4648-c680-4ccf-99e1-c192895ccb73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"
Jul 29 15:44:38.411135 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.411011    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/611b4648-c680-4ccf-99e1-c192895ccb73-openshift-service-ca\") pod \"maas-api-e2e-migrate-adc0e91c-647795c996-k29bp\" (UID: \"611b4648-c680-4ccf-99e1-c192895ccb73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"
Jul 29 15:44:38.411135 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.411044    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gr8t\" (UniqueName: \"kubernetes.io/projected/611b4648-c680-4ccf-99e1-c192895ccb73-kube-api-access-6gr8t\") pod \"maas-api-e2e-migrate-adc0e91c-647795c996-k29bp\" (UID: \"611b4648-c680-4ccf-99e1-c192895ccb73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"
Jul 29 15:44:38.512400 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.512362    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gr8t\" (UniqueName: \"kubernetes.io/projected/611b4648-c680-4ccf-99e1-c192895ccb73-kube-api-access-6gr8t\") pod \"maas-api-e2e-migrate-adc0e91c-647795c996-k29bp\" (UID: \"611b4648-c680-4ccf-99e1-c192895ccb73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"
Jul 29 15:44:38.512583 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.512500    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/611b4648-c680-4ccf-99e1-c192895ccb73-maas-api-tls\") pod \"maas-api-e2e-migrate-adc0e91c-647795c996-k29bp\" (UID: \"611b4648-c680-4ccf-99e1-c192895ccb73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"
Jul 29 15:44:38.512625 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.512586    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/611b4648-c680-4ccf-99e1-c192895ccb73-openshift-service-ca\") pod \"maas-api-e2e-migrate-adc0e91c-647795c996-k29bp\" (UID: \"611b4648-c680-4ccf-99e1-c192895ccb73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"
Jul 29 15:44:38.513205 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.513185    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/611b4648-c680-4ccf-99e1-c192895ccb73-openshift-service-ca\") pod \"maas-api-e2e-migrate-adc0e91c-647795c996-k29bp\" (UID: \"611b4648-c680-4ccf-99e1-c192895ccb73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"
Jul 29 15:44:38.514950 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.514926    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/611b4648-c680-4ccf-99e1-c192895ccb73-maas-api-tls\") pod \"maas-api-e2e-migrate-adc0e91c-647795c996-k29bp\" (UID: \"611b4648-c680-4ccf-99e1-c192895ccb73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"
Jul 29 15:44:38.521215 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.521188    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gr8t\" (UniqueName: \"kubernetes.io/projected/611b4648-c680-4ccf-99e1-c192895ccb73-kube-api-access-6gr8t\") pod \"maas-api-e2e-migrate-adc0e91c-647795c996-k29bp\" (UID: \"611b4648-c680-4ccf-99e1-c192895ccb73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"
Jul 29 15:44:38.603048 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.602971    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"
Jul 29 15:44:38.761888 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:38.761862    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"]
Jul 29 15:44:38.764821 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:44:38.764793    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod611b4648_c680_4ccf_99e1_c192895ccb73.slice/crio-a83edea5b8d23a163f89320ecfa6f63cac58918d8acf244eca6e2de0509e3700 WatchSource:0}: Error finding container a83edea5b8d23a163f89320ecfa6f63cac58918d8acf244eca6e2de0509e3700: Status 404 returned error can't find the container with id a83edea5b8d23a163f89320ecfa6f63cac58918d8acf244eca6e2de0509e3700
Jul 29 15:44:39.105041 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.105012    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"]
Jul 29 15:44:39.109334 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.109308    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"
Jul 29 15:44:39.111597 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.111578    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-adc0e91c\""
Jul 29 15:44:39.111597 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.111586    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-adc0e91c-dockercfg-l2475\""
Jul 29 15:44:39.119969 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.119936    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"]
Jul 29 15:44:39.152717 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.152691    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"]
Jul 29 15:44:39.156161 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.156141    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"
Jul 29 15:44:39.167173 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.167141    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"]
Jul 29 15:44:39.218359 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.218308    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd4vf\" (UniqueName: \"kubernetes.io/projected/b8877212-eee4-4c1e-9817-371a6b99d375-kube-api-access-fd4vf\") pod \"payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4\" (UID: \"b8877212-eee4-4c1e-9817-371a6b99d375\") " pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"
Jul 29 15:44:39.218509 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.218431    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8877212-eee4-4c1e-9817-371a6b99d375-plugins-config-volume\") pod \"payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4\" (UID: \"b8877212-eee4-4c1e-9817-371a6b99d375\") " pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"
Jul 29 15:44:39.319645 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.319611    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68vq5\" (UniqueName: \"kubernetes.io/projected/4d9a5caf-3f97-43d7-ab47-7845876fbcff-kube-api-access-68vq5\") pod \"payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2\" (UID: \"4d9a5caf-3f97-43d7-ab47-7845876fbcff\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"
Jul 29 15:44:39.320119 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.319688    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d9a5caf-3f97-43d7-ab47-7845876fbcff-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2\" (UID: \"4d9a5caf-3f97-43d7-ab47-7845876fbcff\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"
Jul 29 15:44:39.320119 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.319734    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8877212-eee4-4c1e-9817-371a6b99d375-plugins-config-volume\") pod \"payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4\" (UID: \"b8877212-eee4-4c1e-9817-371a6b99d375\") " pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"
Jul 29 15:44:39.320119 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.319804    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd4vf\" (UniqueName: \"kubernetes.io/projected/b8877212-eee4-4c1e-9817-371a6b99d375-kube-api-access-fd4vf\") pod \"payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4\" (UID: \"b8877212-eee4-4c1e-9817-371a6b99d375\") " pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"
Jul 29 15:44:39.320989 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.320962    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8877212-eee4-4c1e-9817-371a6b99d375-plugins-config-volume\") pod \"payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4\" (UID: \"b8877212-eee4-4c1e-9817-371a6b99d375\") " pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"
Jul 29 15:44:39.328929 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.328905    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd4vf\" (UniqueName: \"kubernetes.io/projected/b8877212-eee4-4c1e-9817-371a6b99d375-kube-api-access-fd4vf\") pod \"payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4\" (UID: \"b8877212-eee4-4c1e-9817-371a6b99d375\") " pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"
Jul 29 15:44:39.421158 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.421128    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d9a5caf-3f97-43d7-ab47-7845876fbcff-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2\" (UID: \"4d9a5caf-3f97-43d7-ab47-7845876fbcff\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"
Jul 29 15:44:39.421350 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.421237    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68vq5\" (UniqueName: \"kubernetes.io/projected/4d9a5caf-3f97-43d7-ab47-7845876fbcff-kube-api-access-68vq5\") pod \"payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2\" (UID: \"4d9a5caf-3f97-43d7-ab47-7845876fbcff\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"
Jul 29 15:44:39.421649 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.421629    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"
Jul 29 15:44:39.421996 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.421969    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d9a5caf-3f97-43d7-ab47-7845876fbcff-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2\" (UID: \"4d9a5caf-3f97-43d7-ab47-7845876fbcff\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"
Jul 29 15:44:39.431742 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.431714    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68vq5\" (UniqueName: \"kubernetes.io/projected/4d9a5caf-3f97-43d7-ab47-7845876fbcff-kube-api-access-68vq5\") pod \"payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2\" (UID: \"4d9a5caf-3f97-43d7-ab47-7845876fbcff\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"
Jul 29 15:44:39.465752 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.465718    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"
Jul 29 15:44:39.571183 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.571116    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"]
Jul 29 15:44:39.580585 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.580553    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp" event={"ID":"611b4648-c680-4ccf-99e1-c192895ccb73","Type":"ContainerStarted","Data":"a83edea5b8d23a163f89320ecfa6f63cac58918d8acf244eca6e2de0509e3700"}
Jul 29 15:44:39.628379 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:39.628349    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"]
Jul 29 15:44:39.632722 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:44:39.632696    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d9a5caf_3f97_43d7_ab47_7845876fbcff.slice/crio-879d84fe8ae0a190f0341a720964cd958af32cf5c1eea4fcdb831fd0c4d2eb46 WatchSource:0}: Error finding container 879d84fe8ae0a190f0341a720964cd958af32cf5c1eea4fcdb831fd0c4d2eb46: Status 404 returned error can't find the container with id 879d84fe8ae0a190f0341a720964cd958af32cf5c1eea4fcdb831fd0c4d2eb46
Jul 29 15:44:40.587928 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:40.587896    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4" event={"ID":"b8877212-eee4-4c1e-9817-371a6b99d375","Type":"ContainerStarted","Data":"dd7290ada21dc5f5509ed624de2a5a60c6abaa772a2afd533cf908a7c17152d3"}
Jul 29 15:44:40.588397 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:40.587955    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4" event={"ID":"b8877212-eee4-4c1e-9817-371a6b99d375","Type":"ContainerStarted","Data":"d05971ac00ad0696b6371cb1bec334cc531a363ad4eab4099dd31b8212a7abf0"}
Jul 29 15:44:40.588397 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:40.587983    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"
Jul 29 15:44:40.589625 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:40.589599    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp" event={"ID":"611b4648-c680-4ccf-99e1-c192895ccb73","Type":"ContainerStarted","Data":"17a3d96bef27f07ebabb0e603a5ab0ef68698f76f3b82a47800e4518709341a0"}
Jul 29 15:44:40.589746 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:40.589654    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"
Jul 29 15:44:40.591051 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:40.591028    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2" event={"ID":"4d9a5caf-3f97-43d7-ab47-7845876fbcff","Type":"ContainerStarted","Data":"eca125812d0b62fdc527d47a9ad68da2aa15a749adae3f4c433916508df631ba"}
Jul 29 15:44:40.591136 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:40.591081    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"
Jul 29 15:44:40.591136 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:40.591100    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2" event={"ID":"4d9a5caf-3f97-43d7-ab47-7845876fbcff","Type":"ContainerStarted","Data":"879d84fe8ae0a190f0341a720964cd958af32cf5c1eea4fcdb831fd0c4d2eb46"}
Jul 29 15:44:40.604597 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:40.604559    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4" podStartSLOduration=1.604545467 podStartE2EDuration="1.604545467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:44:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:44:40.602354791 +0000 UTC m=+2013.751648688" watchObservedRunningTime="2026-07-29 15:44:40.604545467 +0000 UTC m=+2013.753839381"
Jul 29 15:44:40.620103 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:40.620049    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp" podStartSLOduration=1.541936902 podStartE2EDuration="2.620031256s" totalImagesPullingTime="1.078094354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:44:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:44:40.619185097 +0000 UTC m=+2013.768478997" watchObservedRunningTime="2026-07-29 15:44:40.620031256 +0000 UTC m=+2013.769325157"
Jul 29 15:44:40.638019 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:40.637973    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2" podStartSLOduration=1.6379576729999998 podStartE2EDuration="1.637957673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:44:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:44:40.635018304 +0000 UTC m=+2013.784312204" watchObservedRunningTime="2026-07-29 15:44:40.637957673 +0000 UTC m=+2013.787251576"
Jul 29 15:44:46.601529 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:46.601500    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"
Jul 29 15:44:51.598531 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:51.598492    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"
Jul 29 15:44:51.598953 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:44:51.598803    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"
Jul 29 15:45:09.694929 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:09.694893    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"]
Jul 29 15:45:09.695718 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:09.695660    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp" podUID="611b4648-c680-4ccf-99e1-c192895ccb73" containerName="maas-api" containerID="cri-o://17a3d96bef27f07ebabb0e603a5ab0ef68698f76f3b82a47800e4518709341a0" gracePeriod=30
Jul 29 15:45:09.756935 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:09.756898    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"]
Jul 29 15:45:09.757195 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:09.757145    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4" podUID="b8877212-eee4-4c1e-9817-371a6b99d375" containerName="payload-processing" containerID="cri-o://dd7290ada21dc5f5509ed624de2a5a60c6abaa772a2afd533cf908a7c17152d3" gracePeriod=30
Jul 29 15:45:09.766224 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:09.766198    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"]
Jul 29 15:45:09.766685 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:09.766466    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2" podUID="4d9a5caf-3f97-43d7-ab47-7845876fbcff" containerName="payload-pre-processing" containerID="cri-o://eca125812d0b62fdc527d47a9ad68da2aa15a749adae3f4c433916508df631ba" gracePeriod=30
Jul 29 15:45:10.054051 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.054022    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"
Jul 29 15:45:10.057356 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.057309    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"
Jul 29 15:45:10.084059 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.083450    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"
Jul 29 15:45:10.208183 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.208146    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/611b4648-c680-4ccf-99e1-c192895ccb73-openshift-service-ca\") pod \"611b4648-c680-4ccf-99e1-c192895ccb73\" (UID: \"611b4648-c680-4ccf-99e1-c192895ccb73\") "
Jul 29 15:45:10.208402 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.208196    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d9a5caf-3f97-43d7-ab47-7845876fbcff-plugins-config-volume\") pod \"4d9a5caf-3f97-43d7-ab47-7845876fbcff\" (UID: \"4d9a5caf-3f97-43d7-ab47-7845876fbcff\") "
Jul 29 15:45:10.208402 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.208237    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gr8t\" (UniqueName: \"kubernetes.io/projected/611b4648-c680-4ccf-99e1-c192895ccb73-kube-api-access-6gr8t\") pod \"611b4648-c680-4ccf-99e1-c192895ccb73\" (UID: \"611b4648-c680-4ccf-99e1-c192895ccb73\") "
Jul 29 15:45:10.208402 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.208266    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fd4vf\" (UniqueName: \"kubernetes.io/projected/b8877212-eee4-4c1e-9817-371a6b99d375-kube-api-access-fd4vf\") pod \"b8877212-eee4-4c1e-9817-371a6b99d375\" (UID: \"b8877212-eee4-4c1e-9817-371a6b99d375\") "
Jul 29 15:45:10.208402 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.208284    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8877212-eee4-4c1e-9817-371a6b99d375-plugins-config-volume\") pod \"b8877212-eee4-4c1e-9817-371a6b99d375\" (UID: \"b8877212-eee4-4c1e-9817-371a6b99d375\") "
Jul 29 15:45:10.208402 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.208307    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/611b4648-c680-4ccf-99e1-c192895ccb73-maas-api-tls\") pod \"611b4648-c680-4ccf-99e1-c192895ccb73\" (UID: \"611b4648-c680-4ccf-99e1-c192895ccb73\") "
Jul 29 15:45:10.208402 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.208363    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68vq5\" (UniqueName: \"kubernetes.io/projected/4d9a5caf-3f97-43d7-ab47-7845876fbcff-kube-api-access-68vq5\") pod \"4d9a5caf-3f97-43d7-ab47-7845876fbcff\" (UID: \"4d9a5caf-3f97-43d7-ab47-7845876fbcff\") "
Jul 29 15:45:10.208749 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.208669    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d9a5caf-3f97-43d7-ab47-7845876fbcff-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4d9a5caf-3f97-43d7-ab47-7845876fbcff" (UID: "4d9a5caf-3f97-43d7-ab47-7845876fbcff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:45:10.208749 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.208669    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8877212-eee4-4c1e-9817-371a6b99d375-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b8877212-eee4-4c1e-9817-371a6b99d375" (UID: "b8877212-eee4-4c1e-9817-371a6b99d375"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:45:10.209553 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.209528    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/611b4648-c680-4ccf-99e1-c192895ccb73-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "611b4648-c680-4ccf-99e1-c192895ccb73" (UID: "611b4648-c680-4ccf-99e1-c192895ccb73"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:45:10.210660 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.210639    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/611b4648-c680-4ccf-99e1-c192895ccb73-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "611b4648-c680-4ccf-99e1-c192895ccb73" (UID: "611b4648-c680-4ccf-99e1-c192895ccb73"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:45:10.210730 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.210700    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/611b4648-c680-4ccf-99e1-c192895ccb73-kube-api-access-6gr8t" (OuterVolumeSpecName: "kube-api-access-6gr8t") pod "611b4648-c680-4ccf-99e1-c192895ccb73" (UID: "611b4648-c680-4ccf-99e1-c192895ccb73"). InnerVolumeSpecName "kube-api-access-6gr8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:45:10.210898 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.210881    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8877212-eee4-4c1e-9817-371a6b99d375-kube-api-access-fd4vf" (OuterVolumeSpecName: "kube-api-access-fd4vf") pod "b8877212-eee4-4c1e-9817-371a6b99d375" (UID: "b8877212-eee4-4c1e-9817-371a6b99d375"). InnerVolumeSpecName "kube-api-access-fd4vf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:45:10.210940 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.210905    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d9a5caf-3f97-43d7-ab47-7845876fbcff-kube-api-access-68vq5" (OuterVolumeSpecName: "kube-api-access-68vq5") pod "4d9a5caf-3f97-43d7-ab47-7845876fbcff" (UID: "4d9a5caf-3f97-43d7-ab47-7845876fbcff"). InnerVolumeSpecName "kube-api-access-68vq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:45:10.309274 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.309200    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d9a5caf-3f97-43d7-ab47-7845876fbcff-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:45:10.309274 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.309227    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gr8t\" (UniqueName: \"kubernetes.io/projected/611b4648-c680-4ccf-99e1-c192895ccb73-kube-api-access-6gr8t\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:45:10.309274 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.309236    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fd4vf\" (UniqueName: \"kubernetes.io/projected/b8877212-eee4-4c1e-9817-371a6b99d375-kube-api-access-fd4vf\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:45:10.309274 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.309247    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8877212-eee4-4c1e-9817-371a6b99d375-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:45:10.309274 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.309257    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/611b4648-c680-4ccf-99e1-c192895ccb73-maas-api-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:45:10.309274 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.309266    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68vq5\" (UniqueName: \"kubernetes.io/projected/4d9a5caf-3f97-43d7-ab47-7845876fbcff-kube-api-access-68vq5\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:45:10.309274 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.309274    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/611b4648-c680-4ccf-99e1-c192895ccb73-openshift-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:45:10.744669 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.744637    2575 generic.go:356] "Generic (PLEG): container finished" podID="4d9a5caf-3f97-43d7-ab47-7845876fbcff" containerID="eca125812d0b62fdc527d47a9ad68da2aa15a749adae3f4c433916508df631ba" exitCode=0
Jul 29 15:45:10.745083 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.744692    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"
Jul 29 15:45:10.745083 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.744724    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2" event={"ID":"4d9a5caf-3f97-43d7-ab47-7845876fbcff","Type":"ContainerDied","Data":"eca125812d0b62fdc527d47a9ad68da2aa15a749adae3f4c433916508df631ba"}
Jul 29 15:45:10.745083 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.744773    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2" event={"ID":"4d9a5caf-3f97-43d7-ab47-7845876fbcff","Type":"ContainerDied","Data":"879d84fe8ae0a190f0341a720964cd958af32cf5c1eea4fcdb831fd0c4d2eb46"}
Jul 29 15:45:10.745083 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.744797    2575 scope.go:117] "RemoveContainer" containerID="eca125812d0b62fdc527d47a9ad68da2aa15a749adae3f4c433916508df631ba"
Jul 29 15:45:10.746193 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.746081    2575 generic.go:356] "Generic (PLEG): container finished" podID="b8877212-eee4-4c1e-9817-371a6b99d375" containerID="dd7290ada21dc5f5509ed624de2a5a60c6abaa772a2afd533cf908a7c17152d3" exitCode=0
Jul 29 15:45:10.746193 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.746141    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"
Jul 29 15:45:10.746193 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.746160    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4" event={"ID":"b8877212-eee4-4c1e-9817-371a6b99d375","Type":"ContainerDied","Data":"dd7290ada21dc5f5509ed624de2a5a60c6abaa772a2afd533cf908a7c17152d3"}
Jul 29 15:45:10.746478 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.746207    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4" event={"ID":"b8877212-eee4-4c1e-9817-371a6b99d375","Type":"ContainerDied","Data":"d05971ac00ad0696b6371cb1bec334cc531a363ad4eab4099dd31b8212a7abf0"}
Jul 29 15:45:10.747671 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.747635    2575 generic.go:356] "Generic (PLEG): container finished" podID="611b4648-c680-4ccf-99e1-c192895ccb73" containerID="17a3d96bef27f07ebabb0e603a5ab0ef68698f76f3b82a47800e4518709341a0" exitCode=0
Jul 29 15:45:10.747805 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.747702    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp" event={"ID":"611b4648-c680-4ccf-99e1-c192895ccb73","Type":"ContainerDied","Data":"17a3d96bef27f07ebabb0e603a5ab0ef68698f76f3b82a47800e4518709341a0"}
Jul 29 15:45:10.747805 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.747741    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp" event={"ID":"611b4648-c680-4ccf-99e1-c192895ccb73","Type":"ContainerDied","Data":"a83edea5b8d23a163f89320ecfa6f63cac58918d8acf244eca6e2de0509e3700"}
Jul 29 15:45:10.747805 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.747750    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"
Jul 29 15:45:10.755241 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.755221    2575 scope.go:117] "RemoveContainer" containerID="eca125812d0b62fdc527d47a9ad68da2aa15a749adae3f4c433916508df631ba"
Jul 29 15:45:10.755670 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:45:10.755649    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eca125812d0b62fdc527d47a9ad68da2aa15a749adae3f4c433916508df631ba\": container with ID starting with eca125812d0b62fdc527d47a9ad68da2aa15a749adae3f4c433916508df631ba not found: ID does not exist" containerID="eca125812d0b62fdc527d47a9ad68da2aa15a749adae3f4c433916508df631ba"
Jul 29 15:45:10.755776 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.755682    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eca125812d0b62fdc527d47a9ad68da2aa15a749adae3f4c433916508df631ba"} err="failed to get container status \"eca125812d0b62fdc527d47a9ad68da2aa15a749adae3f4c433916508df631ba\": rpc error: code = NotFound desc = could not find container \"eca125812d0b62fdc527d47a9ad68da2aa15a749adae3f4c433916508df631ba\": container with ID starting with eca125812d0b62fdc527d47a9ad68da2aa15a749adae3f4c433916508df631ba not found: ID does not exist"
Jul 29 15:45:10.755776 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.755705    2575 scope.go:117] "RemoveContainer" containerID="dd7290ada21dc5f5509ed624de2a5a60c6abaa772a2afd533cf908a7c17152d3"
Jul 29 15:45:10.763571 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.763554    2575 scope.go:117] "RemoveContainer" containerID="dd7290ada21dc5f5509ed624de2a5a60c6abaa772a2afd533cf908a7c17152d3"
Jul 29 15:45:10.763807 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:45:10.763787    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd7290ada21dc5f5509ed624de2a5a60c6abaa772a2afd533cf908a7c17152d3\": container with ID starting with dd7290ada21dc5f5509ed624de2a5a60c6abaa772a2afd533cf908a7c17152d3 not found: ID does not exist" containerID="dd7290ada21dc5f5509ed624de2a5a60c6abaa772a2afd533cf908a7c17152d3"
Jul 29 15:45:10.763848 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.763819    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd7290ada21dc5f5509ed624de2a5a60c6abaa772a2afd533cf908a7c17152d3"} err="failed to get container status \"dd7290ada21dc5f5509ed624de2a5a60c6abaa772a2afd533cf908a7c17152d3\": rpc error: code = NotFound desc = could not find container \"dd7290ada21dc5f5509ed624de2a5a60c6abaa772a2afd533cf908a7c17152d3\": container with ID starting with dd7290ada21dc5f5509ed624de2a5a60c6abaa772a2afd533cf908a7c17152d3 not found: ID does not exist"
Jul 29 15:45:10.763848 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.763839    2575 scope.go:117] "RemoveContainer" containerID="17a3d96bef27f07ebabb0e603a5ab0ef68698f76f3b82a47800e4518709341a0"
Jul 29 15:45:10.770605 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.770580    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"]
Jul 29 15:45:10.772392 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.772335    2575 scope.go:117] "RemoveContainer" containerID="17a3d96bef27f07ebabb0e603a5ab0ef68698f76f3b82a47800e4518709341a0"
Jul 29 15:45:10.772655 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:45:10.772624    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17a3d96bef27f07ebabb0e603a5ab0ef68698f76f3b82a47800e4518709341a0\": container with ID starting with 17a3d96bef27f07ebabb0e603a5ab0ef68698f76f3b82a47800e4518709341a0 not found: ID does not exist" containerID="17a3d96bef27f07ebabb0e603a5ab0ef68698f76f3b82a47800e4518709341a0"
Jul 29 15:45:10.772755 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.772654    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17a3d96bef27f07ebabb0e603a5ab0ef68698f76f3b82a47800e4518709341a0"} err="failed to get container status \"17a3d96bef27f07ebabb0e603a5ab0ef68698f76f3b82a47800e4518709341a0\": rpc error: code = NotFound desc = could not find container \"17a3d96bef27f07ebabb0e603a5ab0ef68698f76f3b82a47800e4518709341a0\": container with ID starting with 17a3d96bef27f07ebabb0e603a5ab0ef68698f76f3b82a47800e4518709341a0 not found: ID does not exist"
Jul 29 15:45:10.774285 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.774267    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-adc0e91c-7597dd587b-sxbg2"]
Jul 29 15:45:10.785601 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.785580    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"]
Jul 29 15:45:10.790945 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.790927    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-adc0e91c-64f6db4c89-69kt4"]
Jul 29 15:45:10.802069 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.802050    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"]
Jul 29 15:45:10.804669 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:10.804650    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-adc0e91c-647795c996-k29bp"]
Jul 29 15:45:11.295880 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:11.295845    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d9a5caf-3f97-43d7-ab47-7845876fbcff" path="/var/lib/kubelet/pods/4d9a5caf-3f97-43d7-ab47-7845876fbcff/volumes"
Jul 29 15:45:11.296230 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:11.296217    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="611b4648-c680-4ccf-99e1-c192895ccb73" path="/var/lib/kubelet/pods/611b4648-c680-4ccf-99e1-c192895ccb73/volumes"
Jul 29 15:45:11.296547 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:11.296534    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8877212-eee4-4c1e-9817-371a6b99d375" path="/var/lib/kubelet/pods/b8877212-eee4-4c1e-9817-371a6b99d375/volumes"
Jul 29 15:45:13.403175 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.403135    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"]
Jul 29 15:45:13.403749 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.403731    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8877212-eee4-4c1e-9817-371a6b99d375" containerName="payload-processing"
Jul 29 15:45:13.403812 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.403753    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8877212-eee4-4c1e-9817-371a6b99d375" containerName="payload-processing"
Jul 29 15:45:13.403812 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.403786    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="611b4648-c680-4ccf-99e1-c192895ccb73" containerName="maas-api"
Jul 29 15:45:13.403812 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.403794    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="611b4648-c680-4ccf-99e1-c192895ccb73" containerName="maas-api"
Jul 29 15:45:13.403915 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.403814    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d9a5caf-3f97-43d7-ab47-7845876fbcff" containerName="payload-pre-processing"
Jul 29 15:45:13.403915 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.403823    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d9a5caf-3f97-43d7-ab47-7845876fbcff" containerName="payload-pre-processing"
Jul 29 15:45:13.403915 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.403903    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="611b4648-c680-4ccf-99e1-c192895ccb73" containerName="maas-api"
Jul 29 15:45:13.404006 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.403916    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8877212-eee4-4c1e-9817-371a6b99d375" containerName="payload-processing"
Jul 29 15:45:13.404006 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.403925    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d9a5caf-3f97-43d7-ab47-7845876fbcff" containerName="payload-pre-processing"
Jul 29 15:45:13.407262 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.407241    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"
Jul 29 15:45:13.409555 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.409532    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-16426b01-serving-cert\""
Jul 29 15:45:13.417337 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.417293    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"]
Jul 29 15:45:13.539802 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.539765    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-maas-api-tls\") pod \"maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp\" (UID: \"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"
Jul 29 15:45:13.540005 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.539815    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-openshift-service-ca\") pod \"maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp\" (UID: \"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"
Jul 29 15:45:13.540005 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.539892    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smcs4\" (UniqueName: \"kubernetes.io/projected/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-kube-api-access-smcs4\") pod \"maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp\" (UID: \"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"
Jul 29 15:45:13.640718 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.640675    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-openshift-service-ca\") pod \"maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp\" (UID: \"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"
Jul 29 15:45:13.640895 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.640733    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smcs4\" (UniqueName: \"kubernetes.io/projected/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-kube-api-access-smcs4\") pod \"maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp\" (UID: \"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"
Jul 29 15:45:13.640895 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.640828    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-maas-api-tls\") pod \"maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp\" (UID: \"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"
Jul 29 15:45:13.641555 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.641523    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-openshift-service-ca\") pod \"maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp\" (UID: \"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"
Jul 29 15:45:13.643235 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.643204    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-maas-api-tls\") pod \"maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp\" (UID: \"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"
Jul 29 15:45:13.649597 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.649573    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smcs4\" (UniqueName: \"kubernetes.io/projected/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-kube-api-access-smcs4\") pod \"maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp\" (UID: \"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"
Jul 29 15:45:13.718542 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.718465    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"
Jul 29 15:45:13.865573 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:13.865547    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"]
Jul 29 15:45:13.867943 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:45:13.867910    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09fcc3f0_67fa_44d8_9acf_02cf2cb8377d.slice/crio-fdbb5f3df76dbeb7a7cf56ce6f2de3ee83a7a28fcbbf38e6ba0e78b85f4110d0 WatchSource:0}: Error finding container fdbb5f3df76dbeb7a7cf56ce6f2de3ee83a7a28fcbbf38e6ba0e78b85f4110d0: Status 404 returned error can't find the container with id fdbb5f3df76dbeb7a7cf56ce6f2de3ee83a7a28fcbbf38e6ba0e78b85f4110d0
Jul 29 15:45:14.091364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.091267    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"]
Jul 29 15:45:14.095919 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.095900    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"
Jul 29 15:45:14.098971 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.098951    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-16426b01\""
Jul 29 15:45:14.106420 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.106391    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"]
Jul 29 15:45:14.117525 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.117504    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"]
Jul 29 15:45:14.121143 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.121125    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"
Jul 29 15:45:14.124830 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.124610    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-16426b01-dockercfg-s6lch\""
Jul 29 15:45:14.133651 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.133628    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"]
Jul 29 15:45:14.247480 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.247447    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvlbc\" (UniqueName: \"kubernetes.io/projected/6ba71571-d564-46d0-aae0-7a5540f4839a-kube-api-access-bvlbc\") pod \"payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw\" (UID: \"6ba71571-d564-46d0-aae0-7a5540f4839a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"
Jul 29 15:45:14.247689 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.247484    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bj8f\" (UniqueName: \"kubernetes.io/projected/c63fad9c-284b-4e1b-8815-b71a5ff58571-kube-api-access-7bj8f\") pod \"payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl\" (UID: \"c63fad9c-284b-4e1b-8815-b71a5ff58571\") " pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"
Jul 29 15:45:14.247689 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.247514    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ba71571-d564-46d0-aae0-7a5540f4839a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw\" (UID: \"6ba71571-d564-46d0-aae0-7a5540f4839a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"
Jul 29 15:45:14.247689 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.247599    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c63fad9c-284b-4e1b-8815-b71a5ff58571-plugins-config-volume\") pod \"payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl\" (UID: \"c63fad9c-284b-4e1b-8815-b71a5ff58571\") " pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"
Jul 29 15:45:14.348299 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.348214    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvlbc\" (UniqueName: \"kubernetes.io/projected/6ba71571-d564-46d0-aae0-7a5540f4839a-kube-api-access-bvlbc\") pod \"payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw\" (UID: \"6ba71571-d564-46d0-aae0-7a5540f4839a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"
Jul 29 15:45:14.348299 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.348252    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bj8f\" (UniqueName: \"kubernetes.io/projected/c63fad9c-284b-4e1b-8815-b71a5ff58571-kube-api-access-7bj8f\") pod \"payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl\" (UID: \"c63fad9c-284b-4e1b-8815-b71a5ff58571\") " pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"
Jul 29 15:45:14.348299 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.348276    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ba71571-d564-46d0-aae0-7a5540f4839a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw\" (UID: \"6ba71571-d564-46d0-aae0-7a5540f4839a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"
Jul 29 15:45:14.348640 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.348350    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c63fad9c-284b-4e1b-8815-b71a5ff58571-plugins-config-volume\") pod \"payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl\" (UID: \"c63fad9c-284b-4e1b-8815-b71a5ff58571\") " pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"
Jul 29 15:45:14.349115 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.349094    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c63fad9c-284b-4e1b-8815-b71a5ff58571-plugins-config-volume\") pod \"payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl\" (UID: \"c63fad9c-284b-4e1b-8815-b71a5ff58571\") " pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"
Jul 29 15:45:14.349190 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.349170    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ba71571-d564-46d0-aae0-7a5540f4839a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw\" (UID: \"6ba71571-d564-46d0-aae0-7a5540f4839a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"
Jul 29 15:45:14.358159 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.358133    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvlbc\" (UniqueName: \"kubernetes.io/projected/6ba71571-d564-46d0-aae0-7a5540f4839a-kube-api-access-bvlbc\") pod \"payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw\" (UID: \"6ba71571-d564-46d0-aae0-7a5540f4839a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"
Jul 29 15:45:14.358906 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.358888    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bj8f\" (UniqueName: \"kubernetes.io/projected/c63fad9c-284b-4e1b-8815-b71a5ff58571-kube-api-access-7bj8f\") pod \"payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl\" (UID: \"c63fad9c-284b-4e1b-8815-b71a5ff58571\") " pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"
Jul 29 15:45:14.408523 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.408494    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"
Jul 29 15:45:14.432754 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.432726    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"
Jul 29 15:45:14.563698 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.563669    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"]
Jul 29 15:45:14.566312 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:45:14.566282    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ba71571_d564_46d0_aae0_7a5540f4839a.slice/crio-1ac2d05cd89c69bf5c1f8e8c383520f307d581caec5c84857ead3fba36b3197b WatchSource:0}: Error finding container 1ac2d05cd89c69bf5c1f8e8c383520f307d581caec5c84857ead3fba36b3197b: Status 404 returned error can't find the container with id 1ac2d05cd89c69bf5c1f8e8c383520f307d581caec5c84857ead3fba36b3197b
Jul 29 15:45:14.597840 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.597702    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"]
Jul 29 15:45:14.599961 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:45:14.599908    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc63fad9c_284b_4e1b_8815_b71a5ff58571.slice/crio-40511189272e07abc415099facb11d4fdbcde68e588a511f1ac7658090fe7dfe WatchSource:0}: Error finding container 40511189272e07abc415099facb11d4fdbcde68e588a511f1ac7658090fe7dfe: Status 404 returned error can't find the container with id 40511189272e07abc415099facb11d4fdbcde68e588a511f1ac7658090fe7dfe
Jul 29 15:45:14.774404 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.774360    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw" event={"ID":"6ba71571-d564-46d0-aae0-7a5540f4839a","Type":"ContainerStarted","Data":"41dfa97012ca944156a05b8279afdf88dac9f6b879a4afc32b7d49417676ac45"}
Jul 29 15:45:14.774585 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.774426    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"
Jul 29 15:45:14.774585 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.774447    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw" event={"ID":"6ba71571-d564-46d0-aae0-7a5540f4839a","Type":"ContainerStarted","Data":"1ac2d05cd89c69bf5c1f8e8c383520f307d581caec5c84857ead3fba36b3197b"}
Jul 29 15:45:14.775938 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.775914    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl" event={"ID":"c63fad9c-284b-4e1b-8815-b71a5ff58571","Type":"ContainerStarted","Data":"3c83fc7631dcf01f055057f25df1ea5802d4f9447a89a88ae358f83c6df60321"}
Jul 29 15:45:14.776047 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.775960    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"
Jul 29 15:45:14.776047 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.775982    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl" event={"ID":"c63fad9c-284b-4e1b-8815-b71a5ff58571","Type":"ContainerStarted","Data":"40511189272e07abc415099facb11d4fdbcde68e588a511f1ac7658090fe7dfe"}
Jul 29 15:45:14.777363 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.777340    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp" event={"ID":"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d","Type":"ContainerStarted","Data":"2925d73db041fb929656ad0ab8052699c7bf3a781c4679738ff21b078e17b007"}
Jul 29 15:45:14.777453 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.777381    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp" event={"ID":"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d","Type":"ContainerStarted","Data":"fdbb5f3df76dbeb7a7cf56ce6f2de3ee83a7a28fcbbf38e6ba0e78b85f4110d0"}
Jul 29 15:45:14.777453 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.777445    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"
Jul 29 15:45:14.792473 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.792427    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw" podStartSLOduration=0.792413275 podStartE2EDuration="792.413275ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:45:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:45:14.791584588 +0000 UTC m=+2047.940878484" watchObservedRunningTime="2026-07-29 15:45:14.792413275 +0000 UTC m=+2047.941707174"
Jul 29 15:45:14.809145 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.809101    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl" podStartSLOduration=0.809082941 podStartE2EDuration="809.082941ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:45:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:45:14.808645259 +0000 UTC m=+2047.957939159" watchObservedRunningTime="2026-07-29 15:45:14.809082941 +0000 UTC m=+2047.958376841"
Jul 29 15:45:14.827486 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:14.827433    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp" podStartSLOduration=1.083656604 podStartE2EDuration="1.827414333s" totalImagesPullingTime="743.757729ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:45:14.825631178 +0000 UTC m=+2047.974925079" watchObservedRunningTime="2026-07-29 15:45:14.827414333 +0000 UTC m=+2047.976708233"
Jul 29 15:45:20.789373 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:20.789310    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"
Jul 29 15:45:25.784549 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:25.784520    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"
Jul 29 15:45:25.784983 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:25.784684    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"
Jul 29 15:45:44.070119 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.070043    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"]
Jul 29 15:45:44.070904 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.070300    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp" podUID="09fcc3f0-67fa-44d8-9acf-02cf2cb8377d" containerName="maas-api" containerID="cri-o://2925d73db041fb929656ad0ab8052699c7bf3a781c4679738ff21b078e17b007" gracePeriod=30
Jul 29 15:45:44.136040 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.135997    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"]
Jul 29 15:45:44.136279 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.136252    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl" podUID="c63fad9c-284b-4e1b-8815-b71a5ff58571" containerName="payload-processing" containerID="cri-o://3c83fc7631dcf01f055057f25df1ea5802d4f9447a89a88ae358f83c6df60321" gracePeriod=30
Jul 29 15:45:44.149434 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.149403    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"]
Jul 29 15:45:44.149687 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.149637    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw" podUID="6ba71571-d564-46d0-aae0-7a5540f4839a" containerName="payload-pre-processing" containerID="cri-o://41dfa97012ca944156a05b8279afdf88dac9f6b879a4afc32b7d49417676ac45" gracePeriod=30
Jul 29 15:45:44.417109 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.417078    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"
Jul 29 15:45:44.452382 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.452355    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"
Jul 29 15:45:44.462432 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.462411    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"
Jul 29 15:45:44.512173 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.512140    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ba71571-d564-46d0-aae0-7a5540f4839a-plugins-config-volume\") pod \"6ba71571-d564-46d0-aae0-7a5540f4839a\" (UID: \"6ba71571-d564-46d0-aae0-7a5540f4839a\") "
Jul 29 15:45:44.512375 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.512188    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-openshift-service-ca\") pod \"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d\" (UID: \"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d\") "
Jul 29 15:45:44.512375 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.512208    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c63fad9c-284b-4e1b-8815-b71a5ff58571-plugins-config-volume\") pod \"c63fad9c-284b-4e1b-8815-b71a5ff58571\" (UID: \"c63fad9c-284b-4e1b-8815-b71a5ff58571\") "
Jul 29 15:45:44.512375 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.512235    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-maas-api-tls\") pod \"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d\" (UID: \"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d\") "
Jul 29 15:45:44.512375 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.512332    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvlbc\" (UniqueName: \"kubernetes.io/projected/6ba71571-d564-46d0-aae0-7a5540f4839a-kube-api-access-bvlbc\") pod \"6ba71571-d564-46d0-aae0-7a5540f4839a\" (UID: \"6ba71571-d564-46d0-aae0-7a5540f4839a\") "
Jul 29 15:45:44.512375 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.512372    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bj8f\" (UniqueName: \"kubernetes.io/projected/c63fad9c-284b-4e1b-8815-b71a5ff58571-kube-api-access-7bj8f\") pod \"c63fad9c-284b-4e1b-8815-b71a5ff58571\" (UID: \"c63fad9c-284b-4e1b-8815-b71a5ff58571\") "
Jul 29 15:45:44.512669 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.512411    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smcs4\" (UniqueName: \"kubernetes.io/projected/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-kube-api-access-smcs4\") pod \"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d\" (UID: \"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d\") "
Jul 29 15:45:44.512669 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.512611    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "09fcc3f0-67fa-44d8-9acf-02cf2cb8377d" (UID: "09fcc3f0-67fa-44d8-9acf-02cf2cb8377d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:45:44.512669 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.512622    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ba71571-d564-46d0-aae0-7a5540f4839a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6ba71571-d564-46d0-aae0-7a5540f4839a" (UID: "6ba71571-d564-46d0-aae0-7a5540f4839a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:45:44.512812 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.512779    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ba71571-d564-46d0-aae0-7a5540f4839a-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:45:44.512812 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.512802    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-openshift-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:45:44.512897 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.512876    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c63fad9c-284b-4e1b-8815-b71a5ff58571-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c63fad9c-284b-4e1b-8815-b71a5ff58571" (UID: "c63fad9c-284b-4e1b-8815-b71a5ff58571"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:45:44.514829 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.514798    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "09fcc3f0-67fa-44d8-9acf-02cf2cb8377d" (UID: "09fcc3f0-67fa-44d8-9acf-02cf2cb8377d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:45:44.514829 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.514813    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ba71571-d564-46d0-aae0-7a5540f4839a-kube-api-access-bvlbc" (OuterVolumeSpecName: "kube-api-access-bvlbc") pod "6ba71571-d564-46d0-aae0-7a5540f4839a" (UID: "6ba71571-d564-46d0-aae0-7a5540f4839a"). InnerVolumeSpecName "kube-api-access-bvlbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:45:44.514969 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.514860    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-kube-api-access-smcs4" (OuterVolumeSpecName: "kube-api-access-smcs4") pod "09fcc3f0-67fa-44d8-9acf-02cf2cb8377d" (UID: "09fcc3f0-67fa-44d8-9acf-02cf2cb8377d"). InnerVolumeSpecName "kube-api-access-smcs4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:45:44.514969 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.514889    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c63fad9c-284b-4e1b-8815-b71a5ff58571-kube-api-access-7bj8f" (OuterVolumeSpecName: "kube-api-access-7bj8f") pod "c63fad9c-284b-4e1b-8815-b71a5ff58571" (UID: "c63fad9c-284b-4e1b-8815-b71a5ff58571"). InnerVolumeSpecName "kube-api-access-7bj8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:45:44.613795 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.613755    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smcs4\" (UniqueName: \"kubernetes.io/projected/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-kube-api-access-smcs4\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:45:44.613795 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.613788    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c63fad9c-284b-4e1b-8815-b71a5ff58571-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:45:44.613795 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.613798    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d-maas-api-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:45:44.613795 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.613807    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvlbc\" (UniqueName: \"kubernetes.io/projected/6ba71571-d564-46d0-aae0-7a5540f4839a-kube-api-access-bvlbc\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:45:44.614073 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.613816    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bj8f\" (UniqueName: \"kubernetes.io/projected/c63fad9c-284b-4e1b-8815-b71a5ff58571-kube-api-access-7bj8f\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:45:44.932208 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.932173    2575 generic.go:356] "Generic (PLEG): container finished" podID="6ba71571-d564-46d0-aae0-7a5540f4839a" containerID="41dfa97012ca944156a05b8279afdf88dac9f6b879a4afc32b7d49417676ac45" exitCode=0
Jul 29 15:45:44.932421 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.932215    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"
Jul 29 15:45:44.932421 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.932261    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw" event={"ID":"6ba71571-d564-46d0-aae0-7a5540f4839a","Type":"ContainerDied","Data":"41dfa97012ca944156a05b8279afdf88dac9f6b879a4afc32b7d49417676ac45"}
Jul 29 15:45:44.932421 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.932315    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw" event={"ID":"6ba71571-d564-46d0-aae0-7a5540f4839a","Type":"ContainerDied","Data":"1ac2d05cd89c69bf5c1f8e8c383520f307d581caec5c84857ead3fba36b3197b"}
Jul 29 15:45:44.932421 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.932353    2575 scope.go:117] "RemoveContainer" containerID="41dfa97012ca944156a05b8279afdf88dac9f6b879a4afc32b7d49417676ac45"
Jul 29 15:45:44.933735 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.933710    2575 generic.go:356] "Generic (PLEG): container finished" podID="c63fad9c-284b-4e1b-8815-b71a5ff58571" containerID="3c83fc7631dcf01f055057f25df1ea5802d4f9447a89a88ae358f83c6df60321" exitCode=0
Jul 29 15:45:44.933884 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.933749    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"
Jul 29 15:45:44.933884 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.933795    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl" event={"ID":"c63fad9c-284b-4e1b-8815-b71a5ff58571","Type":"ContainerDied","Data":"3c83fc7631dcf01f055057f25df1ea5802d4f9447a89a88ae358f83c6df60321"}
Jul 29 15:45:44.933884 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.933860    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl" event={"ID":"c63fad9c-284b-4e1b-8815-b71a5ff58571","Type":"ContainerDied","Data":"40511189272e07abc415099facb11d4fdbcde68e588a511f1ac7658090fe7dfe"}
Jul 29 15:45:44.935384 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.935360    2575 generic.go:356] "Generic (PLEG): container finished" podID="09fcc3f0-67fa-44d8-9acf-02cf2cb8377d" containerID="2925d73db041fb929656ad0ab8052699c7bf3a781c4679738ff21b078e17b007" exitCode=0
Jul 29 15:45:44.935511 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.935402    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp" event={"ID":"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d","Type":"ContainerDied","Data":"2925d73db041fb929656ad0ab8052699c7bf3a781c4679738ff21b078e17b007"}
Jul 29 15:45:44.935511 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.935407    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"
Jul 29 15:45:44.935511 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.935442    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp" event={"ID":"09fcc3f0-67fa-44d8-9acf-02cf2cb8377d","Type":"ContainerDied","Data":"fdbb5f3df76dbeb7a7cf56ce6f2de3ee83a7a28fcbbf38e6ba0e78b85f4110d0"}
Jul 29 15:45:44.942337 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.942299    2575 scope.go:117] "RemoveContainer" containerID="41dfa97012ca944156a05b8279afdf88dac9f6b879a4afc32b7d49417676ac45"
Jul 29 15:45:44.942609 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:45:44.942589    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41dfa97012ca944156a05b8279afdf88dac9f6b879a4afc32b7d49417676ac45\": container with ID starting with 41dfa97012ca944156a05b8279afdf88dac9f6b879a4afc32b7d49417676ac45 not found: ID does not exist" containerID="41dfa97012ca944156a05b8279afdf88dac9f6b879a4afc32b7d49417676ac45"
Jul 29 15:45:44.942691 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.942615    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41dfa97012ca944156a05b8279afdf88dac9f6b879a4afc32b7d49417676ac45"} err="failed to get container status \"41dfa97012ca944156a05b8279afdf88dac9f6b879a4afc32b7d49417676ac45\": rpc error: code = NotFound desc = could not find container \"41dfa97012ca944156a05b8279afdf88dac9f6b879a4afc32b7d49417676ac45\": container with ID starting with 41dfa97012ca944156a05b8279afdf88dac9f6b879a4afc32b7d49417676ac45 not found: ID does not exist"
Jul 29 15:45:44.942691 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.942631    2575 scope.go:117] "RemoveContainer" containerID="3c83fc7631dcf01f055057f25df1ea5802d4f9447a89a88ae358f83c6df60321"
Jul 29 15:45:44.950712 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.950697    2575 scope.go:117] "RemoveContainer" containerID="3c83fc7631dcf01f055057f25df1ea5802d4f9447a89a88ae358f83c6df60321"
Jul 29 15:45:44.950956 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:45:44.950940    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c83fc7631dcf01f055057f25df1ea5802d4f9447a89a88ae358f83c6df60321\": container with ID starting with 3c83fc7631dcf01f055057f25df1ea5802d4f9447a89a88ae358f83c6df60321 not found: ID does not exist" containerID="3c83fc7631dcf01f055057f25df1ea5802d4f9447a89a88ae358f83c6df60321"
Jul 29 15:45:44.951027 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.950962    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c83fc7631dcf01f055057f25df1ea5802d4f9447a89a88ae358f83c6df60321"} err="failed to get container status \"3c83fc7631dcf01f055057f25df1ea5802d4f9447a89a88ae358f83c6df60321\": rpc error: code = NotFound desc = could not find container \"3c83fc7631dcf01f055057f25df1ea5802d4f9447a89a88ae358f83c6df60321\": container with ID starting with 3c83fc7631dcf01f055057f25df1ea5802d4f9447a89a88ae358f83c6df60321 not found: ID does not exist"
Jul 29 15:45:44.951027 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.950976    2575 scope.go:117] "RemoveContainer" containerID="2925d73db041fb929656ad0ab8052699c7bf3a781c4679738ff21b078e17b007"
Jul 29 15:45:44.958607 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.958591    2575 scope.go:117] "RemoveContainer" containerID="2925d73db041fb929656ad0ab8052699c7bf3a781c4679738ff21b078e17b007"
Jul 29 15:45:44.958878 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:45:44.958856    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2925d73db041fb929656ad0ab8052699c7bf3a781c4679738ff21b078e17b007\": container with ID starting with 2925d73db041fb929656ad0ab8052699c7bf3a781c4679738ff21b078e17b007 not found: ID does not exist" containerID="2925d73db041fb929656ad0ab8052699c7bf3a781c4679738ff21b078e17b007"
Jul 29 15:45:44.958941 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.958887    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2925d73db041fb929656ad0ab8052699c7bf3a781c4679738ff21b078e17b007"} err="failed to get container status \"2925d73db041fb929656ad0ab8052699c7bf3a781c4679738ff21b078e17b007\": rpc error: code = NotFound desc = could not find container \"2925d73db041fb929656ad0ab8052699c7bf3a781c4679738ff21b078e17b007\": container with ID starting with 2925d73db041fb929656ad0ab8052699c7bf3a781c4679738ff21b078e17b007 not found: ID does not exist"
Jul 29 15:45:44.959289 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.959272    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"]
Jul 29 15:45:44.966569 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.966550    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-16426b01-668c945d84-wf6cw"]
Jul 29 15:45:44.977501 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.977479    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"]
Jul 29 15:45:44.979909 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.979888    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-16426b01-7cf755fcbc-6w5tp"]
Jul 29 15:45:44.990914 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.990894    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"]
Jul 29 15:45:44.995439 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:44.995420    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-16426b01-c899bc5cb-jstvl"]
Jul 29 15:45:45.295858 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:45.295785    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09fcc3f0-67fa-44d8-9acf-02cf2cb8377d" path="/var/lib/kubelet/pods/09fcc3f0-67fa-44d8-9acf-02cf2cb8377d/volumes"
Jul 29 15:45:45.296203 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:45.296122    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ba71571-d564-46d0-aae0-7a5540f4839a" path="/var/lib/kubelet/pods/6ba71571-d564-46d0-aae0-7a5540f4839a/volumes"
Jul 29 15:45:45.296441 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:45.296428    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c63fad9c-284b-4e1b-8815-b71a5ff58571" path="/var/lib/kubelet/pods/c63fad9c-284b-4e1b-8815-b71a5ff58571/volumes"
Jul 29 15:45:49.514589 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.514553    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"]
Jul 29 15:45:49.515158 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.515122    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ba71571-d564-46d0-aae0-7a5540f4839a" containerName="payload-pre-processing"
Jul 29 15:45:49.515158 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.515155    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ba71571-d564-46d0-aae0-7a5540f4839a" containerName="payload-pre-processing"
Jul 29 15:45:49.515376 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.515193    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09fcc3f0-67fa-44d8-9acf-02cf2cb8377d" containerName="maas-api"
Jul 29 15:45:49.515376 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.515203    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="09fcc3f0-67fa-44d8-9acf-02cf2cb8377d" containerName="maas-api"
Jul 29 15:45:49.515376 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.515223    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c63fad9c-284b-4e1b-8815-b71a5ff58571" containerName="payload-processing"
Jul 29 15:45:49.515376 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.515231    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c63fad9c-284b-4e1b-8815-b71a5ff58571" containerName="payload-processing"
Jul 29 15:45:49.515376 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.515348    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="09fcc3f0-67fa-44d8-9acf-02cf2cb8377d" containerName="maas-api"
Jul 29 15:45:49.515376 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.515365    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c63fad9c-284b-4e1b-8815-b71a5ff58571" containerName="payload-processing"
Jul 29 15:45:49.515376 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.515375    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ba71571-d564-46d0-aae0-7a5540f4839a" containerName="payload-pre-processing"
Jul 29 15:45:49.529986 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.529949    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"
Jul 29 15:45:49.530985 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.530907    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"]
Jul 29 15:45:49.532597 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.532572    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-1609e886-serving-cert\""
Jul 29 15:45:49.663203 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.663160    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62qz8\" (UniqueName: \"kubernetes.io/projected/35fa6bbb-7645-4674-8a33-73ac24e99401-kube-api-access-62qz8\") pod \"maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx\" (UID: \"35fa6bbb-7645-4674-8a33-73ac24e99401\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"
Jul 29 15:45:49.663395 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.663252    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35fa6bbb-7645-4674-8a33-73ac24e99401-maas-api-tls\") pod \"maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx\" (UID: \"35fa6bbb-7645-4674-8a33-73ac24e99401\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"
Jul 29 15:45:49.663395 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.663368    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35fa6bbb-7645-4674-8a33-73ac24e99401-openshift-service-ca\") pod \"maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx\" (UID: \"35fa6bbb-7645-4674-8a33-73ac24e99401\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"
Jul 29 15:45:49.764819 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.764737    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62qz8\" (UniqueName: \"kubernetes.io/projected/35fa6bbb-7645-4674-8a33-73ac24e99401-kube-api-access-62qz8\") pod \"maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx\" (UID: \"35fa6bbb-7645-4674-8a33-73ac24e99401\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"
Jul 29 15:45:49.764819 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.764777    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35fa6bbb-7645-4674-8a33-73ac24e99401-maas-api-tls\") pod \"maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx\" (UID: \"35fa6bbb-7645-4674-8a33-73ac24e99401\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"
Jul 29 15:45:49.764819 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.764816    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35fa6bbb-7645-4674-8a33-73ac24e99401-openshift-service-ca\") pod \"maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx\" (UID: \"35fa6bbb-7645-4674-8a33-73ac24e99401\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"
Jul 29 15:45:49.765555 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.765531    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35fa6bbb-7645-4674-8a33-73ac24e99401-openshift-service-ca\") pod \"maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx\" (UID: \"35fa6bbb-7645-4674-8a33-73ac24e99401\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"
Jul 29 15:45:49.767162 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.767143    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35fa6bbb-7645-4674-8a33-73ac24e99401-maas-api-tls\") pod \"maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx\" (UID: \"35fa6bbb-7645-4674-8a33-73ac24e99401\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"
Jul 29 15:45:49.773896 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.773874    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62qz8\" (UniqueName: \"kubernetes.io/projected/35fa6bbb-7645-4674-8a33-73ac24e99401-kube-api-access-62qz8\") pod \"maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx\" (UID: \"35fa6bbb-7645-4674-8a33-73ac24e99401\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"
Jul 29 15:45:49.842129 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.842100    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"
Jul 29 15:45:49.981392 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:49.981369    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"]
Jul 29 15:45:49.983565 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:45:49.983531    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35fa6bbb_7645_4674_8a33_73ac24e99401.slice/crio-d29b4c60014ab9f6c8874a6185f341da0d7d5c090f6243dfbb8a008250405832 WatchSource:0}: Error finding container d29b4c60014ab9f6c8874a6185f341da0d7d5c090f6243dfbb8a008250405832: Status 404 returned error can't find the container with id d29b4c60014ab9f6c8874a6185f341da0d7d5c090f6243dfbb8a008250405832
Jul 29 15:45:50.371214 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.371170    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"]
Jul 29 15:45:50.375773 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.375725    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"
Jul 29 15:45:50.378302 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.378282    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-1609e886\""
Jul 29 15:45:50.378302 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.378301    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-1609e886-dockercfg-bwbk8\""
Jul 29 15:45:50.383708 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.383673    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"]
Jul 29 15:45:50.446648 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.446615    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"]
Jul 29 15:45:50.450501 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.450480    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"
Jul 29 15:45:50.460879 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.460854    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"]
Jul 29 15:45:50.470309 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.470284    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7xxv\" (UniqueName: \"kubernetes.io/projected/284dd7ba-7f99-4177-96cf-2577b0c46a7d-kube-api-access-s7xxv\") pod \"payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2\" (UID: \"284dd7ba-7f99-4177-96cf-2577b0c46a7d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"
Jul 29 15:45:50.470433 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.470351    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/284dd7ba-7f99-4177-96cf-2577b0c46a7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2\" (UID: \"284dd7ba-7f99-4177-96cf-2577b0c46a7d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"
Jul 29 15:45:50.571475 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.571449    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/284dd7ba-7f99-4177-96cf-2577b0c46a7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2\" (UID: \"284dd7ba-7f99-4177-96cf-2577b0c46a7d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"
Jul 29 15:45:50.571854 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.571524    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0abbe7d6-919a-43ed-aa5b-92c92b3c7830-plugins-config-volume\") pod \"payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4\" (UID: \"0abbe7d6-919a-43ed-aa5b-92c92b3c7830\") " pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"
Jul 29 15:45:50.571854 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.571646    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mwkm\" (UniqueName: \"kubernetes.io/projected/0abbe7d6-919a-43ed-aa5b-92c92b3c7830-kube-api-access-6mwkm\") pod \"payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4\" (UID: \"0abbe7d6-919a-43ed-aa5b-92c92b3c7830\") " pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"
Jul 29 15:45:50.571854 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.571716    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7xxv\" (UniqueName: \"kubernetes.io/projected/284dd7ba-7f99-4177-96cf-2577b0c46a7d-kube-api-access-s7xxv\") pod \"payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2\" (UID: \"284dd7ba-7f99-4177-96cf-2577b0c46a7d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"
Jul 29 15:45:50.572107 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.572088    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/284dd7ba-7f99-4177-96cf-2577b0c46a7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2\" (UID: \"284dd7ba-7f99-4177-96cf-2577b0c46a7d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"
Jul 29 15:45:50.580382 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.580359    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7xxv\" (UniqueName: \"kubernetes.io/projected/284dd7ba-7f99-4177-96cf-2577b0c46a7d-kube-api-access-s7xxv\") pod \"payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2\" (UID: \"284dd7ba-7f99-4177-96cf-2577b0c46a7d\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"
Jul 29 15:45:50.672797 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.672704    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0abbe7d6-919a-43ed-aa5b-92c92b3c7830-plugins-config-volume\") pod \"payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4\" (UID: \"0abbe7d6-919a-43ed-aa5b-92c92b3c7830\") " pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"
Jul 29 15:45:50.672797 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.672781    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mwkm\" (UniqueName: \"kubernetes.io/projected/0abbe7d6-919a-43ed-aa5b-92c92b3c7830-kube-api-access-6mwkm\") pod \"payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4\" (UID: \"0abbe7d6-919a-43ed-aa5b-92c92b3c7830\") " pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"
Jul 29 15:45:50.673448 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.673429    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0abbe7d6-919a-43ed-aa5b-92c92b3c7830-plugins-config-volume\") pod \"payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4\" (UID: \"0abbe7d6-919a-43ed-aa5b-92c92b3c7830\") " pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"
Jul 29 15:45:50.682024 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.682002    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mwkm\" (UniqueName: \"kubernetes.io/projected/0abbe7d6-919a-43ed-aa5b-92c92b3c7830-kube-api-access-6mwkm\") pod \"payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4\" (UID: \"0abbe7d6-919a-43ed-aa5b-92c92b3c7830\") " pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"
Jul 29 15:45:50.687833 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.687811    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"
Jul 29 15:45:50.761577 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.761552    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"
Jul 29 15:45:50.834134 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.834056    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"]
Jul 29 15:45:50.839925 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:45:50.839895    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod284dd7ba_7f99_4177_96cf_2577b0c46a7d.slice/crio-f301ec94f7dbdca7a021c5fce20964583ce3e2c0c259b6e24555db31bf23f8c4 WatchSource:0}: Error finding container f301ec94f7dbdca7a021c5fce20964583ce3e2c0c259b6e24555db31bf23f8c4: Status 404 returned error can't find the container with id f301ec94f7dbdca7a021c5fce20964583ce3e2c0c259b6e24555db31bf23f8c4
Jul 29 15:45:50.920601 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.920531    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"]
Jul 29 15:45:50.923933 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:45:50.923405    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0abbe7d6_919a_43ed_aa5b_92c92b3c7830.slice/crio-2e284171c108f7e96ae3971caff3bc14dbaf1c6d29409cfc87aacd4c822a4f3f WatchSource:0}: Error finding container 2e284171c108f7e96ae3971caff3bc14dbaf1c6d29409cfc87aacd4c822a4f3f: Status 404 returned error can't find the container with id 2e284171c108f7e96ae3971caff3bc14dbaf1c6d29409cfc87aacd4c822a4f3f
Jul 29 15:45:50.973716 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.973680    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx" event={"ID":"35fa6bbb-7645-4674-8a33-73ac24e99401","Type":"ContainerStarted","Data":"a63f8d435374ba0a779c90308d5c8a3f3f6855e701d7a04e1fda6e4f63adc304"}
Jul 29 15:45:50.973841 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.973755    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx" event={"ID":"35fa6bbb-7645-4674-8a33-73ac24e99401","Type":"ContainerStarted","Data":"d29b4c60014ab9f6c8874a6185f341da0d7d5c090f6243dfbb8a008250405832"}
Jul 29 15:45:50.973841 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.973788    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"
Jul 29 15:45:50.975192 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.975161    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4" event={"ID":"0abbe7d6-919a-43ed-aa5b-92c92b3c7830","Type":"ContainerStarted","Data":"2e284171c108f7e96ae3971caff3bc14dbaf1c6d29409cfc87aacd4c822a4f3f"}
Jul 29 15:45:50.976523 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.976491    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2" event={"ID":"284dd7ba-7f99-4177-96cf-2577b0c46a7d","Type":"ContainerStarted","Data":"21138e1f2fc7ed990aaa2288b34a5f448c38a97bfe4b5e79a51ca62020759847"}
Jul 29 15:45:50.976610 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.976527    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2" event={"ID":"284dd7ba-7f99-4177-96cf-2577b0c46a7d","Type":"ContainerStarted","Data":"f301ec94f7dbdca7a021c5fce20964583ce3e2c0c259b6e24555db31bf23f8c4"}
Jul 29 15:45:50.976671 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.976611    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"
Jul 29 15:45:50.994125 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:50.994080    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx" podStartSLOduration=1.481549459 podStartE2EDuration="1.994065529s" totalImagesPullingTime="512.51607ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:45:50.992553335 +0000 UTC m=+2084.141847235" watchObservedRunningTime="2026-07-29 15:45:50.994065529 +0000 UTC m=+2084.143359429"
Jul 29 15:45:51.011194 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:51.011137    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2" podStartSLOduration=1.011120835 podStartE2EDuration="1.011120835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:45:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:45:51.010440708 +0000 UTC m=+2084.159734608" watchObservedRunningTime="2026-07-29 15:45:51.011120835 +0000 UTC m=+2084.160414735"
Jul 29 15:45:51.982787 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:51.982746    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4" event={"ID":"0abbe7d6-919a-43ed-aa5b-92c92b3c7830","Type":"ContainerStarted","Data":"8f5ff7145a6edbc2e4df3b9357527f1fe07395c73b2a24cdc684d2167c972913"}
Jul 29 15:45:51.983231 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:51.982906    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"
Jul 29 15:45:52.007457 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:52.007399    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4" podStartSLOduration=2.007381264 podStartE2EDuration="2.007381264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:45:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:45:52.006251501 +0000 UTC m=+2085.155545404" watchObservedRunningTime="2026-07-29 15:45:52.007381264 +0000 UTC m=+2085.156675168"
Jul 29 15:45:56.989243 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:45:56.989211    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"
Jul 29 15:46:01.984801 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:01.984766    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"
Jul 29 15:46:02.990475 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:02.990446    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"
Jul 29 15:46:07.379059 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:07.379030    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:46:07.385884 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:07.385856    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:46:07.386040 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:07.386022    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:46:07.392627 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:07.392608    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:46:20.238958 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.238904    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"]
Jul 29 15:46:20.239522 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.239194    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx" podUID="35fa6bbb-7645-4674-8a33-73ac24e99401" containerName="maas-api" containerID="cri-o://a63f8d435374ba0a779c90308d5c8a3f3f6855e701d7a04e1fda6e4f63adc304" gracePeriod=30
Jul 29 15:46:20.313842 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.313803    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"]
Jul 29 15:46:20.314093 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.314062    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4" podUID="0abbe7d6-919a-43ed-aa5b-92c92b3c7830" containerName="payload-processing" containerID="cri-o://8f5ff7145a6edbc2e4df3b9357527f1fe07395c73b2a24cdc684d2167c972913" gracePeriod=30
Jul 29 15:46:20.322163 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.322089    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"]
Jul 29 15:46:20.322666 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.322365    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2" podUID="284dd7ba-7f99-4177-96cf-2577b0c46a7d" containerName="payload-pre-processing" containerID="cri-o://21138e1f2fc7ed990aaa2288b34a5f448c38a97bfe4b5e79a51ca62020759847" gracePeriod=30
Jul 29 15:46:20.627534 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.626825    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"
Jul 29 15:46:20.633522 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.633500    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"
Jul 29 15:46:20.635154 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.635132    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/284dd7ba-7f99-4177-96cf-2577b0c46a7d-plugins-config-volume\") pod \"284dd7ba-7f99-4177-96cf-2577b0c46a7d\" (UID: \"284dd7ba-7f99-4177-96cf-2577b0c46a7d\") "
Jul 29 15:46:20.635272 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.635240    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7xxv\" (UniqueName: \"kubernetes.io/projected/284dd7ba-7f99-4177-96cf-2577b0c46a7d-kube-api-access-s7xxv\") pod \"284dd7ba-7f99-4177-96cf-2577b0c46a7d\" (UID: \"284dd7ba-7f99-4177-96cf-2577b0c46a7d\") "
Jul 29 15:46:20.635670 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.635497    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/284dd7ba-7f99-4177-96cf-2577b0c46a7d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "284dd7ba-7f99-4177-96cf-2577b0c46a7d" (UID: "284dd7ba-7f99-4177-96cf-2577b0c46a7d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:46:20.635670 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.635619    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/284dd7ba-7f99-4177-96cf-2577b0c46a7d-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:46:20.637837 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.637807    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/284dd7ba-7f99-4177-96cf-2577b0c46a7d-kube-api-access-s7xxv" (OuterVolumeSpecName: "kube-api-access-s7xxv") pod "284dd7ba-7f99-4177-96cf-2577b0c46a7d" (UID: "284dd7ba-7f99-4177-96cf-2577b0c46a7d"). InnerVolumeSpecName "kube-api-access-s7xxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:46:20.638101 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.638078    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"
Jul 29 15:46:20.736107 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.736073    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mwkm\" (UniqueName: \"kubernetes.io/projected/0abbe7d6-919a-43ed-aa5b-92c92b3c7830-kube-api-access-6mwkm\") pod \"0abbe7d6-919a-43ed-aa5b-92c92b3c7830\" (UID: \"0abbe7d6-919a-43ed-aa5b-92c92b3c7830\") "
Jul 29 15:46:20.736287 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.736117    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62qz8\" (UniqueName: \"kubernetes.io/projected/35fa6bbb-7645-4674-8a33-73ac24e99401-kube-api-access-62qz8\") pod \"35fa6bbb-7645-4674-8a33-73ac24e99401\" (UID: \"35fa6bbb-7645-4674-8a33-73ac24e99401\") "
Jul 29 15:46:20.736287 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.736186    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35fa6bbb-7645-4674-8a33-73ac24e99401-maas-api-tls\") pod \"35fa6bbb-7645-4674-8a33-73ac24e99401\" (UID: \"35fa6bbb-7645-4674-8a33-73ac24e99401\") "
Jul 29 15:46:20.736287 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.736205    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35fa6bbb-7645-4674-8a33-73ac24e99401-openshift-service-ca\") pod \"35fa6bbb-7645-4674-8a33-73ac24e99401\" (UID: \"35fa6bbb-7645-4674-8a33-73ac24e99401\") "
Jul 29 15:46:20.736287 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.736220    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0abbe7d6-919a-43ed-aa5b-92c92b3c7830-plugins-config-volume\") pod \"0abbe7d6-919a-43ed-aa5b-92c92b3c7830\" (UID: \"0abbe7d6-919a-43ed-aa5b-92c92b3c7830\") "
Jul 29 15:46:20.736581 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.736431    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7xxv\" (UniqueName: \"kubernetes.io/projected/284dd7ba-7f99-4177-96cf-2577b0c46a7d-kube-api-access-s7xxv\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:46:20.736640 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.736619    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35fa6bbb-7645-4674-8a33-73ac24e99401-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "35fa6bbb-7645-4674-8a33-73ac24e99401" (UID: "35fa6bbb-7645-4674-8a33-73ac24e99401"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:46:20.736708 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.736678    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0abbe7d6-919a-43ed-aa5b-92c92b3c7830-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0abbe7d6-919a-43ed-aa5b-92c92b3c7830" (UID: "0abbe7d6-919a-43ed-aa5b-92c92b3c7830"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:46:20.738403 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.738372    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0abbe7d6-919a-43ed-aa5b-92c92b3c7830-kube-api-access-6mwkm" (OuterVolumeSpecName: "kube-api-access-6mwkm") pod "0abbe7d6-919a-43ed-aa5b-92c92b3c7830" (UID: "0abbe7d6-919a-43ed-aa5b-92c92b3c7830"). InnerVolumeSpecName "kube-api-access-6mwkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:46:20.738489 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.738450    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35fa6bbb-7645-4674-8a33-73ac24e99401-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "35fa6bbb-7645-4674-8a33-73ac24e99401" (UID: "35fa6bbb-7645-4674-8a33-73ac24e99401"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:46:20.738489 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.738457    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35fa6bbb-7645-4674-8a33-73ac24e99401-kube-api-access-62qz8" (OuterVolumeSpecName: "kube-api-access-62qz8") pod "35fa6bbb-7645-4674-8a33-73ac24e99401" (UID: "35fa6bbb-7645-4674-8a33-73ac24e99401"). InnerVolumeSpecName "kube-api-access-62qz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:46:20.836859 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.836789    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35fa6bbb-7645-4674-8a33-73ac24e99401-maas-api-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:46:20.836859 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.836813    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35fa6bbb-7645-4674-8a33-73ac24e99401-openshift-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:46:20.836859 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.836823    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0abbe7d6-919a-43ed-aa5b-92c92b3c7830-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:46:20.836859 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.836832    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mwkm\" (UniqueName: \"kubernetes.io/projected/0abbe7d6-919a-43ed-aa5b-92c92b3c7830-kube-api-access-6mwkm\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:46:20.836859 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:20.836843    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62qz8\" (UniqueName: \"kubernetes.io/projected/35fa6bbb-7645-4674-8a33-73ac24e99401-kube-api-access-62qz8\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:46:21.131644 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.131607    2575 generic.go:356] "Generic (PLEG): container finished" podID="0abbe7d6-919a-43ed-aa5b-92c92b3c7830" containerID="8f5ff7145a6edbc2e4df3b9357527f1fe07395c73b2a24cdc684d2167c972913" exitCode=0
Jul 29 15:46:21.131840 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.131668    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"
Jul 29 15:46:21.131840 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.131692    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4" event={"ID":"0abbe7d6-919a-43ed-aa5b-92c92b3c7830","Type":"ContainerDied","Data":"8f5ff7145a6edbc2e4df3b9357527f1fe07395c73b2a24cdc684d2167c972913"}
Jul 29 15:46:21.131840 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.131745    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4" event={"ID":"0abbe7d6-919a-43ed-aa5b-92c92b3c7830","Type":"ContainerDied","Data":"2e284171c108f7e96ae3971caff3bc14dbaf1c6d29409cfc87aacd4c822a4f3f"}
Jul 29 15:46:21.131840 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.131778    2575 scope.go:117] "RemoveContainer" containerID="8f5ff7145a6edbc2e4df3b9357527f1fe07395c73b2a24cdc684d2167c972913"
Jul 29 15:46:21.133073 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.133049    2575 generic.go:356] "Generic (PLEG): container finished" podID="284dd7ba-7f99-4177-96cf-2577b0c46a7d" containerID="21138e1f2fc7ed990aaa2288b34a5f448c38a97bfe4b5e79a51ca62020759847" exitCode=0
Jul 29 15:46:21.133177 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.133101    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"
Jul 29 15:46:21.133177 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.133135    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2" event={"ID":"284dd7ba-7f99-4177-96cf-2577b0c46a7d","Type":"ContainerDied","Data":"21138e1f2fc7ed990aaa2288b34a5f448c38a97bfe4b5e79a51ca62020759847"}
Jul 29 15:46:21.133287 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.133183    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2" event={"ID":"284dd7ba-7f99-4177-96cf-2577b0c46a7d","Type":"ContainerDied","Data":"f301ec94f7dbdca7a021c5fce20964583ce3e2c0c259b6e24555db31bf23f8c4"}
Jul 29 15:46:21.134718 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.134646    2575 generic.go:356] "Generic (PLEG): container finished" podID="35fa6bbb-7645-4674-8a33-73ac24e99401" containerID="a63f8d435374ba0a779c90308d5c8a3f3f6855e701d7a04e1fda6e4f63adc304" exitCode=0
Jul 29 15:46:21.134718 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.134677    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx" event={"ID":"35fa6bbb-7645-4674-8a33-73ac24e99401","Type":"ContainerDied","Data":"a63f8d435374ba0a779c90308d5c8a3f3f6855e701d7a04e1fda6e4f63adc304"}
Jul 29 15:46:21.134902 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.134727    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx" event={"ID":"35fa6bbb-7645-4674-8a33-73ac24e99401","Type":"ContainerDied","Data":"d29b4c60014ab9f6c8874a6185f341da0d7d5c090f6243dfbb8a008250405832"}
Jul 29 15:46:21.134902 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.134685    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"
Jul 29 15:46:21.143032 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.143009    2575 scope.go:117] "RemoveContainer" containerID="8f5ff7145a6edbc2e4df3b9357527f1fe07395c73b2a24cdc684d2167c972913"
Jul 29 15:46:21.143420 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:46:21.143383    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f5ff7145a6edbc2e4df3b9357527f1fe07395c73b2a24cdc684d2167c972913\": container with ID starting with 8f5ff7145a6edbc2e4df3b9357527f1fe07395c73b2a24cdc684d2167c972913 not found: ID does not exist" containerID="8f5ff7145a6edbc2e4df3b9357527f1fe07395c73b2a24cdc684d2167c972913"
Jul 29 15:46:21.143517 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.143432    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f5ff7145a6edbc2e4df3b9357527f1fe07395c73b2a24cdc684d2167c972913"} err="failed to get container status \"8f5ff7145a6edbc2e4df3b9357527f1fe07395c73b2a24cdc684d2167c972913\": rpc error: code = NotFound desc = could not find container \"8f5ff7145a6edbc2e4df3b9357527f1fe07395c73b2a24cdc684d2167c972913\": container with ID starting with 8f5ff7145a6edbc2e4df3b9357527f1fe07395c73b2a24cdc684d2167c972913 not found: ID does not exist"
Jul 29 15:46:21.143517 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.143457    2575 scope.go:117] "RemoveContainer" containerID="21138e1f2fc7ed990aaa2288b34a5f448c38a97bfe4b5e79a51ca62020759847"
Jul 29 15:46:21.157969 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.157943    2575 scope.go:117] "RemoveContainer" containerID="21138e1f2fc7ed990aaa2288b34a5f448c38a97bfe4b5e79a51ca62020759847"
Jul 29 15:46:21.158202 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:46:21.158184    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21138e1f2fc7ed990aaa2288b34a5f448c38a97bfe4b5e79a51ca62020759847\": container with ID starting with 21138e1f2fc7ed990aaa2288b34a5f448c38a97bfe4b5e79a51ca62020759847 not found: ID does not exist" containerID="21138e1f2fc7ed990aaa2288b34a5f448c38a97bfe4b5e79a51ca62020759847"
Jul 29 15:46:21.158247 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.158209    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21138e1f2fc7ed990aaa2288b34a5f448c38a97bfe4b5e79a51ca62020759847"} err="failed to get container status \"21138e1f2fc7ed990aaa2288b34a5f448c38a97bfe4b5e79a51ca62020759847\": rpc error: code = NotFound desc = could not find container \"21138e1f2fc7ed990aaa2288b34a5f448c38a97bfe4b5e79a51ca62020759847\": container with ID starting with 21138e1f2fc7ed990aaa2288b34a5f448c38a97bfe4b5e79a51ca62020759847 not found: ID does not exist"
Jul 29 15:46:21.158247 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.158226    2575 scope.go:117] "RemoveContainer" containerID="a63f8d435374ba0a779c90308d5c8a3f3f6855e701d7a04e1fda6e4f63adc304"
Jul 29 15:46:21.160999 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.160979    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"]
Jul 29 15:46:21.167203 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.167182    2575 scope.go:117] "RemoveContainer" containerID="a63f8d435374ba0a779c90308d5c8a3f3f6855e701d7a04e1fda6e4f63adc304"
Jul 29 15:46:21.167474 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:46:21.167458    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a63f8d435374ba0a779c90308d5c8a3f3f6855e701d7a04e1fda6e4f63adc304\": container with ID starting with a63f8d435374ba0a779c90308d5c8a3f3f6855e701d7a04e1fda6e4f63adc304 not found: ID does not exist" containerID="a63f8d435374ba0a779c90308d5c8a3f3f6855e701d7a04e1fda6e4f63adc304"
Jul 29 15:46:21.167523 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.167481    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a63f8d435374ba0a779c90308d5c8a3f3f6855e701d7a04e1fda6e4f63adc304"} err="failed to get container status \"a63f8d435374ba0a779c90308d5c8a3f3f6855e701d7a04e1fda6e4f63adc304\": rpc error: code = NotFound desc = could not find container \"a63f8d435374ba0a779c90308d5c8a3f3f6855e701d7a04e1fda6e4f63adc304\": container with ID starting with a63f8d435374ba0a779c90308d5c8a3f3f6855e701d7a04e1fda6e4f63adc304 not found: ID does not exist"
Jul 29 15:46:21.167856 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.167838    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-1609e886-85b46cf9b6-4gkz4"]
Jul 29 15:46:21.178955 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.178933    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"]
Jul 29 15:46:21.184864 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.184837    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-1609e886-659bc68d8d-95wk2"]
Jul 29 15:46:21.199701 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.199677    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"]
Jul 29 15:46:21.205020 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.204997    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-1609e886-596d5bb6b5-r2btx"]
Jul 29 15:46:21.295424 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.295397    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0abbe7d6-919a-43ed-aa5b-92c92b3c7830" path="/var/lib/kubelet/pods/0abbe7d6-919a-43ed-aa5b-92c92b3c7830/volumes"
Jul 29 15:46:21.295948 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.295924    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="284dd7ba-7f99-4177-96cf-2577b0c46a7d" path="/var/lib/kubelet/pods/284dd7ba-7f99-4177-96cf-2577b0c46a7d/volumes"
Jul 29 15:46:21.296353 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:46:21.296335    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35fa6bbb-7645-4674-8a33-73ac24e99401" path="/var/lib/kubelet/pods/35fa6bbb-7645-4674-8a33-73ac24e99401/volumes"
Jul 29 15:50:31.893491 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:31.893456    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"]
Jul 29 15:50:31.896406 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:31.894524    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0abbe7d6-919a-43ed-aa5b-92c92b3c7830" containerName="payload-processing"
Jul 29 15:50:31.896406 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:31.894547    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0abbe7d6-919a-43ed-aa5b-92c92b3c7830" containerName="payload-processing"
Jul 29 15:50:31.896406 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:31.894587    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="284dd7ba-7f99-4177-96cf-2577b0c46a7d" containerName="payload-pre-processing"
Jul 29 15:50:31.896406 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:31.894597    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="284dd7ba-7f99-4177-96cf-2577b0c46a7d" containerName="payload-pre-processing"
Jul 29 15:50:31.896406 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:31.894612    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35fa6bbb-7645-4674-8a33-73ac24e99401" containerName="maas-api"
Jul 29 15:50:31.896406 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:31.894620    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="35fa6bbb-7645-4674-8a33-73ac24e99401" containerName="maas-api"
Jul 29 15:50:31.896406 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:31.894714    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="284dd7ba-7f99-4177-96cf-2577b0c46a7d" containerName="payload-pre-processing"
Jul 29 15:50:31.896406 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:31.894728    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0abbe7d6-919a-43ed-aa5b-92c92b3c7830" containerName="payload-processing"
Jul 29 15:50:31.896406 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:31.894740    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="35fa6bbb-7645-4674-8a33-73ac24e99401" containerName="maas-api"
Jul 29 15:50:31.898273 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:31.898253    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"
Jul 29 15:50:31.900343 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:31.900288    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f2a43d0f-serving-cert\""
Jul 29 15:50:31.907004 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:31.906979    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"]
Jul 29 15:50:32.011107 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.011074    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c0598b0-76a6-4461-9fe1-400990d25d94-openshift-service-ca\") pod \"maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq\" (UID: \"7c0598b0-76a6-4461-9fe1-400990d25d94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"
Jul 29 15:50:32.011283 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.011113    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c0598b0-76a6-4461-9fe1-400990d25d94-maas-api-tls\") pod \"maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq\" (UID: \"7c0598b0-76a6-4461-9fe1-400990d25d94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"
Jul 29 15:50:32.011283 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.011158    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vzzr\" (UniqueName: \"kubernetes.io/projected/7c0598b0-76a6-4461-9fe1-400990d25d94-kube-api-access-6vzzr\") pod \"maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq\" (UID: \"7c0598b0-76a6-4461-9fe1-400990d25d94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"
Jul 29 15:50:32.112604 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.112567    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c0598b0-76a6-4461-9fe1-400990d25d94-openshift-service-ca\") pod \"maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq\" (UID: \"7c0598b0-76a6-4461-9fe1-400990d25d94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"
Jul 29 15:50:32.112748 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.112611    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c0598b0-76a6-4461-9fe1-400990d25d94-maas-api-tls\") pod \"maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq\" (UID: \"7c0598b0-76a6-4461-9fe1-400990d25d94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"
Jul 29 15:50:32.112748 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.112648    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vzzr\" (UniqueName: \"kubernetes.io/projected/7c0598b0-76a6-4461-9fe1-400990d25d94-kube-api-access-6vzzr\") pod \"maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq\" (UID: \"7c0598b0-76a6-4461-9fe1-400990d25d94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"
Jul 29 15:50:32.112825 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:50:32.112778    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-serving-cert: secret "maas-api-e2e-mt-f2a43d0f-serving-cert" not found
Jul 29 15:50:32.112859 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:50:32.112838    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c0598b0-76a6-4461-9fe1-400990d25d94-maas-api-tls podName:7c0598b0-76a6-4461-9fe1-400990d25d94 nodeName:}" failed. No retries permitted until 2026-07-29 15:50:32.612821501 +0000 UTC m=+2365.762115378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7c0598b0-76a6-4461-9fe1-400990d25d94-maas-api-tls") pod "maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq" (UID: "7c0598b0-76a6-4461-9fe1-400990d25d94") : secret "maas-api-e2e-mt-f2a43d0f-serving-cert" not found
Jul 29 15:50:32.113175 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.113157    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c0598b0-76a6-4461-9fe1-400990d25d94-openshift-service-ca\") pod \"maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq\" (UID: \"7c0598b0-76a6-4461-9fe1-400990d25d94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"
Jul 29 15:50:32.121972 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.121948    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vzzr\" (UniqueName: \"kubernetes.io/projected/7c0598b0-76a6-4461-9fe1-400990d25d94-kube-api-access-6vzzr\") pod \"maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq\" (UID: \"7c0598b0-76a6-4461-9fe1-400990d25d94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"
Jul 29 15:50:32.616733 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.616698    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c0598b0-76a6-4461-9fe1-400990d25d94-maas-api-tls\") pod \"maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq\" (UID: \"7c0598b0-76a6-4461-9fe1-400990d25d94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"
Jul 29 15:50:32.619246 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.619223    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c0598b0-76a6-4461-9fe1-400990d25d94-maas-api-tls\") pod \"maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq\" (UID: \"7c0598b0-76a6-4461-9fe1-400990d25d94\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"
Jul 29 15:50:32.810347 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.810292    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"
Jul 29 15:50:32.816145 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.816120    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"]
Jul 29 15:50:32.829247 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.828607    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"
Jul 29 15:50:32.831836 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.831771    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"]
Jul 29 15:50:32.833263 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.833244    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-f2a43d0f\""
Jul 29 15:50:32.833600 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.833578    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-f2a43d0f-dockercfg-2264r\""
Jul 29 15:50:32.894517 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.894488    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"]
Jul 29 15:50:32.898175 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.898153    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"
Jul 29 15:50:32.908836 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.908599    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"]
Jul 29 15:50:32.961714 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.961690    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"]
Jul 29 15:50:32.964248 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:50:32.964197    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c0598b0_76a6_4461_9fe1_400990d25d94.slice/crio-0da30ce621b1eea3fa5742ebd24a33348dd37965fe7197893d2e2eda3cd5074e WatchSource:0}: Error finding container 0da30ce621b1eea3fa5742ebd24a33348dd37965fe7197893d2e2eda3cd5074e: Status 404 returned error can't find the container with id 0da30ce621b1eea3fa5742ebd24a33348dd37965fe7197893d2e2eda3cd5074e
Jul 29 15:50:32.965559 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:32.965542    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 15:50:33.023647 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:33.023597    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b09c3712-b3be-49bc-bea0-6a1cc4e05e99-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk\" (UID: \"b09c3712-b3be-49bc-bea0-6a1cc4e05e99\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"
Jul 29 15:50:33.023647 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:33.023647    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ff42290-6b5b-4bf4-b07f-0b87c3d486f4-plugins-config-volume\") pod \"payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z\" (UID: \"5ff42290-6b5b-4bf4-b07f-0b87c3d486f4\") " pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"
Jul 29 15:50:33.023884 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:33.023695    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6grg\" (UniqueName: \"kubernetes.io/projected/5ff42290-6b5b-4bf4-b07f-0b87c3d486f4-kube-api-access-j6grg\") pod \"payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z\" (UID: \"5ff42290-6b5b-4bf4-b07f-0b87c3d486f4\") " pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"
Jul 29 15:50:33.023884 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:33.023728    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zdtq\" (UniqueName: \"kubernetes.io/projected/b09c3712-b3be-49bc-bea0-6a1cc4e05e99-kube-api-access-7zdtq\") pod \"payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk\" (UID: \"b09c3712-b3be-49bc-bea0-6a1cc4e05e99\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"
Jul 29 15:50:33.124880 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:33.124836    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6grg\" (UniqueName: \"kubernetes.io/projected/5ff42290-6b5b-4bf4-b07f-0b87c3d486f4-kube-api-access-j6grg\") pod \"payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z\" (UID: \"5ff42290-6b5b-4bf4-b07f-0b87c3d486f4\") " pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"
Jul 29 15:50:33.125111 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:33.124891    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zdtq\" (UniqueName: \"kubernetes.io/projected/b09c3712-b3be-49bc-bea0-6a1cc4e05e99-kube-api-access-7zdtq\") pod \"payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk\" (UID: \"b09c3712-b3be-49bc-bea0-6a1cc4e05e99\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"
Jul 29 15:50:33.125111 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:33.125034    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b09c3712-b3be-49bc-bea0-6a1cc4e05e99-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk\" (UID: \"b09c3712-b3be-49bc-bea0-6a1cc4e05e99\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"
Jul 29 15:50:33.125111 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:33.125060    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ff42290-6b5b-4bf4-b07f-0b87c3d486f4-plugins-config-volume\") pod \"payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z\" (UID: \"5ff42290-6b5b-4bf4-b07f-0b87c3d486f4\") " pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"
Jul 29 15:50:33.125830 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:33.125797    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b09c3712-b3be-49bc-bea0-6a1cc4e05e99-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk\" (UID: \"b09c3712-b3be-49bc-bea0-6a1cc4e05e99\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"
Jul 29 15:50:33.125969 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:33.125911    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ff42290-6b5b-4bf4-b07f-0b87c3d486f4-plugins-config-volume\") pod \"payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z\" (UID: \"5ff42290-6b5b-4bf4-b07f-0b87c3d486f4\") " pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"
Jul 29 15:50:33.133708 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:33.133687    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zdtq\" (UniqueName: \"kubernetes.io/projected/b09c3712-b3be-49bc-bea0-6a1cc4e05e99-kube-api-access-7zdtq\") pod \"payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk\" (UID: \"b09c3712-b3be-49bc-bea0-6a1cc4e05e99\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"
Jul 29 15:50:33.133816 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:33.133771    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6grg\" (UniqueName: \"kubernetes.io/projected/5ff42290-6b5b-4bf4-b07f-0b87c3d486f4-kube-api-access-j6grg\") pod \"payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z\" (UID: \"5ff42290-6b5b-4bf4-b07f-0b87c3d486f4\") " pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"
Jul 29 15:50:33.149517 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:33.149492    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"
Jul 29 15:50:33.209955 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:33.209922    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"
Jul 29 15:50:33.316310 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:33.316274    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq" event={"ID":"7c0598b0-76a6-4461-9fe1-400990d25d94","Type":"ContainerStarted","Data":"0da30ce621b1eea3fa5742ebd24a33348dd37965fe7197893d2e2eda3cd5074e"}
Jul 29 15:50:33.352680 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:33.352528    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"]
Jul 29 15:50:33.354735 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:50:33.354712    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ff42290_6b5b_4bf4_b07f_0b87c3d486f4.slice/crio-1e90bb15be3ab7f59d1f658665e002e56472f15f78d168e6411aae10b5f2f6cd WatchSource:0}: Error finding container 1e90bb15be3ab7f59d1f658665e002e56472f15f78d168e6411aae10b5f2f6cd: Status 404 returned error can't find the container with id 1e90bb15be3ab7f59d1f658665e002e56472f15f78d168e6411aae10b5f2f6cd
Jul 29 15:50:33.496362 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:33.496313    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"]
Jul 29 15:50:33.498834 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:50:33.498811    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb09c3712_b3be_49bc_bea0_6a1cc4e05e99.slice/crio-09c2ab8254848acfcff082e577308788bd2bdf9298f1c297444182b3deee272a WatchSource:0}: Error finding container 09c2ab8254848acfcff082e577308788bd2bdf9298f1c297444182b3deee272a: Status 404 returned error can't find the container with id 09c2ab8254848acfcff082e577308788bd2bdf9298f1c297444182b3deee272a
Jul 29 15:50:34.323583 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:34.323546    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq" event={"ID":"7c0598b0-76a6-4461-9fe1-400990d25d94","Type":"ContainerStarted","Data":"dd6fdc85a9d27c2d6cf1b7a9799ad317951e98a0c16827258f464d6b871b1d07"}
Jul 29 15:50:34.324071 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:34.323691    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"
Jul 29 15:50:34.325374 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:34.325346    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk" event={"ID":"b09c3712-b3be-49bc-bea0-6a1cc4e05e99","Type":"ContainerStarted","Data":"77945f36998839393046699695225e4585e9bbb4d050bb33bd968f519629a183"}
Jul 29 15:50:34.325525 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:34.325393    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk" event={"ID":"b09c3712-b3be-49bc-bea0-6a1cc4e05e99","Type":"ContainerStarted","Data":"09c2ab8254848acfcff082e577308788bd2bdf9298f1c297444182b3deee272a"}
Jul 29 15:50:34.325525 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:34.325445    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"
Jul 29 15:50:34.326910 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:34.326884    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z" event={"ID":"5ff42290-6b5b-4bf4-b07f-0b87c3d486f4","Type":"ContainerStarted","Data":"0054d4da830aa7c8770db8711c467cc57e61eb6636a0abcaf8ef073c6f7947ca"}
Jul 29 15:50:34.326996 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:34.326925    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z" event={"ID":"5ff42290-6b5b-4bf4-b07f-0b87c3d486f4","Type":"ContainerStarted","Data":"1e90bb15be3ab7f59d1f658665e002e56472f15f78d168e6411aae10b5f2f6cd"}
Jul 29 15:50:34.327085 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:34.327026    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"
Jul 29 15:50:34.343020 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:34.342971    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq" podStartSLOduration=2.92022379 podStartE2EDuration="3.342954816s" totalImagesPullingTime="422.731026ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:50:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:50:34.340043622 +0000 UTC m=+2367.489337518" watchObservedRunningTime="2026-07-29 15:50:34.342954816 +0000 UTC m=+2367.492248718"
Jul 29 15:50:34.357455 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:34.357413    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z" podStartSLOduration=2.357399156 podStartE2EDuration="2.357399156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:50:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:50:34.355760657 +0000 UTC m=+2367.505054558" watchObservedRunningTime="2026-07-29 15:50:34.357399156 +0000 UTC m=+2367.506693051"
Jul 29 15:50:40.337235 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:40.337200    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"
Jul 29 15:50:40.360147 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:40.360098    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk" podStartSLOduration=8.360083458 podStartE2EDuration="8.360083458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:50:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:50:34.371331833 +0000 UTC m=+2367.520625724" watchObservedRunningTime="2026-07-29 15:50:40.360083458 +0000 UTC m=+2373.509377357"
Jul 29 15:50:45.333055 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:45.333028    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"
Jul 29 15:50:45.333479 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:50:45.333088    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"
Jul 29 15:51:07.406765 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:07.406732    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:51:07.413233 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:07.413205    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:51:07.415085 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:07.415061    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:51:07.421960 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:07.421944    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:51:15.064423 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.064376    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"]
Jul 29 15:51:15.065179 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.065135    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq" podUID="7c0598b0-76a6-4461-9fe1-400990d25d94" containerName="maas-api" containerID="cri-o://dd6fdc85a9d27c2d6cf1b7a9799ad317951e98a0c16827258f464d6b871b1d07" gracePeriod=30
Jul 29 15:51:15.131942 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.131902    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"]
Jul 29 15:51:15.132182 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.132155    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z" podUID="5ff42290-6b5b-4bf4-b07f-0b87c3d486f4" containerName="payload-processing" containerID="cri-o://0054d4da830aa7c8770db8711c467cc57e61eb6636a0abcaf8ef073c6f7947ca" gracePeriod=30
Jul 29 15:51:15.138150 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.138120    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"]
Jul 29 15:51:15.138383 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.138359    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk" podUID="b09c3712-b3be-49bc-bea0-6a1cc4e05e99" containerName="payload-pre-processing" containerID="cri-o://77945f36998839393046699695225e4585e9bbb4d050bb33bd968f519629a183" gracePeriod=30
Jul 29 15:51:15.331942 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.331816    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk" podUID="b09c3712-b3be-49bc-bea0-6a1cc4e05e99" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.59:9004: connect: connection refused"
Jul 29 15:51:15.331942 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.331833    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq" podUID="7c0598b0-76a6-4461-9fe1-400990d25d94" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.58:8443/health\": dial tcp 10.133.0.58:8443: connect: connection refused"
Jul 29 15:51:15.385279 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.385253    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"
Jul 29 15:51:15.413937 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.413908    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"
Jul 29 15:51:15.426610 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.426585    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"
Jul 29 15:51:15.505763 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.505734    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vzzr\" (UniqueName: \"kubernetes.io/projected/7c0598b0-76a6-4461-9fe1-400990d25d94-kube-api-access-6vzzr\") pod \"7c0598b0-76a6-4461-9fe1-400990d25d94\" (UID: \"7c0598b0-76a6-4461-9fe1-400990d25d94\") "
Jul 29 15:51:15.505927 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.505826    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ff42290-6b5b-4bf4-b07f-0b87c3d486f4-plugins-config-volume\") pod \"5ff42290-6b5b-4bf4-b07f-0b87c3d486f4\" (UID: \"5ff42290-6b5b-4bf4-b07f-0b87c3d486f4\") "
Jul 29 15:51:15.505927 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.505857    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c0598b0-76a6-4461-9fe1-400990d25d94-maas-api-tls\") pod \"7c0598b0-76a6-4461-9fe1-400990d25d94\" (UID: \"7c0598b0-76a6-4461-9fe1-400990d25d94\") "
Jul 29 15:51:15.505927 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.505889    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c0598b0-76a6-4461-9fe1-400990d25d94-openshift-service-ca\") pod \"7c0598b0-76a6-4461-9fe1-400990d25d94\" (UID: \"7c0598b0-76a6-4461-9fe1-400990d25d94\") "
Jul 29 15:51:15.505927 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.505915    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6grg\" (UniqueName: \"kubernetes.io/projected/5ff42290-6b5b-4bf4-b07f-0b87c3d486f4-kube-api-access-j6grg\") pod \"5ff42290-6b5b-4bf4-b07f-0b87c3d486f4\" (UID: \"5ff42290-6b5b-4bf4-b07f-0b87c3d486f4\") "
Jul 29 15:51:15.506252 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.506227    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ff42290-6b5b-4bf4-b07f-0b87c3d486f4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5ff42290-6b5b-4bf4-b07f-0b87c3d486f4" (UID: "5ff42290-6b5b-4bf4-b07f-0b87c3d486f4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:51:15.506252 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.506240    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c0598b0-76a6-4461-9fe1-400990d25d94-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7c0598b0-76a6-4461-9fe1-400990d25d94" (UID: "7c0598b0-76a6-4461-9fe1-400990d25d94"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:51:15.507779 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.507753    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c0598b0-76a6-4461-9fe1-400990d25d94-kube-api-access-6vzzr" (OuterVolumeSpecName: "kube-api-access-6vzzr") pod "7c0598b0-76a6-4461-9fe1-400990d25d94" (UID: "7c0598b0-76a6-4461-9fe1-400990d25d94"). InnerVolumeSpecName "kube-api-access-6vzzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:51:15.508043 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.508021    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ff42290-6b5b-4bf4-b07f-0b87c3d486f4-kube-api-access-j6grg" (OuterVolumeSpecName: "kube-api-access-j6grg") pod "5ff42290-6b5b-4bf4-b07f-0b87c3d486f4" (UID: "5ff42290-6b5b-4bf4-b07f-0b87c3d486f4"). InnerVolumeSpecName "kube-api-access-j6grg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:51:15.508157 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.508139    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c0598b0-76a6-4461-9fe1-400990d25d94-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7c0598b0-76a6-4461-9fe1-400990d25d94" (UID: "7c0598b0-76a6-4461-9fe1-400990d25d94"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:51:15.525125 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.525102    2575 generic.go:356] "Generic (PLEG): container finished" podID="7c0598b0-76a6-4461-9fe1-400990d25d94" containerID="dd6fdc85a9d27c2d6cf1b7a9799ad317951e98a0c16827258f464d6b871b1d07" exitCode=0
Jul 29 15:51:15.525231 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.525147    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"
Jul 29 15:51:15.525231 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.525181    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq" event={"ID":"7c0598b0-76a6-4461-9fe1-400990d25d94","Type":"ContainerDied","Data":"dd6fdc85a9d27c2d6cf1b7a9799ad317951e98a0c16827258f464d6b871b1d07"}
Jul 29 15:51:15.525361 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.525238    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq" event={"ID":"7c0598b0-76a6-4461-9fe1-400990d25d94","Type":"ContainerDied","Data":"0da30ce621b1eea3fa5742ebd24a33348dd37965fe7197893d2e2eda3cd5074e"}
Jul 29 15:51:15.525361 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.525270    2575 scope.go:117] "RemoveContainer" containerID="dd6fdc85a9d27c2d6cf1b7a9799ad317951e98a0c16827258f464d6b871b1d07"
Jul 29 15:51:15.526523 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.526503    2575 generic.go:356] "Generic (PLEG): container finished" podID="b09c3712-b3be-49bc-bea0-6a1cc4e05e99" containerID="77945f36998839393046699695225e4585e9bbb4d050bb33bd968f519629a183" exitCode=0
Jul 29 15:51:15.526622 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.526556    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"
Jul 29 15:51:15.526622 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.526614    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk" event={"ID":"b09c3712-b3be-49bc-bea0-6a1cc4e05e99","Type":"ContainerDied","Data":"77945f36998839393046699695225e4585e9bbb4d050bb33bd968f519629a183"}
Jul 29 15:51:15.526731 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.526654    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk" event={"ID":"b09c3712-b3be-49bc-bea0-6a1cc4e05e99","Type":"ContainerDied","Data":"09c2ab8254848acfcff082e577308788bd2bdf9298f1c297444182b3deee272a"}
Jul 29 15:51:15.528005 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.527980    2575 generic.go:356] "Generic (PLEG): container finished" podID="5ff42290-6b5b-4bf4-b07f-0b87c3d486f4" containerID="0054d4da830aa7c8770db8711c467cc57e61eb6636a0abcaf8ef073c6f7947ca" exitCode=0
Jul 29 15:51:15.528107 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.528040    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"
Jul 29 15:51:15.528107 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.528050    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z" event={"ID":"5ff42290-6b5b-4bf4-b07f-0b87c3d486f4","Type":"ContainerDied","Data":"0054d4da830aa7c8770db8711c467cc57e61eb6636a0abcaf8ef073c6f7947ca"}
Jul 29 15:51:15.528107 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.528087    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z" event={"ID":"5ff42290-6b5b-4bf4-b07f-0b87c3d486f4","Type":"ContainerDied","Data":"1e90bb15be3ab7f59d1f658665e002e56472f15f78d168e6411aae10b5f2f6cd"}
Jul 29 15:51:15.535401 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.535374    2575 scope.go:117] "RemoveContainer" containerID="dd6fdc85a9d27c2d6cf1b7a9799ad317951e98a0c16827258f464d6b871b1d07"
Jul 29 15:51:15.535699 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:51:15.535674    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd6fdc85a9d27c2d6cf1b7a9799ad317951e98a0c16827258f464d6b871b1d07\": container with ID starting with dd6fdc85a9d27c2d6cf1b7a9799ad317951e98a0c16827258f464d6b871b1d07 not found: ID does not exist" containerID="dd6fdc85a9d27c2d6cf1b7a9799ad317951e98a0c16827258f464d6b871b1d07"
Jul 29 15:51:15.535764 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.535711    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd6fdc85a9d27c2d6cf1b7a9799ad317951e98a0c16827258f464d6b871b1d07"} err="failed to get container status \"dd6fdc85a9d27c2d6cf1b7a9799ad317951e98a0c16827258f464d6b871b1d07\": rpc error: code = NotFound desc = could not find container \"dd6fdc85a9d27c2d6cf1b7a9799ad317951e98a0c16827258f464d6b871b1d07\": container with ID starting with dd6fdc85a9d27c2d6cf1b7a9799ad317951e98a0c16827258f464d6b871b1d07 not found: ID does not exist"
Jul 29 15:51:15.535764 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.535733    2575 scope.go:117] "RemoveContainer" containerID="77945f36998839393046699695225e4585e9bbb4d050bb33bd968f519629a183"
Jul 29 15:51:15.543160 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.543146    2575 scope.go:117] "RemoveContainer" containerID="77945f36998839393046699695225e4585e9bbb4d050bb33bd968f519629a183"
Jul 29 15:51:15.543385 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:51:15.543368    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77945f36998839393046699695225e4585e9bbb4d050bb33bd968f519629a183\": container with ID starting with 77945f36998839393046699695225e4585e9bbb4d050bb33bd968f519629a183 not found: ID does not exist" containerID="77945f36998839393046699695225e4585e9bbb4d050bb33bd968f519629a183"
Jul 29 15:51:15.543438 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.543391    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77945f36998839393046699695225e4585e9bbb4d050bb33bd968f519629a183"} err="failed to get container status \"77945f36998839393046699695225e4585e9bbb4d050bb33bd968f519629a183\": rpc error: code = NotFound desc = could not find container \"77945f36998839393046699695225e4585e9bbb4d050bb33bd968f519629a183\": container with ID starting with 77945f36998839393046699695225e4585e9bbb4d050bb33bd968f519629a183 not found: ID does not exist"
Jul 29 15:51:15.543438 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.543406    2575 scope.go:117] "RemoveContainer" containerID="0054d4da830aa7c8770db8711c467cc57e61eb6636a0abcaf8ef073c6f7947ca"
Jul 29 15:51:15.550721 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.550704    2575 scope.go:117] "RemoveContainer" containerID="0054d4da830aa7c8770db8711c467cc57e61eb6636a0abcaf8ef073c6f7947ca"
Jul 29 15:51:15.550963 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:51:15.550943    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0054d4da830aa7c8770db8711c467cc57e61eb6636a0abcaf8ef073c6f7947ca\": container with ID starting with 0054d4da830aa7c8770db8711c467cc57e61eb6636a0abcaf8ef073c6f7947ca not found: ID does not exist" containerID="0054d4da830aa7c8770db8711c467cc57e61eb6636a0abcaf8ef073c6f7947ca"
Jul 29 15:51:15.551031 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.550973    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0054d4da830aa7c8770db8711c467cc57e61eb6636a0abcaf8ef073c6f7947ca"} err="failed to get container status \"0054d4da830aa7c8770db8711c467cc57e61eb6636a0abcaf8ef073c6f7947ca\": rpc error: code = NotFound desc = could not find container \"0054d4da830aa7c8770db8711c467cc57e61eb6636a0abcaf8ef073c6f7947ca\": container with ID starting with 0054d4da830aa7c8770db8711c467cc57e61eb6636a0abcaf8ef073c6f7947ca not found: ID does not exist"
Jul 29 15:51:15.551882 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.551864    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"]
Jul 29 15:51:15.556481 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.556461    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f2a43d0f-b6d4df6d-646cq"]
Jul 29 15:51:15.568195 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.568171    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"]
Jul 29 15:51:15.573091 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.573070    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z"]
Jul 29 15:51:15.606882 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.606858    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zdtq\" (UniqueName: \"kubernetes.io/projected/b09c3712-b3be-49bc-bea0-6a1cc4e05e99-kube-api-access-7zdtq\") pod \"b09c3712-b3be-49bc-bea0-6a1cc4e05e99\" (UID: \"b09c3712-b3be-49bc-bea0-6a1cc4e05e99\") "
Jul 29 15:51:15.606986 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.606916    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b09c3712-b3be-49bc-bea0-6a1cc4e05e99-plugins-config-volume\") pod \"b09c3712-b3be-49bc-bea0-6a1cc4e05e99\" (UID: \"b09c3712-b3be-49bc-bea0-6a1cc4e05e99\") "
Jul 29 15:51:15.607138 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.607123    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ff42290-6b5b-4bf4-b07f-0b87c3d486f4-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:51:15.607208 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.607144    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c0598b0-76a6-4461-9fe1-400990d25d94-maas-api-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:51:15.607208 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.607159    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c0598b0-76a6-4461-9fe1-400990d25d94-openshift-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:51:15.607208 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.607181    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6grg\" (UniqueName: \"kubernetes.io/projected/5ff42290-6b5b-4bf4-b07f-0b87c3d486f4-kube-api-access-j6grg\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:51:15.607208 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.607195    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vzzr\" (UniqueName: \"kubernetes.io/projected/7c0598b0-76a6-4461-9fe1-400990d25d94-kube-api-access-6vzzr\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:51:15.607372 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.607277    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b09c3712-b3be-49bc-bea0-6a1cc4e05e99-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b09c3712-b3be-49bc-bea0-6a1cc4e05e99" (UID: "b09c3712-b3be-49bc-bea0-6a1cc4e05e99"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:51:15.608721 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.608704    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b09c3712-b3be-49bc-bea0-6a1cc4e05e99-kube-api-access-7zdtq" (OuterVolumeSpecName: "kube-api-access-7zdtq") pod "b09c3712-b3be-49bc-bea0-6a1cc4e05e99" (UID: "b09c3712-b3be-49bc-bea0-6a1cc4e05e99"). InnerVolumeSpecName "kube-api-access-7zdtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:51:15.708385 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.708357    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zdtq\" (UniqueName: \"kubernetes.io/projected/b09c3712-b3be-49bc-bea0-6a1cc4e05e99-kube-api-access-7zdtq\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:51:15.708385 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.708381    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b09c3712-b3be-49bc-bea0-6a1cc4e05e99-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:51:15.852243 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.852213    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"]
Jul 29 15:51:15.856839 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:15.856815    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f2a43d0f-64c67d6c5b-vwbtk"]
Jul 29 15:51:16.332746 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:16.332657    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-f2a43d0f-5d4b7b8bcd-ppq6z" podUID="5ff42290-6b5b-4bf4-b07f-0b87c3d486f4" containerName="payload-processing" probeResult="failure" output="dial tcp 10.133.0.60:9004: i/o timeout"
Jul 29 15:51:17.295099 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:17.295067    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ff42290-6b5b-4bf4-b07f-0b87c3d486f4" path="/var/lib/kubelet/pods/5ff42290-6b5b-4bf4-b07f-0b87c3d486f4/volumes"
Jul 29 15:51:17.295571 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:17.295551    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c0598b0-76a6-4461-9fe1-400990d25d94" path="/var/lib/kubelet/pods/7c0598b0-76a6-4461-9fe1-400990d25d94/volumes"
Jul 29 15:51:17.295898 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:51:17.295886    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b09c3712-b3be-49bc-bea0-6a1cc4e05e99" path="/var/lib/kubelet/pods/b09c3712-b3be-49bc-bea0-6a1cc4e05e99/volumes"
Jul 29 15:52:04.827837 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:04.827801    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"]
Jul 29 15:52:04.828531 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:04.827992    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp" podUID="04fcbf32-55b8-4dee-9078-8d29e6119ad7" containerName="manager" containerID="cri-o://7ce7294ba9d9579da040079bcbdfb32ff96d135bd75600411bf024d300f82840" gracePeriod=10
Jul 29 15:52:05.768848 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:05.768815    2575 generic.go:356] "Generic (PLEG): container finished" podID="04fcbf32-55b8-4dee-9078-8d29e6119ad7" containerID="7ce7294ba9d9579da040079bcbdfb32ff96d135bd75600411bf024d300f82840" exitCode=0
Jul 29 15:52:05.768974 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:05.768882    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp" event={"ID":"04fcbf32-55b8-4dee-9078-8d29e6119ad7","Type":"ContainerDied","Data":"7ce7294ba9d9579da040079bcbdfb32ff96d135bd75600411bf024d300f82840"}
Jul 29 15:52:05.768974 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:05.768933    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp" event={"ID":"04fcbf32-55b8-4dee-9078-8d29e6119ad7","Type":"ContainerDied","Data":"24613d58298fb2a6263c98cf82bef52f0bdb8ec60a197a3330a95633ce3792da"}
Jul 29 15:52:05.768974 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:05.768949    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24613d58298fb2a6263c98cf82bef52f0bdb8ec60a197a3330a95633ce3792da"
Jul 29 15:52:05.782664 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:05.782642    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"
Jul 29 15:52:05.928019 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:05.927943    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzsrz\" (UniqueName: \"kubernetes.io/projected/04fcbf32-55b8-4dee-9078-8d29e6119ad7-kube-api-access-wzsrz\") pod \"04fcbf32-55b8-4dee-9078-8d29e6119ad7\" (UID: \"04fcbf32-55b8-4dee-9078-8d29e6119ad7\") "
Jul 29 15:52:05.928019 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:05.928008    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04fcbf32-55b8-4dee-9078-8d29e6119ad7-extensions-socket-volume\") pod \"04fcbf32-55b8-4dee-9078-8d29e6119ad7\" (UID: \"04fcbf32-55b8-4dee-9078-8d29e6119ad7\") "
Jul 29 15:52:05.928517 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:05.928462    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04fcbf32-55b8-4dee-9078-8d29e6119ad7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "04fcbf32-55b8-4dee-9078-8d29e6119ad7" (UID: "04fcbf32-55b8-4dee-9078-8d29e6119ad7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 15:52:05.929933 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:05.929911    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04fcbf32-55b8-4dee-9078-8d29e6119ad7-kube-api-access-wzsrz" (OuterVolumeSpecName: "kube-api-access-wzsrz") pod "04fcbf32-55b8-4dee-9078-8d29e6119ad7" (UID: "04fcbf32-55b8-4dee-9078-8d29e6119ad7"). InnerVolumeSpecName "kube-api-access-wzsrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:52:06.028885 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:06.028857    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzsrz\" (UniqueName: \"kubernetes.io/projected/04fcbf32-55b8-4dee-9078-8d29e6119ad7-kube-api-access-wzsrz\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:52:06.028885 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:06.028879    2575 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/04fcbf32-55b8-4dee-9078-8d29e6119ad7-extensions-socket-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:52:06.773173 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:06.773136    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"
Jul 29 15:52:06.794976 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:06.794953    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"]
Jul 29 15:52:06.801915 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:06.801894    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fchzp"]
Jul 29 15:52:07.295332 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:07.295293    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04fcbf32-55b8-4dee-9078-8d29e6119ad7" path="/var/lib/kubelet/pods/04fcbf32-55b8-4dee-9078-8d29e6119ad7/volumes"
Jul 29 15:52:07.549015 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:52:07.548947    2575 scope.go:117] "RemoveContainer" containerID="7ce7294ba9d9579da040079bcbdfb32ff96d135bd75600411bf024d300f82840"
Jul 29 15:53:20.742940 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.742907    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvt52"]
Jul 29 15:53:20.743593 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.743282    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c0598b0-76a6-4461-9fe1-400990d25d94" containerName="maas-api"
Jul 29 15:53:20.743593 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.743294    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c0598b0-76a6-4461-9fe1-400990d25d94" containerName="maas-api"
Jul 29 15:53:20.743593 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.743304    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ff42290-6b5b-4bf4-b07f-0b87c3d486f4" containerName="payload-processing"
Jul 29 15:53:20.743593 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.743310    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ff42290-6b5b-4bf4-b07f-0b87c3d486f4" containerName="payload-processing"
Jul 29 15:53:20.743593 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.743342    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b09c3712-b3be-49bc-bea0-6a1cc4e05e99" containerName="payload-pre-processing"
Jul 29 15:53:20.743593 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.743347    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b09c3712-b3be-49bc-bea0-6a1cc4e05e99" containerName="payload-pre-processing"
Jul 29 15:53:20.743593 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.743355    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04fcbf32-55b8-4dee-9078-8d29e6119ad7" containerName="manager"
Jul 29 15:53:20.743593 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.743360    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="04fcbf32-55b8-4dee-9078-8d29e6119ad7" containerName="manager"
Jul 29 15:53:20.743593 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.743412    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b09c3712-b3be-49bc-bea0-6a1cc4e05e99" containerName="payload-pre-processing"
Jul 29 15:53:20.743593 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.743420    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c0598b0-76a6-4461-9fe1-400990d25d94" containerName="maas-api"
Jul 29 15:53:20.743593 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.743428    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="04fcbf32-55b8-4dee-9078-8d29e6119ad7" containerName="manager"
Jul 29 15:53:20.743593 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.743434    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ff42290-6b5b-4bf4-b07f-0b87c3d486f4" containerName="payload-processing"
Jul 29 15:53:20.746502 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.746487    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvt52"
Jul 29 15:53:20.749964 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.749944    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rhvk2\""
Jul 29 15:53:20.763660 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.763637    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvt52"]
Jul 29 15:53:20.767099 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.767076    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksdsh\" (UniqueName: \"kubernetes.io/projected/e0de3ccd-e482-46ac-9b8f-7aa6c1173147-kube-api-access-ksdsh\") pod \"kuadrant-operator-controller-manager-86878c769f-kvt52\" (UID: \"e0de3ccd-e482-46ac-9b8f-7aa6c1173147\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvt52"
Jul 29 15:53:20.767233 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.767125    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0de3ccd-e482-46ac-9b8f-7aa6c1173147-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kvt52\" (UID: \"e0de3ccd-e482-46ac-9b8f-7aa6c1173147\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvt52"
Jul 29 15:53:20.867807 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.867779    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksdsh\" (UniqueName: \"kubernetes.io/projected/e0de3ccd-e482-46ac-9b8f-7aa6c1173147-kube-api-access-ksdsh\") pod \"kuadrant-operator-controller-manager-86878c769f-kvt52\" (UID: \"e0de3ccd-e482-46ac-9b8f-7aa6c1173147\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvt52"
Jul 29 15:53:20.867969 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.867835    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0de3ccd-e482-46ac-9b8f-7aa6c1173147-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kvt52\" (UID: \"e0de3ccd-e482-46ac-9b8f-7aa6c1173147\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvt52"
Jul 29 15:53:20.868147 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.868132    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0de3ccd-e482-46ac-9b8f-7aa6c1173147-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-kvt52\" (UID: \"e0de3ccd-e482-46ac-9b8f-7aa6c1173147\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvt52"
Jul 29 15:53:20.878918 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:20.878893    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksdsh\" (UniqueName: \"kubernetes.io/projected/e0de3ccd-e482-46ac-9b8f-7aa6c1173147-kube-api-access-ksdsh\") pod \"kuadrant-operator-controller-manager-86878c769f-kvt52\" (UID: \"e0de3ccd-e482-46ac-9b8f-7aa6c1173147\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvt52"
Jul 29 15:53:21.056223 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:21.056129    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvt52"
Jul 29 15:53:21.204999 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:21.204964    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvt52"]
Jul 29 15:53:21.209156 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:53:21.209132    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0de3ccd_e482_46ac_9b8f_7aa6c1173147.slice/crio-148b50c0b4274287a6fa3bb9fc180e66128425348191a74e443851c679d58f60 WatchSource:0}: Error finding container 148b50c0b4274287a6fa3bb9fc180e66128425348191a74e443851c679d58f60: Status 404 returned error can't find the container with id 148b50c0b4274287a6fa3bb9fc180e66128425348191a74e443851c679d58f60
Jul 29 15:53:22.108495 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:22.108457    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvt52" event={"ID":"e0de3ccd-e482-46ac-9b8f-7aa6c1173147","Type":"ContainerStarted","Data":"176b83a5ba6d324fe320568650b69f800e527211e8167d1ec7db1994e6659f9b"}
Jul 29 15:53:22.108897 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:22.108517    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvt52" event={"ID":"e0de3ccd-e482-46ac-9b8f-7aa6c1173147","Type":"ContainerStarted","Data":"148b50c0b4274287a6fa3bb9fc180e66128425348191a74e443851c679d58f60"}
Jul 29 15:53:22.108897 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:22.108616    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvt52"
Jul 29 15:53:22.130794 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:22.130750    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvt52" podStartSLOduration=2.130736013 podStartE2EDuration="2.130736013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:53:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:53:22.129827001 +0000 UTC m=+2535.279120899" watchObservedRunningTime="2026-07-29 15:53:22.130736013 +0000 UTC m=+2535.280029911"
Jul 29 15:53:33.115465 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:33.115430    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-kvt52"
Jul 29 15:53:43.327314 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:43.327276    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"]
Jul 29 15:53:43.331710 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:43.331690    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"
Jul 29 15:53:43.333932 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:43.333915    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-87cc52d8-serving-cert\""
Jul 29 15:53:43.340020 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:43.339998    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"]
Jul 29 15:53:43.465247 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:43.465212    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlc8t\" (UniqueName: \"kubernetes.io/projected/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-kube-api-access-dlc8t\") pod \"maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8\" (UID: \"621995fc-a5f6-4b7b-8cdb-dd0a01db0477\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"
Jul 29 15:53:43.465421 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:43.465304    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-maas-api-tls\") pod \"maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8\" (UID: \"621995fc-a5f6-4b7b-8cdb-dd0a01db0477\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"
Jul 29 15:53:43.465421 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:43.465355    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-openshift-service-ca\") pod \"maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8\" (UID: \"621995fc-a5f6-4b7b-8cdb-dd0a01db0477\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"
Jul 29 15:53:43.565803 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:43.565768    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-maas-api-tls\") pod \"maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8\" (UID: \"621995fc-a5f6-4b7b-8cdb-dd0a01db0477\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"
Jul 29 15:53:43.565803 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:43.565814    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-openshift-service-ca\") pod \"maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8\" (UID: \"621995fc-a5f6-4b7b-8cdb-dd0a01db0477\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"
Jul 29 15:53:43.566055 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:43.565858    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlc8t\" (UniqueName: \"kubernetes.io/projected/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-kube-api-access-dlc8t\") pod \"maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8\" (UID: \"621995fc-a5f6-4b7b-8cdb-dd0a01db0477\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"
Jul 29 15:53:43.566055 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:53:43.565923    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-serving-cert: secret "maas-api-e2e-mt-87cc52d8-serving-cert" not found
Jul 29 15:53:43.566055 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:53:43.565992    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-maas-api-tls podName:621995fc-a5f6-4b7b-8cdb-dd0a01db0477 nodeName:}" failed. No retries permitted until 2026-07-29 15:53:44.065972215 +0000 UTC m=+2557.215266111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-maas-api-tls") pod "maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8" (UID: "621995fc-a5f6-4b7b-8cdb-dd0a01db0477") : secret "maas-api-e2e-mt-87cc52d8-serving-cert" not found
Jul 29 15:53:43.566652 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:43.566630    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-openshift-service-ca\") pod \"maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8\" (UID: \"621995fc-a5f6-4b7b-8cdb-dd0a01db0477\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"
Jul 29 15:53:43.577060 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:43.577035    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlc8t\" (UniqueName: \"kubernetes.io/projected/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-kube-api-access-dlc8t\") pod \"maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8\" (UID: \"621995fc-a5f6-4b7b-8cdb-dd0a01db0477\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"
Jul 29 15:53:44.060546 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.060510    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"]
Jul 29 15:53:44.064178 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.064162    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"
Jul 29 15:53:44.066479 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.066457    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-87cc52d8\""
Jul 29 15:53:44.070703 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.070674    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-maas-api-tls\") pod \"maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8\" (UID: \"621995fc-a5f6-4b7b-8cdb-dd0a01db0477\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"
Jul 29 15:53:44.073593 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.073571    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"]
Jul 29 15:53:44.073700 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.073605    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-maas-api-tls\") pod \"maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8\" (UID: \"621995fc-a5f6-4b7b-8cdb-dd0a01db0477\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"
Jul 29 15:53:44.087089 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.087060    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"]
Jul 29 15:53:44.091162 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.091139    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"
Jul 29 15:53:44.093468 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.093450    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-87cc52d8-dockercfg-462cf\""
Jul 29 15:53:44.102424 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.102403    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"]
Jul 29 15:53:44.171878 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.171839    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v97sf\" (UniqueName: \"kubernetes.io/projected/654f7b87-ca97-4326-8e05-ab9061bcb301-kube-api-access-v97sf\") pod \"payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr\" (UID: \"654f7b87-ca97-4326-8e05-ab9061bcb301\") " pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"
Jul 29 15:53:44.172035 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.171888    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sw2b\" (UniqueName: \"kubernetes.io/projected/7d843136-274b-43a7-af0f-7d8668314528-kube-api-access-5sw2b\") pod \"payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8\" (UID: \"7d843136-274b-43a7-af0f-7d8668314528\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"
Jul 29 15:53:44.172080 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.172033    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/654f7b87-ca97-4326-8e05-ab9061bcb301-plugins-config-volume\") pod \"payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr\" (UID: \"654f7b87-ca97-4326-8e05-ab9061bcb301\") " pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"
Jul 29 15:53:44.172080 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.172065    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d843136-274b-43a7-af0f-7d8668314528-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8\" (UID: \"7d843136-274b-43a7-af0f-7d8668314528\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"
Jul 29 15:53:44.242784 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.242753    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"
Jul 29 15:53:44.273006 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.272979    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v97sf\" (UniqueName: \"kubernetes.io/projected/654f7b87-ca97-4326-8e05-ab9061bcb301-kube-api-access-v97sf\") pod \"payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr\" (UID: \"654f7b87-ca97-4326-8e05-ab9061bcb301\") " pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"
Jul 29 15:53:44.273153 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.273021    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sw2b\" (UniqueName: \"kubernetes.io/projected/7d843136-274b-43a7-af0f-7d8668314528-kube-api-access-5sw2b\") pod \"payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8\" (UID: \"7d843136-274b-43a7-af0f-7d8668314528\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"
Jul 29 15:53:44.273153 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.273090    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/654f7b87-ca97-4326-8e05-ab9061bcb301-plugins-config-volume\") pod \"payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr\" (UID: \"654f7b87-ca97-4326-8e05-ab9061bcb301\") " pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"
Jul 29 15:53:44.273153 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.273111    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d843136-274b-43a7-af0f-7d8668314528-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8\" (UID: \"7d843136-274b-43a7-af0f-7d8668314528\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"
Jul 29 15:53:44.273847 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.273821    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/654f7b87-ca97-4326-8e05-ab9061bcb301-plugins-config-volume\") pod \"payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr\" (UID: \"654f7b87-ca97-4326-8e05-ab9061bcb301\") " pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"
Jul 29 15:53:44.273977 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.273827    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d843136-274b-43a7-af0f-7d8668314528-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8\" (UID: \"7d843136-274b-43a7-af0f-7d8668314528\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"
Jul 29 15:53:44.282696 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.282676    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v97sf\" (UniqueName: \"kubernetes.io/projected/654f7b87-ca97-4326-8e05-ab9061bcb301-kube-api-access-v97sf\") pod \"payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr\" (UID: \"654f7b87-ca97-4326-8e05-ab9061bcb301\") " pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"
Jul 29 15:53:44.283717 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.283701    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sw2b\" (UniqueName: \"kubernetes.io/projected/7d843136-274b-43a7-af0f-7d8668314528-kube-api-access-5sw2b\") pod \"payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8\" (UID: \"7d843136-274b-43a7-af0f-7d8668314528\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"
Jul 29 15:53:44.375811 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.375778    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"
Jul 29 15:53:44.396193 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.396169    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"]
Jul 29 15:53:44.399094 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:53:44.399068    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod621995fc_a5f6_4b7b_8cdb_dd0a01db0477.slice/crio-12c46747319802f51bde240b10ed45ebc2789eff1bdd0431cbf2c5f8626d674d WatchSource:0}: Error finding container 12c46747319802f51bde240b10ed45ebc2789eff1bdd0431cbf2c5f8626d674d: Status 404 returned error can't find the container with id 12c46747319802f51bde240b10ed45ebc2789eff1bdd0431cbf2c5f8626d674d
Jul 29 15:53:44.405131 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.405104    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"
Jul 29 15:53:44.731263 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.731230    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"]
Jul 29 15:53:44.734811 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:53:44.734782    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d843136_274b_43a7_af0f_7d8668314528.slice/crio-7bcddb48dfe925e6b13d37bf8576622c15bc4607cb529ceda08cc669588314dd WatchSource:0}: Error finding container 7bcddb48dfe925e6b13d37bf8576622c15bc4607cb529ceda08cc669588314dd: Status 404 returned error can't find the container with id 7bcddb48dfe925e6b13d37bf8576622c15bc4607cb529ceda08cc669588314dd
Jul 29 15:53:44.762774 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:44.762755    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"]
Jul 29 15:53:44.768452 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:53:44.768427    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod654f7b87_ca97_4326_8e05_ab9061bcb301.slice/crio-c58db4af28f0697dccbdec18e58b9c4ab3999aa03235add8672728f221a1eadb WatchSource:0}: Error finding container c58db4af28f0697dccbdec18e58b9c4ab3999aa03235add8672728f221a1eadb: Status 404 returned error can't find the container with id c58db4af28f0697dccbdec18e58b9c4ab3999aa03235add8672728f221a1eadb
Jul 29 15:53:45.216830 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:45.216796    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8" event={"ID":"621995fc-a5f6-4b7b-8cdb-dd0a01db0477","Type":"ContainerStarted","Data":"b99b91010a6d81a55d1e8e15b9e21eee68685474cc9959809a29ce07bb4dc85f"}
Jul 29 15:53:45.216968 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:45.216932    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"
Jul 29 15:53:45.217019 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:45.216986    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8" event={"ID":"621995fc-a5f6-4b7b-8cdb-dd0a01db0477","Type":"ContainerStarted","Data":"12c46747319802f51bde240b10ed45ebc2789eff1bdd0431cbf2c5f8626d674d"}
Jul 29 15:53:45.218385 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:45.218355    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8" event={"ID":"7d843136-274b-43a7-af0f-7d8668314528","Type":"ContainerStarted","Data":"afb4b6b026a041e65f16210696e08d4f9c8fbbd4480a48ece05cecb60c21d6e4"}
Jul 29 15:53:45.218509 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:45.218407    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8" event={"ID":"7d843136-274b-43a7-af0f-7d8668314528","Type":"ContainerStarted","Data":"7bcddb48dfe925e6b13d37bf8576622c15bc4607cb529ceda08cc669588314dd"}
Jul 29 15:53:45.218509 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:45.218482    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"
Jul 29 15:53:45.220336 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:45.220302    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr" event={"ID":"654f7b87-ca97-4326-8e05-ab9061bcb301","Type":"ContainerStarted","Data":"8e65709b424f1231186dbb7753acef115a8be8747886228725f0ec2e8fce9df2"}
Jul 29 15:53:45.220438 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:45.220384    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr" event={"ID":"654f7b87-ca97-4326-8e05-ab9061bcb301","Type":"ContainerStarted","Data":"c58db4af28f0697dccbdec18e58b9c4ab3999aa03235add8672728f221a1eadb"}
Jul 29 15:53:45.220782 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:45.220765    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"
Jul 29 15:53:45.236507 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:45.236462    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8" podStartSLOduration=1.535029068 podStartE2EDuration="2.236445679s" totalImagesPullingTime="701.416611ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:53:45.234759207 +0000 UTC m=+2558.384053107" watchObservedRunningTime="2026-07-29 15:53:45.236445679 +0000 UTC m=+2558.385739577"
Jul 29 15:53:45.252926 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:45.252880    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8" podStartSLOduration=1.252871121 podStartE2EDuration="1.252871121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:53:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:53:45.2527079 +0000 UTC m=+2558.402001799" watchObservedRunningTime="2026-07-29 15:53:45.252871121 +0000 UTC m=+2558.402165020"
Jul 29 15:53:45.270852 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:45.270810    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr" podStartSLOduration=1.270797241 podStartE2EDuration="1.270797241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:53:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:53:45.26922267 +0000 UTC m=+2558.418516568" watchObservedRunningTime="2026-07-29 15:53:45.270797241 +0000 UTC m=+2558.420091140"
Jul 29 15:53:51.240662 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:51.240634    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"
Jul 29 15:53:56.232521 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:56.232488    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"
Jul 29 15:53:56.233073 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:53:56.232568    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"
Jul 29 15:54:57.813108 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:57.813076    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"]
Jul 29 15:54:57.813637 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:57.813359    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8" podUID="621995fc-a5f6-4b7b-8cdb-dd0a01db0477" containerName="maas-api" containerID="cri-o://b99b91010a6d81a55d1e8e15b9e21eee68685474cc9959809a29ce07bb4dc85f" gracePeriod=30
Jul 29 15:54:57.879958 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:57.879926    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"]
Jul 29 15:54:57.880181 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:57.880156    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr" podUID="654f7b87-ca97-4326-8e05-ab9061bcb301" containerName="payload-processing" containerID="cri-o://8e65709b424f1231186dbb7753acef115a8be8747886228725f0ec2e8fce9df2" gracePeriod=30
Jul 29 15:54:57.884508 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:57.884482    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"]
Jul 29 15:54:57.884760 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:57.884703    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8" podUID="7d843136-274b-43a7-af0f-7d8668314528" containerName="payload-pre-processing" containerID="cri-o://afb4b6b026a041e65f16210696e08d4f9c8fbbd4480a48ece05cecb60c21d6e4" gracePeriod=30
Jul 29 15:54:58.151951 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.151917    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"
Jul 29 15:54:58.155102 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.155074    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"
Jul 29 15:54:58.181278 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.181256    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"
Jul 29 15:54:58.324407 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.324306    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sw2b\" (UniqueName: \"kubernetes.io/projected/7d843136-274b-43a7-af0f-7d8668314528-kube-api-access-5sw2b\") pod \"7d843136-274b-43a7-af0f-7d8668314528\" (UID: \"7d843136-274b-43a7-af0f-7d8668314528\") "
Jul 29 15:54:58.324407 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.324380    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlc8t\" (UniqueName: \"kubernetes.io/projected/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-kube-api-access-dlc8t\") pod \"621995fc-a5f6-4b7b-8cdb-dd0a01db0477\" (UID: \"621995fc-a5f6-4b7b-8cdb-dd0a01db0477\") "
Jul 29 15:54:58.324633 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.324408    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d843136-274b-43a7-af0f-7d8668314528-plugins-config-volume\") pod \"7d843136-274b-43a7-af0f-7d8668314528\" (UID: \"7d843136-274b-43a7-af0f-7d8668314528\") "
Jul 29 15:54:58.324633 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.324445    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-openshift-service-ca\") pod \"621995fc-a5f6-4b7b-8cdb-dd0a01db0477\" (UID: \"621995fc-a5f6-4b7b-8cdb-dd0a01db0477\") "
Jul 29 15:54:58.324633 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.324538    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v97sf\" (UniqueName: \"kubernetes.io/projected/654f7b87-ca97-4326-8e05-ab9061bcb301-kube-api-access-v97sf\") pod \"654f7b87-ca97-4326-8e05-ab9061bcb301\" (UID: \"654f7b87-ca97-4326-8e05-ab9061bcb301\") "
Jul 29 15:54:58.324633 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.324562    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/654f7b87-ca97-4326-8e05-ab9061bcb301-plugins-config-volume\") pod \"654f7b87-ca97-4326-8e05-ab9061bcb301\" (UID: \"654f7b87-ca97-4326-8e05-ab9061bcb301\") "
Jul 29 15:54:58.324633 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.324590    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-maas-api-tls\") pod \"621995fc-a5f6-4b7b-8cdb-dd0a01db0477\" (UID: \"621995fc-a5f6-4b7b-8cdb-dd0a01db0477\") "
Jul 29 15:54:58.324906 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.324792    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d843136-274b-43a7-af0f-7d8668314528-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7d843136-274b-43a7-af0f-7d8668314528" (UID: "7d843136-274b-43a7-af0f-7d8668314528"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:54:58.324969 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.324945    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "621995fc-a5f6-4b7b-8cdb-dd0a01db0477" (UID: "621995fc-a5f6-4b7b-8cdb-dd0a01db0477"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:54:58.325356 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.325308    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/654f7b87-ca97-4326-8e05-ab9061bcb301-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "654f7b87-ca97-4326-8e05-ab9061bcb301" (UID: "654f7b87-ca97-4326-8e05-ab9061bcb301"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 15:54:58.326635 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.326604    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-kube-api-access-dlc8t" (OuterVolumeSpecName: "kube-api-access-dlc8t") pod "621995fc-a5f6-4b7b-8cdb-dd0a01db0477" (UID: "621995fc-a5f6-4b7b-8cdb-dd0a01db0477"). InnerVolumeSpecName "kube-api-access-dlc8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:54:58.326864 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.326830    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/654f7b87-ca97-4326-8e05-ab9061bcb301-kube-api-access-v97sf" (OuterVolumeSpecName: "kube-api-access-v97sf") pod "654f7b87-ca97-4326-8e05-ab9061bcb301" (UID: "654f7b87-ca97-4326-8e05-ab9061bcb301"). InnerVolumeSpecName "kube-api-access-v97sf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:54:58.327002 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.326976    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "621995fc-a5f6-4b7b-8cdb-dd0a01db0477" (UID: "621995fc-a5f6-4b7b-8cdb-dd0a01db0477"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 15:54:58.327096 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.327022    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d843136-274b-43a7-af0f-7d8668314528-kube-api-access-5sw2b" (OuterVolumeSpecName: "kube-api-access-5sw2b") pod "7d843136-274b-43a7-af0f-7d8668314528" (UID: "7d843136-274b-43a7-af0f-7d8668314528"). InnerVolumeSpecName "kube-api-access-5sw2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 15:54:58.425886 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.425852    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v97sf\" (UniqueName: \"kubernetes.io/projected/654f7b87-ca97-4326-8e05-ab9061bcb301-kube-api-access-v97sf\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:54:58.426075 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.425892    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/654f7b87-ca97-4326-8e05-ab9061bcb301-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:54:58.426075 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.425915    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-maas-api-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:54:58.426075 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.425926    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sw2b\" (UniqueName: \"kubernetes.io/projected/7d843136-274b-43a7-af0f-7d8668314528-kube-api-access-5sw2b\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:54:58.426075 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.425935    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlc8t\" (UniqueName: \"kubernetes.io/projected/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-kube-api-access-dlc8t\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:54:58.426075 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.425943    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d843136-274b-43a7-af0f-7d8668314528-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:54:58.426075 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.425952    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621995fc-a5f6-4b7b-8cdb-dd0a01db0477-openshift-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 15:54:58.585483 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.585398    2575 generic.go:356] "Generic (PLEG): container finished" podID="621995fc-a5f6-4b7b-8cdb-dd0a01db0477" containerID="b99b91010a6d81a55d1e8e15b9e21eee68685474cc9959809a29ce07bb4dc85f" exitCode=0
Jul 29 15:54:58.585483 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.585439    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"
Jul 29 15:54:58.585710 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.585480    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8" event={"ID":"621995fc-a5f6-4b7b-8cdb-dd0a01db0477","Type":"ContainerDied","Data":"b99b91010a6d81a55d1e8e15b9e21eee68685474cc9959809a29ce07bb4dc85f"}
Jul 29 15:54:58.585710 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.585527    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8" event={"ID":"621995fc-a5f6-4b7b-8cdb-dd0a01db0477","Type":"ContainerDied","Data":"12c46747319802f51bde240b10ed45ebc2789eff1bdd0431cbf2c5f8626d674d"}
Jul 29 15:54:58.585710 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.585549    2575 scope.go:117] "RemoveContainer" containerID="b99b91010a6d81a55d1e8e15b9e21eee68685474cc9959809a29ce07bb4dc85f"
Jul 29 15:54:58.586865 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.586845    2575 generic.go:356] "Generic (PLEG): container finished" podID="7d843136-274b-43a7-af0f-7d8668314528" containerID="afb4b6b026a041e65f16210696e08d4f9c8fbbd4480a48ece05cecb60c21d6e4" exitCode=0
Jul 29 15:54:58.586990 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.586888    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"
Jul 29 15:54:58.586990 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.586874    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8" event={"ID":"7d843136-274b-43a7-af0f-7d8668314528","Type":"ContainerDied","Data":"afb4b6b026a041e65f16210696e08d4f9c8fbbd4480a48ece05cecb60c21d6e4"}
Jul 29 15:54:58.586990 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.586937    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8" event={"ID":"7d843136-274b-43a7-af0f-7d8668314528","Type":"ContainerDied","Data":"7bcddb48dfe925e6b13d37bf8576622c15bc4607cb529ceda08cc669588314dd"}
Jul 29 15:54:58.588144 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.588124    2575 generic.go:356] "Generic (PLEG): container finished" podID="654f7b87-ca97-4326-8e05-ab9061bcb301" containerID="8e65709b424f1231186dbb7753acef115a8be8747886228725f0ec2e8fce9df2" exitCode=0
Jul 29 15:54:58.588243 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.588170    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"
Jul 29 15:54:58.588243 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.588185    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr" event={"ID":"654f7b87-ca97-4326-8e05-ab9061bcb301","Type":"ContainerDied","Data":"8e65709b424f1231186dbb7753acef115a8be8747886228725f0ec2e8fce9df2"}
Jul 29 15:54:58.588243 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.588224    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr" event={"ID":"654f7b87-ca97-4326-8e05-ab9061bcb301","Type":"ContainerDied","Data":"c58db4af28f0697dccbdec18e58b9c4ab3999aa03235add8672728f221a1eadb"}
Jul 29 15:54:58.596180 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.596158    2575 scope.go:117] "RemoveContainer" containerID="b99b91010a6d81a55d1e8e15b9e21eee68685474cc9959809a29ce07bb4dc85f"
Jul 29 15:54:58.596550 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:54:58.596523    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b99b91010a6d81a55d1e8e15b9e21eee68685474cc9959809a29ce07bb4dc85f\": container with ID starting with b99b91010a6d81a55d1e8e15b9e21eee68685474cc9959809a29ce07bb4dc85f not found: ID does not exist" containerID="b99b91010a6d81a55d1e8e15b9e21eee68685474cc9959809a29ce07bb4dc85f"
Jul 29 15:54:58.596648 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.596558    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b99b91010a6d81a55d1e8e15b9e21eee68685474cc9959809a29ce07bb4dc85f"} err="failed to get container status \"b99b91010a6d81a55d1e8e15b9e21eee68685474cc9959809a29ce07bb4dc85f\": rpc error: code = NotFound desc = could not find container \"b99b91010a6d81a55d1e8e15b9e21eee68685474cc9959809a29ce07bb4dc85f\": container with ID starting with b99b91010a6d81a55d1e8e15b9e21eee68685474cc9959809a29ce07bb4dc85f not found: ID does not exist"
Jul 29 15:54:58.596648 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.596582    2575 scope.go:117] "RemoveContainer" containerID="afb4b6b026a041e65f16210696e08d4f9c8fbbd4480a48ece05cecb60c21d6e4"
Jul 29 15:54:58.604284 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.604267    2575 scope.go:117] "RemoveContainer" containerID="afb4b6b026a041e65f16210696e08d4f9c8fbbd4480a48ece05cecb60c21d6e4"
Jul 29 15:54:58.604570 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:54:58.604550    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afb4b6b026a041e65f16210696e08d4f9c8fbbd4480a48ece05cecb60c21d6e4\": container with ID starting with afb4b6b026a041e65f16210696e08d4f9c8fbbd4480a48ece05cecb60c21d6e4 not found: ID does not exist" containerID="afb4b6b026a041e65f16210696e08d4f9c8fbbd4480a48ece05cecb60c21d6e4"
Jul 29 15:54:58.604663 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.604578    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afb4b6b026a041e65f16210696e08d4f9c8fbbd4480a48ece05cecb60c21d6e4"} err="failed to get container status \"afb4b6b026a041e65f16210696e08d4f9c8fbbd4480a48ece05cecb60c21d6e4\": rpc error: code = NotFound desc = could not find container \"afb4b6b026a041e65f16210696e08d4f9c8fbbd4480a48ece05cecb60c21d6e4\": container with ID starting with afb4b6b026a041e65f16210696e08d4f9c8fbbd4480a48ece05cecb60c21d6e4 not found: ID does not exist"
Jul 29 15:54:58.604663 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.604599    2575 scope.go:117] "RemoveContainer" containerID="8e65709b424f1231186dbb7753acef115a8be8747886228725f0ec2e8fce9df2"
Jul 29 15:54:58.611442 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.611425    2575 scope.go:117] "RemoveContainer" containerID="8e65709b424f1231186dbb7753acef115a8be8747886228725f0ec2e8fce9df2"
Jul 29 15:54:58.611684 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:54:58.611668    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e65709b424f1231186dbb7753acef115a8be8747886228725f0ec2e8fce9df2\": container with ID starting with 8e65709b424f1231186dbb7753acef115a8be8747886228725f0ec2e8fce9df2 not found: ID does not exist" containerID="8e65709b424f1231186dbb7753acef115a8be8747886228725f0ec2e8fce9df2"
Jul 29 15:54:58.611730 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.611691    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e65709b424f1231186dbb7753acef115a8be8747886228725f0ec2e8fce9df2"} err="failed to get container status \"8e65709b424f1231186dbb7753acef115a8be8747886228725f0ec2e8fce9df2\": rpc error: code = NotFound desc = could not find container \"8e65709b424f1231186dbb7753acef115a8be8747886228725f0ec2e8fce9df2\": container with ID starting with 8e65709b424f1231186dbb7753acef115a8be8747886228725f0ec2e8fce9df2 not found: ID does not exist"
Jul 29 15:54:58.614906 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.614886    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"]
Jul 29 15:54:58.623424 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.623401    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-87cc52d8-657c44cc4d-tvmb8"]
Jul 29 15:54:58.635334 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.635296    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"]
Jul 29 15:54:58.642278 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.642256    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-87cc52d8-cc8d9cdbd-5p5q8"]
Jul 29 15:54:58.653651 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.653633    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"]
Jul 29 15:54:58.660391 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:58.660372    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-87cc52d8-7657c9698-57bjr"]
Jul 29 15:54:59.295017 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:59.294986    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="621995fc-a5f6-4b7b-8cdb-dd0a01db0477" path="/var/lib/kubelet/pods/621995fc-a5f6-4b7b-8cdb-dd0a01db0477/volumes"
Jul 29 15:54:59.295417 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:59.295356    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="654f7b87-ca97-4326-8e05-ab9061bcb301" path="/var/lib/kubelet/pods/654f7b87-ca97-4326-8e05-ab9061bcb301/volumes"
Jul 29 15:54:59.295659 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:54:59.295646    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d843136-274b-43a7-af0f-7d8668314528" path="/var/lib/kubelet/pods/7d843136-274b-43a7-af0f-7d8668314528/volumes"
Jul 29 15:56:07.437049 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:56:07.436968    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:56:07.444906 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:56:07.444885    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:56:07.445905 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:56:07.445883    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 15:56:07.452935 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:56:07.452918    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 15:57:39.469726 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.469649    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"]
Jul 29 15:57:39.470176 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.470033    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="621995fc-a5f6-4b7b-8cdb-dd0a01db0477" containerName="maas-api"
Jul 29 15:57:39.470176 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.470046    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="621995fc-a5f6-4b7b-8cdb-dd0a01db0477" containerName="maas-api"
Jul 29 15:57:39.470176 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.470059    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="654f7b87-ca97-4326-8e05-ab9061bcb301" containerName="payload-processing"
Jul 29 15:57:39.470176 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.470065    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="654f7b87-ca97-4326-8e05-ab9061bcb301" containerName="payload-processing"
Jul 29 15:57:39.470176 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.470085    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d843136-274b-43a7-af0f-7d8668314528" containerName="payload-pre-processing"
Jul 29 15:57:39.470176 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.470091    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d843136-274b-43a7-af0f-7d8668314528" containerName="payload-pre-processing"
Jul 29 15:57:39.470176 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.470148    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="621995fc-a5f6-4b7b-8cdb-dd0a01db0477" containerName="maas-api"
Jul 29 15:57:39.470176 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.470164    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="654f7b87-ca97-4326-8e05-ab9061bcb301" containerName="payload-processing"
Jul 29 15:57:39.470176 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.470177    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d843136-274b-43a7-af0f-7d8668314528" containerName="payload-pre-processing"
Jul 29 15:57:39.473659 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.473639    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"
Jul 29 15:57:39.475890 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.475870    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-9750ef-serving-cert\""
Jul 29 15:57:39.485886 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.485862    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"]
Jul 29 15:57:39.564509 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.564476    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ef56b0-fff6-4b6e-93a4-26633ed483b3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9750ef-564db7454-sdh5d\" (UID: \"94ef56b0-fff6-4b6e-93a4-26633ed483b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"
Jul 29 15:57:39.564665 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.564534    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94ef56b0-fff6-4b6e-93a4-26633ed483b3-maas-api-tls\") pod \"maas-api-e2e-inf-a-9750ef-564db7454-sdh5d\" (UID: \"94ef56b0-fff6-4b6e-93a4-26633ed483b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"
Jul 29 15:57:39.564665 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.564557    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv2lz\" (UniqueName: \"kubernetes.io/projected/94ef56b0-fff6-4b6e-93a4-26633ed483b3-kube-api-access-hv2lz\") pod \"maas-api-e2e-inf-a-9750ef-564db7454-sdh5d\" (UID: \"94ef56b0-fff6-4b6e-93a4-26633ed483b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"
Jul 29 15:57:39.665213 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.665180    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ef56b0-fff6-4b6e-93a4-26633ed483b3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9750ef-564db7454-sdh5d\" (UID: \"94ef56b0-fff6-4b6e-93a4-26633ed483b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"
Jul 29 15:57:39.665387 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.665235    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94ef56b0-fff6-4b6e-93a4-26633ed483b3-maas-api-tls\") pod \"maas-api-e2e-inf-a-9750ef-564db7454-sdh5d\" (UID: \"94ef56b0-fff6-4b6e-93a4-26633ed483b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"
Jul 29 15:57:39.665387 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.665372    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv2lz\" (UniqueName: \"kubernetes.io/projected/94ef56b0-fff6-4b6e-93a4-26633ed483b3-kube-api-access-hv2lz\") pod \"maas-api-e2e-inf-a-9750ef-564db7454-sdh5d\" (UID: \"94ef56b0-fff6-4b6e-93a4-26633ed483b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"
Jul 29 15:57:39.665471 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:57:39.665432    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-serving-cert: secret "maas-api-e2e-inf-a-9750ef-serving-cert" not found
Jul 29 15:57:39.665511 ip-10-0-135-116 kubenswrapper[2575]: E0729 15:57:39.665505    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94ef56b0-fff6-4b6e-93a4-26633ed483b3-maas-api-tls podName:94ef56b0-fff6-4b6e-93a4-26633ed483b3 nodeName:}" failed. No retries permitted until 2026-07-29 15:57:40.165484385 +0000 UTC m=+2793.314778283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/94ef56b0-fff6-4b6e-93a4-26633ed483b3-maas-api-tls") pod "maas-api-e2e-inf-a-9750ef-564db7454-sdh5d" (UID: "94ef56b0-fff6-4b6e-93a4-26633ed483b3") : secret "maas-api-e2e-inf-a-9750ef-serving-cert" not found
Jul 29 15:57:39.665920 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.665902    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ef56b0-fff6-4b6e-93a4-26633ed483b3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-9750ef-564db7454-sdh5d\" (UID: \"94ef56b0-fff6-4b6e-93a4-26633ed483b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"
Jul 29 15:57:39.684982 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:39.684959    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv2lz\" (UniqueName: \"kubernetes.io/projected/94ef56b0-fff6-4b6e-93a4-26633ed483b3-kube-api-access-hv2lz\") pod \"maas-api-e2e-inf-a-9750ef-564db7454-sdh5d\" (UID: \"94ef56b0-fff6-4b6e-93a4-26633ed483b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"
Jul 29 15:57:40.170931 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.170897    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94ef56b0-fff6-4b6e-93a4-26633ed483b3-maas-api-tls\") pod \"maas-api-e2e-inf-a-9750ef-564db7454-sdh5d\" (UID: \"94ef56b0-fff6-4b6e-93a4-26633ed483b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"
Jul 29 15:57:40.173684 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.173654    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94ef56b0-fff6-4b6e-93a4-26633ed483b3-maas-api-tls\") pod \"maas-api-e2e-inf-a-9750ef-564db7454-sdh5d\" (UID: \"94ef56b0-fff6-4b6e-93a4-26633ed483b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"
Jul 29 15:57:40.317651 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.317616    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"]
Jul 29 15:57:40.321663 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.321641    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"
Jul 29 15:57:40.324239 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.324208    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-9750ef\""
Jul 29 15:57:40.351713 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.351676    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"]
Jul 29 15:57:40.372370 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.372342    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d56eb4b4-2c87-4a5c-b6d7-236050c8a274-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth\" (UID: \"d56eb4b4-2c87-4a5c-b6d7-236050c8a274\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"
Jul 29 15:57:40.372515 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.372378    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfbb2\" (UniqueName: \"kubernetes.io/projected/d56eb4b4-2c87-4a5c-b6d7-236050c8a274-kube-api-access-hfbb2\") pod \"payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth\" (UID: \"d56eb4b4-2c87-4a5c-b6d7-236050c8a274\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"
Jul 29 15:57:40.379434 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.379411    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"]
Jul 29 15:57:40.382922 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.382906    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"
Jul 29 15:57:40.383067 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.383048    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"
Jul 29 15:57:40.390389 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.390370    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-9750ef-dockercfg-kv727\""
Jul 29 15:57:40.406490 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.406469    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"]
Jul 29 15:57:40.473030 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.472986    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5745876d-e1d0-4152-bd0a-f3f866e6732c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t\" (UID: \"5745876d-e1d0-4152-bd0a-f3f866e6732c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"
Jul 29 15:57:40.473444 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.473060    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d56eb4b4-2c87-4a5c-b6d7-236050c8a274-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth\" (UID: \"d56eb4b4-2c87-4a5c-b6d7-236050c8a274\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"
Jul 29 15:57:40.473444 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.473097    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfbb2\" (UniqueName: \"kubernetes.io/projected/d56eb4b4-2c87-4a5c-b6d7-236050c8a274-kube-api-access-hfbb2\") pod \"payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth\" (UID: \"d56eb4b4-2c87-4a5c-b6d7-236050c8a274\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"
Jul 29 15:57:40.473444 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.473125    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfcnb\" (UniqueName: \"kubernetes.io/projected/5745876d-e1d0-4152-bd0a-f3f866e6732c-kube-api-access-pfcnb\") pod \"payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t\" (UID: \"5745876d-e1d0-4152-bd0a-f3f866e6732c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"
Jul 29 15:57:40.474162 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.474124    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d56eb4b4-2c87-4a5c-b6d7-236050c8a274-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth\" (UID: \"d56eb4b4-2c87-4a5c-b6d7-236050c8a274\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"
Jul 29 15:57:40.484236 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.484209    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfbb2\" (UniqueName: \"kubernetes.io/projected/d56eb4b4-2c87-4a5c-b6d7-236050c8a274-kube-api-access-hfbb2\") pod \"payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth\" (UID: \"d56eb4b4-2c87-4a5c-b6d7-236050c8a274\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"
Jul 29 15:57:40.528731 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.528695    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"]
Jul 29 15:57:40.530562 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:57:40.530503    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94ef56b0_fff6_4b6e_93a4_26633ed483b3.slice/crio-c89042a149aa5acafbca083e7ff42faa42c077de209380f5be497c37d603a3cc WatchSource:0}: Error finding container c89042a149aa5acafbca083e7ff42faa42c077de209380f5be497c37d603a3cc: Status 404 returned error can't find the container with id c89042a149aa5acafbca083e7ff42faa42c077de209380f5be497c37d603a3cc
Jul 29 15:57:40.531807 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.531790    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 15:57:40.573848 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.573813    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5745876d-e1d0-4152-bd0a-f3f866e6732c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t\" (UID: \"5745876d-e1d0-4152-bd0a-f3f866e6732c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"
Jul 29 15:57:40.573994 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.573854    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfcnb\" (UniqueName: \"kubernetes.io/projected/5745876d-e1d0-4152-bd0a-f3f866e6732c-kube-api-access-pfcnb\") pod \"payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t\" (UID: \"5745876d-e1d0-4152-bd0a-f3f866e6732c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"
Jul 29 15:57:40.574516 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.574495    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5745876d-e1d0-4152-bd0a-f3f866e6732c-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t\" (UID: \"5745876d-e1d0-4152-bd0a-f3f866e6732c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"
Jul 29 15:57:40.583275 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.583257    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfcnb\" (UniqueName: \"kubernetes.io/projected/5745876d-e1d0-4152-bd0a-f3f866e6732c-kube-api-access-pfcnb\") pod \"payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t\" (UID: \"5745876d-e1d0-4152-bd0a-f3f866e6732c\") " pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"
Jul 29 15:57:40.633012 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.632978    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"
Jul 29 15:57:40.693264 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.693194    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"
Jul 29 15:57:40.784224 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.784202    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"]
Jul 29 15:57:40.787196 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:57:40.787154    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd56eb4b4_2c87_4a5c_b6d7_236050c8a274.slice/crio-004ab218c596d3194ab884f1ba984a187bd8e7881efaa969e82887978fe130e9 WatchSource:0}: Error finding container 004ab218c596d3194ab884f1ba984a187bd8e7881efaa969e82887978fe130e9: Status 404 returned error can't find the container with id 004ab218c596d3194ab884f1ba984a187bd8e7881efaa969e82887978fe130e9
Jul 29 15:57:40.864332 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:40.864293    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"]
Jul 29 15:57:40.866536 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:57:40.866509    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5745876d_e1d0_4152_bd0a_f3f866e6732c.slice/crio-58cd8ac01567c1ffe89dc345cc679c12b8ac8d49aa5c1878f8ddf6865c7007c2 WatchSource:0}: Error finding container 58cd8ac01567c1ffe89dc345cc679c12b8ac8d49aa5c1878f8ddf6865c7007c2: Status 404 returned error can't find the container with id 58cd8ac01567c1ffe89dc345cc679c12b8ac8d49aa5c1878f8ddf6865c7007c2
Jul 29 15:57:41.354356 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:41.354292    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t" event={"ID":"5745876d-e1d0-4152-bd0a-f3f866e6732c","Type":"ContainerStarted","Data":"cb5022459d907933adaac1635cbffae87645c1931a01e3b91c2d04ba076e8f81"}
Jul 29 15:57:41.354582 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:41.354370    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t" event={"ID":"5745876d-e1d0-4152-bd0a-f3f866e6732c","Type":"ContainerStarted","Data":"58cd8ac01567c1ffe89dc345cc679c12b8ac8d49aa5c1878f8ddf6865c7007c2"}
Jul 29 15:57:41.354582 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:41.354402    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"
Jul 29 15:57:41.356085 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:41.356056    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth" event={"ID":"d56eb4b4-2c87-4a5c-b6d7-236050c8a274","Type":"ContainerStarted","Data":"1325ad09ae55052a7d72a3dc3727e773f2041f562b5b5b0d6c769d9326727a6a"}
Jul 29 15:57:41.356218 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:41.356112    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth" event={"ID":"d56eb4b4-2c87-4a5c-b6d7-236050c8a274","Type":"ContainerStarted","Data":"004ab218c596d3194ab884f1ba984a187bd8e7881efaa969e82887978fe130e9"}
Jul 29 15:57:41.356218 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:41.356144    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"
Jul 29 15:57:41.357737 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:41.357712    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d" event={"ID":"94ef56b0-fff6-4b6e-93a4-26633ed483b3","Type":"ContainerStarted","Data":"841b02483c4b8a39b7a70b171dc02ed340f4a003ac67c960853c64813b2e6d54"}
Jul 29 15:57:41.357842 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:41.357749    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d" event={"ID":"94ef56b0-fff6-4b6e-93a4-26633ed483b3","Type":"ContainerStarted","Data":"c89042a149aa5acafbca083e7ff42faa42c077de209380f5be497c37d603a3cc"}
Jul 29 15:57:41.357842 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:41.357781    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"
Jul 29 15:57:41.374497 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:41.374455    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t" podStartSLOduration=1.374440845 podStartE2EDuration="1.374440845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:57:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:57:41.371948319 +0000 UTC m=+2794.521242219" watchObservedRunningTime="2026-07-29 15:57:41.374440845 +0000 UTC m=+2794.523734723"
Jul 29 15:57:41.390068 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:41.390023    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth" podStartSLOduration=1.390005104 podStartE2EDuration="1.390005104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:57:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:57:41.389269692 +0000 UTC m=+2794.538563591" watchObservedRunningTime="2026-07-29 15:57:41.390005104 +0000 UTC m=+2794.539299005"
Jul 29 15:57:41.407568 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:41.407492    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d" podStartSLOduration=1.798285664 podStartE2EDuration="2.407477722s" totalImagesPullingTime="609.192058ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:57:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:57:41.40668708 +0000 UTC m=+2794.555980980" watchObservedRunningTime="2026-07-29 15:57:41.407477722 +0000 UTC m=+2794.556771622"
Jul 29 15:57:47.367900 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:47.367867    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"
Jul 29 15:57:52.364650 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:52.364595    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"
Jul 29 15:57:52.365085 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:57:52.364888    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"
Jul 29 15:58:08.837506 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:08.837455    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"]
Jul 29 15:58:08.842592 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:08.842563    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"
Jul 29 15:58:08.845727 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:08.845700    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-dfc210-serving-cert\""
Jul 29 15:58:08.857542 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:08.857507    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"]
Jul 29 15:58:08.951471 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:08.951421    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06e122f8-82c5-4a9a-bb7f-93b922338efe-maas-api-tls\") pod \"maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k\" (UID: \"06e122f8-82c5-4a9a-bb7f-93b922338efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"
Jul 29 15:58:08.951739 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:08.951492    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v9dp\" (UniqueName: \"kubernetes.io/projected/06e122f8-82c5-4a9a-bb7f-93b922338efe-kube-api-access-6v9dp\") pod \"maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k\" (UID: \"06e122f8-82c5-4a9a-bb7f-93b922338efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"
Jul 29 15:58:08.951739 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:08.951642    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06e122f8-82c5-4a9a-bb7f-93b922338efe-openshift-service-ca\") pod \"maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k\" (UID: \"06e122f8-82c5-4a9a-bb7f-93b922338efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"
Jul 29 15:58:09.052993 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.052945    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06e122f8-82c5-4a9a-bb7f-93b922338efe-openshift-service-ca\") pod \"maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k\" (UID: \"06e122f8-82c5-4a9a-bb7f-93b922338efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"
Jul 29 15:58:09.053272 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.053039    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06e122f8-82c5-4a9a-bb7f-93b922338efe-maas-api-tls\") pod \"maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k\" (UID: \"06e122f8-82c5-4a9a-bb7f-93b922338efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"
Jul 29 15:58:09.053272 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.053081    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v9dp\" (UniqueName: \"kubernetes.io/projected/06e122f8-82c5-4a9a-bb7f-93b922338efe-kube-api-access-6v9dp\") pod \"maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k\" (UID: \"06e122f8-82c5-4a9a-bb7f-93b922338efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"
Jul 29 15:58:09.053800 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.053769    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06e122f8-82c5-4a9a-bb7f-93b922338efe-openshift-service-ca\") pod \"maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k\" (UID: \"06e122f8-82c5-4a9a-bb7f-93b922338efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"
Jul 29 15:58:09.055981 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.055957    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06e122f8-82c5-4a9a-bb7f-93b922338efe-maas-api-tls\") pod \"maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k\" (UID: \"06e122f8-82c5-4a9a-bb7f-93b922338efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"
Jul 29 15:58:09.067434 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.067405    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v9dp\" (UniqueName: \"kubernetes.io/projected/06e122f8-82c5-4a9a-bb7f-93b922338efe-kube-api-access-6v9dp\") pod \"maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k\" (UID: \"06e122f8-82c5-4a9a-bb7f-93b922338efe\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"
Jul 29 15:58:09.154465 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.154421    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"
Jul 29 15:58:09.320180 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.320136    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"]
Jul 29 15:58:09.324244 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:58:09.324201    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06e122f8_82c5_4a9a_bb7f_93b922338efe.slice/crio-e5da98cfa53d5cdcf5f297f2df3c483cc061ae90ac5e93fc2b1d11a9e4a3e2ef WatchSource:0}: Error finding container e5da98cfa53d5cdcf5f297f2df3c483cc061ae90ac5e93fc2b1d11a9e4a3e2ef: Status 404 returned error can't find the container with id e5da98cfa53d5cdcf5f297f2df3c483cc061ae90ac5e93fc2b1d11a9e4a3e2ef
Jul 29 15:58:09.500614 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.500502    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k" event={"ID":"06e122f8-82c5-4a9a-bb7f-93b922338efe","Type":"ContainerStarted","Data":"e5da98cfa53d5cdcf5f297f2df3c483cc061ae90ac5e93fc2b1d11a9e4a3e2ef"}
Jul 29 15:58:09.669647 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.669599    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"]
Jul 29 15:58:09.674770 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.674741    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"
Jul 29 15:58:09.677076 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.677048    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-dfc210\""
Jul 29 15:58:09.677263 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.677138    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-dfc210-dockercfg-vbnls\""
Jul 29 15:58:09.685361 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.685313    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"]
Jul 29 15:58:09.758747 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.758718    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t4wc\" (UniqueName: \"kubernetes.io/projected/19190c95-c5cf-47fb-95a1-fab97a9e076f-kube-api-access-8t4wc\") pod \"payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq\" (UID: \"19190c95-c5cf-47fb-95a1-fab97a9e076f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"
Jul 29 15:58:09.758895 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.758772    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19190c95-c5cf-47fb-95a1-fab97a9e076f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq\" (UID: \"19190c95-c5cf-47fb-95a1-fab97a9e076f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"
Jul 29 15:58:09.769182 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.769157    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"]
Jul 29 15:58:09.773635 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.773611    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"
Jul 29 15:58:09.788552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.788422    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"]
Jul 29 15:58:09.859895 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.859851    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t4wc\" (UniqueName: \"kubernetes.io/projected/19190c95-c5cf-47fb-95a1-fab97a9e076f-kube-api-access-8t4wc\") pod \"payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq\" (UID: \"19190c95-c5cf-47fb-95a1-fab97a9e076f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"
Jul 29 15:58:09.860523 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.859915    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m96t9\" (UniqueName: \"kubernetes.io/projected/38b30f24-595f-451f-92b6-2571b74a577c-kube-api-access-m96t9\") pod \"payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p\" (UID: \"38b30f24-595f-451f-92b6-2571b74a577c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"
Jul 29 15:58:09.860523 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.859955    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19190c95-c5cf-47fb-95a1-fab97a9e076f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq\" (UID: \"19190c95-c5cf-47fb-95a1-fab97a9e076f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"
Jul 29 15:58:09.860523 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.859990    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38b30f24-595f-451f-92b6-2571b74a577c-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p\" (UID: \"38b30f24-595f-451f-92b6-2571b74a577c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"
Jul 29 15:58:09.860846 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.860823    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19190c95-c5cf-47fb-95a1-fab97a9e076f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq\" (UID: \"19190c95-c5cf-47fb-95a1-fab97a9e076f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"
Jul 29 15:58:09.873970 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.873935    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t4wc\" (UniqueName: \"kubernetes.io/projected/19190c95-c5cf-47fb-95a1-fab97a9e076f-kube-api-access-8t4wc\") pod \"payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq\" (UID: \"19190c95-c5cf-47fb-95a1-fab97a9e076f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"
Jul 29 15:58:09.960552 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.960504    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m96t9\" (UniqueName: \"kubernetes.io/projected/38b30f24-595f-451f-92b6-2571b74a577c-kube-api-access-m96t9\") pod \"payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p\" (UID: \"38b30f24-595f-451f-92b6-2571b74a577c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"
Jul 29 15:58:09.960788 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.960574    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38b30f24-595f-451f-92b6-2571b74a577c-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p\" (UID: \"38b30f24-595f-451f-92b6-2571b74a577c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"
Jul 29 15:58:09.961431 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.961406    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38b30f24-595f-451f-92b6-2571b74a577c-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p\" (UID: \"38b30f24-595f-451f-92b6-2571b74a577c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"
Jul 29 15:58:09.981334 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.981289    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m96t9\" (UniqueName: \"kubernetes.io/projected/38b30f24-595f-451f-92b6-2571b74a577c-kube-api-access-m96t9\") pod \"payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p\" (UID: \"38b30f24-595f-451f-92b6-2571b74a577c\") " pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"
Jul 29 15:58:09.989148 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:09.989108    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"
Jul 29 15:58:10.090994 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:10.090885    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"
Jul 29 15:58:10.161364 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:10.158078    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"]
Jul 29 15:58:10.171667 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:58:10.171519    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19190c95_c5cf_47fb_95a1_fab97a9e076f.slice/crio-853fad44da674056ca26e6d93c664d49be243104e4d548a87c62eca637835bbc WatchSource:0}: Error finding container 853fad44da674056ca26e6d93c664d49be243104e4d548a87c62eca637835bbc: Status 404 returned error can't find the container with id 853fad44da674056ca26e6d93c664d49be243104e4d548a87c62eca637835bbc
Jul 29 15:58:10.276223 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:10.276171    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"]
Jul 29 15:58:10.279021 ip-10-0-135-116 kubenswrapper[2575]: W0729 15:58:10.278987    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38b30f24_595f_451f_92b6_2571b74a577c.slice/crio-b4a2425305d3d539301c8741a611db54f6786de70996ed52ca6cbe6e70e8d2e3 WatchSource:0}: Error finding container b4a2425305d3d539301c8741a611db54f6786de70996ed52ca6cbe6e70e8d2e3: Status 404 returned error can't find the container with id b4a2425305d3d539301c8741a611db54f6786de70996ed52ca6cbe6e70e8d2e3
Jul 29 15:58:10.507119 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:10.507076    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq" event={"ID":"19190c95-c5cf-47fb-95a1-fab97a9e076f","Type":"ContainerStarted","Data":"2a440b1fce8e6af35aa4247bb09854ef42e46260c7756b3fb4ec5d0f419c807e"}
Jul 29 15:58:10.507385 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:10.507158    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"
Jul 29 15:58:10.507385 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:10.507191    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq" event={"ID":"19190c95-c5cf-47fb-95a1-fab97a9e076f","Type":"ContainerStarted","Data":"853fad44da674056ca26e6d93c664d49be243104e4d548a87c62eca637835bbc"}
Jul 29 15:58:10.508913 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:10.508879    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k" event={"ID":"06e122f8-82c5-4a9a-bb7f-93b922338efe","Type":"ContainerStarted","Data":"a4403f75d50f85ff2bda784257999f1e553ed28b2da74946ef228a148bbcc3cf"}
Jul 29 15:58:10.509085 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:10.508970    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"
Jul 29 15:58:10.510479 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:10.510457    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p" event={"ID":"38b30f24-595f-451f-92b6-2571b74a577c","Type":"ContainerStarted","Data":"a775d1b7bb264d6d4ada2e47b6f9a2aad2cb655e8c4434cfa379dbb0f58daaa6"}
Jul 29 15:58:10.510595 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:10.510500    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p" event={"ID":"38b30f24-595f-451f-92b6-2571b74a577c","Type":"ContainerStarted","Data":"b4a2425305d3d539301c8741a611db54f6786de70996ed52ca6cbe6e70e8d2e3"}
Jul 29 15:58:10.510595 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:10.510579    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"
Jul 29 15:58:10.528243 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:10.528180    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq" podStartSLOduration=1.528147607 podStartE2EDuration="1.528147607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:58:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:58:10.525508953 +0000 UTC m=+2823.674802855" watchObservedRunningTime="2026-07-29 15:58:10.528147607 +0000 UTC m=+2823.677441485"
Jul 29 15:58:10.546761 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:10.546690    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k" podStartSLOduration=2.13300726 podStartE2EDuration="2.546671801s" totalImagesPullingTime="413.664541ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:58:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:58:10.544439882 +0000 UTC m=+2823.693733787" watchObservedRunningTime="2026-07-29 15:58:10.546671801 +0000 UTC m=+2823.695965701"
Jul 29 15:58:10.563901 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:10.563785    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p" podStartSLOduration=1.563763844 podStartE2EDuration="1.563763844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 15:58:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 15:58:10.560673913 +0000 UTC m=+2823.709967813" watchObservedRunningTime="2026-07-29 15:58:10.563763844 +0000 UTC m=+2823.713057831"
Jul 29 15:58:16.523228 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:16.523168    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"
Jul 29 15:58:21.519572 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:21.519523    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"
Jul 29 15:58:21.520074 ip-10-0-135-116 kubenswrapper[2575]: I0729 15:58:21.519788    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"
Jul 29 16:00:26.167730 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.167666    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"]
Jul 29 16:00:26.168543 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.167961    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d" podUID="94ef56b0-fff6-4b6e-93a4-26633ed483b3" containerName="maas-api" containerID="cri-o://841b02483c4b8a39b7a70b171dc02ed340f4a003ac67c960853c64813b2e6d54" gracePeriod=30
Jul 29 16:00:26.243740 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.243664    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"]
Jul 29 16:00:26.244049 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.243980    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t" podUID="5745876d-e1d0-4152-bd0a-f3f866e6732c" containerName="payload-processing" containerID="cri-o://cb5022459d907933adaac1635cbffae87645c1931a01e3b91c2d04ba076e8f81" gracePeriod=30
Jul 29 16:00:26.252982 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.252155    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"]
Jul 29 16:00:26.252982 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.252463    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth" podUID="d56eb4b4-2c87-4a5c-b6d7-236050c8a274" containerName="payload-pre-processing" containerID="cri-o://1325ad09ae55052a7d72a3dc3727e773f2041f562b5b5b0d6c769d9326727a6a" gracePeriod=30
Jul 29 16:00:26.534967 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.534927    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"
Jul 29 16:00:26.569019 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.568982    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"
Jul 29 16:00:26.583518 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.583482    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"
Jul 29 16:00:26.646307 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.646257    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hv2lz\" (UniqueName: \"kubernetes.io/projected/94ef56b0-fff6-4b6e-93a4-26633ed483b3-kube-api-access-hv2lz\") pod \"94ef56b0-fff6-4b6e-93a4-26633ed483b3\" (UID: \"94ef56b0-fff6-4b6e-93a4-26633ed483b3\") "
Jul 29 16:00:26.646592 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.646353    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94ef56b0-fff6-4b6e-93a4-26633ed483b3-maas-api-tls\") pod \"94ef56b0-fff6-4b6e-93a4-26633ed483b3\" (UID: \"94ef56b0-fff6-4b6e-93a4-26633ed483b3\") "
Jul 29 16:00:26.646592 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.646443    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfcnb\" (UniqueName: \"kubernetes.io/projected/5745876d-e1d0-4152-bd0a-f3f866e6732c-kube-api-access-pfcnb\") pod \"5745876d-e1d0-4152-bd0a-f3f866e6732c\" (UID: \"5745876d-e1d0-4152-bd0a-f3f866e6732c\") "
Jul 29 16:00:26.646592 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.646476    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d56eb4b4-2c87-4a5c-b6d7-236050c8a274-plugins-config-volume\") pod \"d56eb4b4-2c87-4a5c-b6d7-236050c8a274\" (UID: \"d56eb4b4-2c87-4a5c-b6d7-236050c8a274\") "
Jul 29 16:00:26.646592 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.646530    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfbb2\" (UniqueName: \"kubernetes.io/projected/d56eb4b4-2c87-4a5c-b6d7-236050c8a274-kube-api-access-hfbb2\") pod \"d56eb4b4-2c87-4a5c-b6d7-236050c8a274\" (UID: \"d56eb4b4-2c87-4a5c-b6d7-236050c8a274\") "
Jul 29 16:00:26.646592 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.646585    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5745876d-e1d0-4152-bd0a-f3f866e6732c-plugins-config-volume\") pod \"5745876d-e1d0-4152-bd0a-f3f866e6732c\" (UID: \"5745876d-e1d0-4152-bd0a-f3f866e6732c\") "
Jul 29 16:00:26.646853 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.646632    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ef56b0-fff6-4b6e-93a4-26633ed483b3-openshift-service-ca\") pod \"94ef56b0-fff6-4b6e-93a4-26633ed483b3\" (UID: \"94ef56b0-fff6-4b6e-93a4-26633ed483b3\") "
Jul 29 16:00:26.647069 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.646925    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d56eb4b4-2c87-4a5c-b6d7-236050c8a274-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d56eb4b4-2c87-4a5c-b6d7-236050c8a274" (UID: "d56eb4b4-2c87-4a5c-b6d7-236050c8a274"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:00:26.647069 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.647032    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d56eb4b4-2c87-4a5c-b6d7-236050c8a274-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:00:26.647069 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.647034    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5745876d-e1d0-4152-bd0a-f3f866e6732c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5745876d-e1d0-4152-bd0a-f3f866e6732c" (UID: "5745876d-e1d0-4152-bd0a-f3f866e6732c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:00:26.647616 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.647343    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94ef56b0-fff6-4b6e-93a4-26633ed483b3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "94ef56b0-fff6-4b6e-93a4-26633ed483b3" (UID: "94ef56b0-fff6-4b6e-93a4-26633ed483b3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:00:26.648899 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.648870    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94ef56b0-fff6-4b6e-93a4-26633ed483b3-kube-api-access-hv2lz" (OuterVolumeSpecName: "kube-api-access-hv2lz") pod "94ef56b0-fff6-4b6e-93a4-26633ed483b3" (UID: "94ef56b0-fff6-4b6e-93a4-26633ed483b3"). InnerVolumeSpecName "kube-api-access-hv2lz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:00:26.649039 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.648911    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5745876d-e1d0-4152-bd0a-f3f866e6732c-kube-api-access-pfcnb" (OuterVolumeSpecName: "kube-api-access-pfcnb") pod "5745876d-e1d0-4152-bd0a-f3f866e6732c" (UID: "5745876d-e1d0-4152-bd0a-f3f866e6732c"). InnerVolumeSpecName "kube-api-access-pfcnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:00:26.649039 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.648922    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94ef56b0-fff6-4b6e-93a4-26633ed483b3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "94ef56b0-fff6-4b6e-93a4-26633ed483b3" (UID: "94ef56b0-fff6-4b6e-93a4-26633ed483b3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:00:26.649039 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.648954    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d56eb4b4-2c87-4a5c-b6d7-236050c8a274-kube-api-access-hfbb2" (OuterVolumeSpecName: "kube-api-access-hfbb2") pod "d56eb4b4-2c87-4a5c-b6d7-236050c8a274" (UID: "d56eb4b4-2c87-4a5c-b6d7-236050c8a274"). InnerVolumeSpecName "kube-api-access-hfbb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:00:26.748746 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.748631    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94ef56b0-fff6-4b6e-93a4-26633ed483b3-openshift-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:00:26.748746 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.748666    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hv2lz\" (UniqueName: \"kubernetes.io/projected/94ef56b0-fff6-4b6e-93a4-26633ed483b3-kube-api-access-hv2lz\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:00:26.748746 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.748678    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94ef56b0-fff6-4b6e-93a4-26633ed483b3-maas-api-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:00:26.748746 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.748687    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfcnb\" (UniqueName: \"kubernetes.io/projected/5745876d-e1d0-4152-bd0a-f3f866e6732c-kube-api-access-pfcnb\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:00:26.748746 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.748696    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfbb2\" (UniqueName: \"kubernetes.io/projected/d56eb4b4-2c87-4a5c-b6d7-236050c8a274-kube-api-access-hfbb2\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:00:26.748746 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:26.748705    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5745876d-e1d0-4152-bd0a-f3f866e6732c-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:00:27.278649 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.278578    2575 generic.go:356] "Generic (PLEG): container finished" podID="94ef56b0-fff6-4b6e-93a4-26633ed483b3" containerID="841b02483c4b8a39b7a70b171dc02ed340f4a003ac67c960853c64813b2e6d54" exitCode=0
Jul 29 16:00:27.278649 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.278641    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d" event={"ID":"94ef56b0-fff6-4b6e-93a4-26633ed483b3","Type":"ContainerDied","Data":"841b02483c4b8a39b7a70b171dc02ed340f4a003ac67c960853c64813b2e6d54"}
Jul 29 16:00:27.279333 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.278663    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"
Jul 29 16:00:27.279333 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.278752    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d" event={"ID":"94ef56b0-fff6-4b6e-93a4-26633ed483b3","Type":"ContainerDied","Data":"c89042a149aa5acafbca083e7ff42faa42c077de209380f5be497c37d603a3cc"}
Jul 29 16:00:27.279333 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.278780    2575 scope.go:117] "RemoveContainer" containerID="841b02483c4b8a39b7a70b171dc02ed340f4a003ac67c960853c64813b2e6d54"
Jul 29 16:00:27.280796 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.280777    2575 generic.go:356] "Generic (PLEG): container finished" podID="5745876d-e1d0-4152-bd0a-f3f866e6732c" containerID="cb5022459d907933adaac1635cbffae87645c1931a01e3b91c2d04ba076e8f81" exitCode=0
Jul 29 16:00:27.280933 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.280840    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"
Jul 29 16:00:27.280933 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.280866    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t" event={"ID":"5745876d-e1d0-4152-bd0a-f3f866e6732c","Type":"ContainerDied","Data":"cb5022459d907933adaac1635cbffae87645c1931a01e3b91c2d04ba076e8f81"}
Jul 29 16:00:27.281047 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.280931    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t" event={"ID":"5745876d-e1d0-4152-bd0a-f3f866e6732c","Type":"ContainerDied","Data":"58cd8ac01567c1ffe89dc345cc679c12b8ac8d49aa5c1878f8ddf6865c7007c2"}
Jul 29 16:00:27.282602 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.282542    2575 generic.go:356] "Generic (PLEG): container finished" podID="d56eb4b4-2c87-4a5c-b6d7-236050c8a274" containerID="1325ad09ae55052a7d72a3dc3727e773f2041f562b5b5b0d6c769d9326727a6a" exitCode=0
Jul 29 16:00:27.282602 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.282589    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"
Jul 29 16:00:27.282853 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.282605    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth" event={"ID":"d56eb4b4-2c87-4a5c-b6d7-236050c8a274","Type":"ContainerDied","Data":"1325ad09ae55052a7d72a3dc3727e773f2041f562b5b5b0d6c769d9326727a6a"}
Jul 29 16:00:27.282853 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.282654    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth" event={"ID":"d56eb4b4-2c87-4a5c-b6d7-236050c8a274","Type":"ContainerDied","Data":"004ab218c596d3194ab884f1ba984a187bd8e7881efaa969e82887978fe130e9"}
Jul 29 16:00:27.291789 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.291762    2575 scope.go:117] "RemoveContainer" containerID="841b02483c4b8a39b7a70b171dc02ed340f4a003ac67c960853c64813b2e6d54"
Jul 29 16:00:27.292135 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:00:27.292114    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"841b02483c4b8a39b7a70b171dc02ed340f4a003ac67c960853c64813b2e6d54\": container with ID starting with 841b02483c4b8a39b7a70b171dc02ed340f4a003ac67c960853c64813b2e6d54 not found: ID does not exist" containerID="841b02483c4b8a39b7a70b171dc02ed340f4a003ac67c960853c64813b2e6d54"
Jul 29 16:00:27.292230 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.292149    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"841b02483c4b8a39b7a70b171dc02ed340f4a003ac67c960853c64813b2e6d54"} err="failed to get container status \"841b02483c4b8a39b7a70b171dc02ed340f4a003ac67c960853c64813b2e6d54\": rpc error: code = NotFound desc = could not find container \"841b02483c4b8a39b7a70b171dc02ed340f4a003ac67c960853c64813b2e6d54\": container with ID starting with 841b02483c4b8a39b7a70b171dc02ed340f4a003ac67c960853c64813b2e6d54 not found: ID does not exist"
Jul 29 16:00:27.292230 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.292174    2575 scope.go:117] "RemoveContainer" containerID="cb5022459d907933adaac1635cbffae87645c1931a01e3b91c2d04ba076e8f81"
Jul 29 16:00:27.306013 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.305993    2575 scope.go:117] "RemoveContainer" containerID="cb5022459d907933adaac1635cbffae87645c1931a01e3b91c2d04ba076e8f81"
Jul 29 16:00:27.306361 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:00:27.306334    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb5022459d907933adaac1635cbffae87645c1931a01e3b91c2d04ba076e8f81\": container with ID starting with cb5022459d907933adaac1635cbffae87645c1931a01e3b91c2d04ba076e8f81 not found: ID does not exist" containerID="cb5022459d907933adaac1635cbffae87645c1931a01e3b91c2d04ba076e8f81"
Jul 29 16:00:27.306459 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.306372    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb5022459d907933adaac1635cbffae87645c1931a01e3b91c2d04ba076e8f81"} err="failed to get container status \"cb5022459d907933adaac1635cbffae87645c1931a01e3b91c2d04ba076e8f81\": rpc error: code = NotFound desc = could not find container \"cb5022459d907933adaac1635cbffae87645c1931a01e3b91c2d04ba076e8f81\": container with ID starting with cb5022459d907933adaac1635cbffae87645c1931a01e3b91c2d04ba076e8f81 not found: ID does not exist"
Jul 29 16:00:27.306459 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.306398    2575 scope.go:117] "RemoveContainer" containerID="1325ad09ae55052a7d72a3dc3727e773f2041f562b5b5b0d6c769d9326727a6a"
Jul 29 16:00:27.311243 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.311204    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"]
Jul 29 16:00:27.315581 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.315559    2575 scope.go:117] "RemoveContainer" containerID="1325ad09ae55052a7d72a3dc3727e773f2041f562b5b5b0d6c769d9326727a6a"
Jul 29 16:00:27.316143 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:00:27.316103    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1325ad09ae55052a7d72a3dc3727e773f2041f562b5b5b0d6c769d9326727a6a\": container with ID starting with 1325ad09ae55052a7d72a3dc3727e773f2041f562b5b5b0d6c769d9326727a6a not found: ID does not exist" containerID="1325ad09ae55052a7d72a3dc3727e773f2041f562b5b5b0d6c769d9326727a6a"
Jul 29 16:00:27.316292 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.316150    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1325ad09ae55052a7d72a3dc3727e773f2041f562b5b5b0d6c769d9326727a6a"} err="failed to get container status \"1325ad09ae55052a7d72a3dc3727e773f2041f562b5b5b0d6c769d9326727a6a\": rpc error: code = NotFound desc = could not find container \"1325ad09ae55052a7d72a3dc3727e773f2041f562b5b5b0d6c769d9326727a6a\": container with ID starting with 1325ad09ae55052a7d72a3dc3727e773f2041f562b5b5b0d6c769d9326727a6a not found: ID does not exist"
Jul 29 16:00:27.321476 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.321450    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-9750ef-564db7454-sdh5d"]
Jul 29 16:00:27.333496 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.333465    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"]
Jul 29 16:00:27.338337 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.338292    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-9750ef-56bbc64444-4gsth"]
Jul 29 16:00:27.349957 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.349931    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"]
Jul 29 16:00:27.354707 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:27.354683    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-9750ef-5d59c47df4-6ng4t"]
Jul 29 16:00:29.296474 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:29.296383    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5745876d-e1d0-4152-bd0a-f3f866e6732c" path="/var/lib/kubelet/pods/5745876d-e1d0-4152-bd0a-f3f866e6732c/volumes"
Jul 29 16:00:29.296899 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:29.296806    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94ef56b0-fff6-4b6e-93a4-26633ed483b3" path="/var/lib/kubelet/pods/94ef56b0-fff6-4b6e-93a4-26633ed483b3/volumes"
Jul 29 16:00:29.297129 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:29.297115    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d56eb4b4-2c87-4a5c-b6d7-236050c8a274" path="/var/lib/kubelet/pods/d56eb4b4-2c87-4a5c-b6d7-236050c8a274/volumes"
Jul 29 16:00:39.023048 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.023001    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"]
Jul 29 16:00:39.025990 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.023313    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k" podUID="06e122f8-82c5-4a9a-bb7f-93b922338efe" containerName="maas-api" containerID="cri-o://a4403f75d50f85ff2bda784257999f1e553ed28b2da74946ef228a148bbcc3cf" gracePeriod=30
Jul 29 16:00:39.096311 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.096269    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"]
Jul 29 16:00:39.096840 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.096597    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p" podUID="38b30f24-595f-451f-92b6-2571b74a577c" containerName="payload-processing" containerID="cri-o://a775d1b7bb264d6d4ada2e47b6f9a2aad2cb655e8c4434cfa379dbb0f58daaa6" gracePeriod=30
Jul 29 16:00:39.104696 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.104656    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"]
Jul 29 16:00:39.104928 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.104897    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq" podUID="19190c95-c5cf-47fb-95a1-fab97a9e076f" containerName="payload-pre-processing" containerID="cri-o://2a440b1fce8e6af35aa4247bb09854ef42e46260c7756b3fb4ec5d0f419c807e" gracePeriod=30
Jul 29 16:00:39.357962 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.357917    2575 generic.go:356] "Generic (PLEG): container finished" podID="19190c95-c5cf-47fb-95a1-fab97a9e076f" containerID="2a440b1fce8e6af35aa4247bb09854ef42e46260c7756b3fb4ec5d0f419c807e" exitCode=0
Jul 29 16:00:39.357962 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.357943    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq" event={"ID":"19190c95-c5cf-47fb-95a1-fab97a9e076f","Type":"ContainerDied","Data":"2a440b1fce8e6af35aa4247bb09854ef42e46260c7756b3fb4ec5d0f419c807e"}
Jul 29 16:00:39.360746 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.360712    2575 generic.go:356] "Generic (PLEG): container finished" podID="06e122f8-82c5-4a9a-bb7f-93b922338efe" containerID="a4403f75d50f85ff2bda784257999f1e553ed28b2da74946ef228a148bbcc3cf" exitCode=0
Jul 29 16:00:39.360953 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.360875    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k" event={"ID":"06e122f8-82c5-4a9a-bb7f-93b922338efe","Type":"ContainerDied","Data":"a4403f75d50f85ff2bda784257999f1e553ed28b2da74946ef228a148bbcc3cf"}
Jul 29 16:00:39.364505 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.364471    2575 generic.go:356] "Generic (PLEG): container finished" podID="38b30f24-595f-451f-92b6-2571b74a577c" containerID="a775d1b7bb264d6d4ada2e47b6f9a2aad2cb655e8c4434cfa379dbb0f58daaa6" exitCode=0
Jul 29 16:00:39.364678 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.364522    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p" event={"ID":"38b30f24-595f-451f-92b6-2571b74a577c","Type":"ContainerDied","Data":"a775d1b7bb264d6d4ada2e47b6f9a2aad2cb655e8c4434cfa379dbb0f58daaa6"}
Jul 29 16:00:39.386793 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.386762    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"
Jul 29 16:00:39.400271 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.400242    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"
Jul 29 16:00:39.440931 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.440895    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"
Jul 29 16:00:39.480029 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.479981    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06e122f8-82c5-4a9a-bb7f-93b922338efe-maas-api-tls\") pod \"06e122f8-82c5-4a9a-bb7f-93b922338efe\" (UID: \"06e122f8-82c5-4a9a-bb7f-93b922338efe\") "
Jul 29 16:00:39.480265 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.480072    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m96t9\" (UniqueName: \"kubernetes.io/projected/38b30f24-595f-451f-92b6-2571b74a577c-kube-api-access-m96t9\") pod \"38b30f24-595f-451f-92b6-2571b74a577c\" (UID: \"38b30f24-595f-451f-92b6-2571b74a577c\") "
Jul 29 16:00:39.480265 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.480121    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v9dp\" (UniqueName: \"kubernetes.io/projected/06e122f8-82c5-4a9a-bb7f-93b922338efe-kube-api-access-6v9dp\") pod \"06e122f8-82c5-4a9a-bb7f-93b922338efe\" (UID: \"06e122f8-82c5-4a9a-bb7f-93b922338efe\") "
Jul 29 16:00:39.480265 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.480143    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19190c95-c5cf-47fb-95a1-fab97a9e076f-plugins-config-volume\") pod \"19190c95-c5cf-47fb-95a1-fab97a9e076f\" (UID: \"19190c95-c5cf-47fb-95a1-fab97a9e076f\") "
Jul 29 16:00:39.480265 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.480184    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38b30f24-595f-451f-92b6-2571b74a577c-plugins-config-volume\") pod \"38b30f24-595f-451f-92b6-2571b74a577c\" (UID: \"38b30f24-595f-451f-92b6-2571b74a577c\") "
Jul 29 16:00:39.480265 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.480214    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8t4wc\" (UniqueName: \"kubernetes.io/projected/19190c95-c5cf-47fb-95a1-fab97a9e076f-kube-api-access-8t4wc\") pod \"19190c95-c5cf-47fb-95a1-fab97a9e076f\" (UID: \"19190c95-c5cf-47fb-95a1-fab97a9e076f\") "
Jul 29 16:00:39.480717 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.480682    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38b30f24-595f-451f-92b6-2571b74a577c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "38b30f24-595f-451f-92b6-2571b74a577c" (UID: "38b30f24-595f-451f-92b6-2571b74a577c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:00:39.480862 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.480708    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19190c95-c5cf-47fb-95a1-fab97a9e076f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "19190c95-c5cf-47fb-95a1-fab97a9e076f" (UID: "19190c95-c5cf-47fb-95a1-fab97a9e076f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:00:39.480862 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.480770    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06e122f8-82c5-4a9a-bb7f-93b922338efe-openshift-service-ca\") pod \"06e122f8-82c5-4a9a-bb7f-93b922338efe\" (UID: \"06e122f8-82c5-4a9a-bb7f-93b922338efe\") "
Jul 29 16:00:39.481211 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.481170    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06e122f8-82c5-4a9a-bb7f-93b922338efe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "06e122f8-82c5-4a9a-bb7f-93b922338efe" (UID: "06e122f8-82c5-4a9a-bb7f-93b922338efe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:00:39.481444 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.481416    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38b30f24-595f-451f-92b6-2571b74a577c-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:00:39.481558 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.481447    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06e122f8-82c5-4a9a-bb7f-93b922338efe-openshift-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:00:39.481558 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.481464    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19190c95-c5cf-47fb-95a1-fab97a9e076f-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:00:39.482998 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.482966    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38b30f24-595f-451f-92b6-2571b74a577c-kube-api-access-m96t9" (OuterVolumeSpecName: "kube-api-access-m96t9") pod "38b30f24-595f-451f-92b6-2571b74a577c" (UID: "38b30f24-595f-451f-92b6-2571b74a577c"). InnerVolumeSpecName "kube-api-access-m96t9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:00:39.483126 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.483044    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06e122f8-82c5-4a9a-bb7f-93b922338efe-kube-api-access-6v9dp" (OuterVolumeSpecName: "kube-api-access-6v9dp") pod "06e122f8-82c5-4a9a-bb7f-93b922338efe" (UID: "06e122f8-82c5-4a9a-bb7f-93b922338efe"). InnerVolumeSpecName "kube-api-access-6v9dp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:00:39.483245 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.483185    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19190c95-c5cf-47fb-95a1-fab97a9e076f-kube-api-access-8t4wc" (OuterVolumeSpecName: "kube-api-access-8t4wc") pod "19190c95-c5cf-47fb-95a1-fab97a9e076f" (UID: "19190c95-c5cf-47fb-95a1-fab97a9e076f"). InnerVolumeSpecName "kube-api-access-8t4wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:00:39.483752 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.483731    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06e122f8-82c5-4a9a-bb7f-93b922338efe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "06e122f8-82c5-4a9a-bb7f-93b922338efe" (UID: "06e122f8-82c5-4a9a-bb7f-93b922338efe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:00:39.582606 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.582485    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06e122f8-82c5-4a9a-bb7f-93b922338efe-maas-api-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:00:39.582606 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.582522    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m96t9\" (UniqueName: \"kubernetes.io/projected/38b30f24-595f-451f-92b6-2571b74a577c-kube-api-access-m96t9\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:00:39.582606 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.582536    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v9dp\" (UniqueName: \"kubernetes.io/projected/06e122f8-82c5-4a9a-bb7f-93b922338efe-kube-api-access-6v9dp\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:00:39.582606 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:39.582548    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8t4wc\" (UniqueName: \"kubernetes.io/projected/19190c95-c5cf-47fb-95a1-fab97a9e076f-kube-api-access-8t4wc\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:00:40.371973 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:40.371939    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"
Jul 29 16:00:40.372562 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:40.371970    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k" event={"ID":"06e122f8-82c5-4a9a-bb7f-93b922338efe","Type":"ContainerDied","Data":"e5da98cfa53d5cdcf5f297f2df3c483cc061ae90ac5e93fc2b1d11a9e4a3e2ef"}
Jul 29 16:00:40.372562 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:40.372060    2575 scope.go:117] "RemoveContainer" containerID="a4403f75d50f85ff2bda784257999f1e553ed28b2da74946ef228a148bbcc3cf"
Jul 29 16:00:40.373617 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:40.373552    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"
Jul 29 16:00:40.373617 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:40.373585    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p" event={"ID":"38b30f24-595f-451f-92b6-2571b74a577c","Type":"ContainerDied","Data":"b4a2425305d3d539301c8741a611db54f6786de70996ed52ca6cbe6e70e8d2e3"}
Jul 29 16:00:40.375315 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:40.375290    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"
Jul 29 16:00:40.375315 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:40.375291    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq" event={"ID":"19190c95-c5cf-47fb-95a1-fab97a9e076f","Type":"ContainerDied","Data":"853fad44da674056ca26e6d93c664d49be243104e4d548a87c62eca637835bbc"}
Jul 29 16:00:40.382306 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:40.382275    2575 scope.go:117] "RemoveContainer" containerID="a775d1b7bb264d6d4ada2e47b6f9a2aad2cb655e8c4434cfa379dbb0f58daaa6"
Jul 29 16:00:40.393514 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:40.393493    2575 scope.go:117] "RemoveContainer" containerID="2a440b1fce8e6af35aa4247bb09854ef42e46260c7756b3fb4ec5d0f419c807e"
Jul 29 16:00:40.409134 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:40.409107    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"]
Jul 29 16:00:40.413005 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:40.412970    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-dfc210-5548c96c68-7cksq"]
Jul 29 16:00:40.426362 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:40.426313    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"]
Jul 29 16:00:40.430407 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:40.430376    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-dfc210-66cc895cb9-t5w7k"]
Jul 29 16:00:40.442123 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:40.442090    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"]
Jul 29 16:00:40.446726 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:40.446703    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-dfc210-f99497548-mwv2p"]
Jul 29 16:00:41.296164 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:41.296108    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06e122f8-82c5-4a9a-bb7f-93b922338efe" path="/var/lib/kubelet/pods/06e122f8-82c5-4a9a-bb7f-93b922338efe/volumes"
Jul 29 16:00:41.296517 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:41.296503    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19190c95-c5cf-47fb-95a1-fab97a9e076f" path="/var/lib/kubelet/pods/19190c95-c5cf-47fb-95a1-fab97a9e076f/volumes"
Jul 29 16:00:41.296826 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:41.296815    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38b30f24-595f-451f-92b6-2571b74a577c" path="/var/lib/kubelet/pods/38b30f24-595f-451f-92b6-2571b74a577c/volumes"
Jul 29 16:00:50.478493 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.478456    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"]
Jul 29 16:00:50.478964 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.478879    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d56eb4b4-2c87-4a5c-b6d7-236050c8a274" containerName="payload-pre-processing"
Jul 29 16:00:50.478964 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.478892    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d56eb4b4-2c87-4a5c-b6d7-236050c8a274" containerName="payload-pre-processing"
Jul 29 16:00:50.478964 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.478905    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19190c95-c5cf-47fb-95a1-fab97a9e076f" containerName="payload-pre-processing"
Jul 29 16:00:50.478964 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.478911    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="19190c95-c5cf-47fb-95a1-fab97a9e076f" containerName="payload-pre-processing"
Jul 29 16:00:50.478964 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.478918    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38b30f24-595f-451f-92b6-2571b74a577c" containerName="payload-processing"
Jul 29 16:00:50.478964 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.478924    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="38b30f24-595f-451f-92b6-2571b74a577c" containerName="payload-processing"
Jul 29 16:00:50.478964 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.478935    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06e122f8-82c5-4a9a-bb7f-93b922338efe" containerName="maas-api"
Jul 29 16:00:50.478964 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.478940    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="06e122f8-82c5-4a9a-bb7f-93b922338efe" containerName="maas-api"
Jul 29 16:00:50.478964 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.478948    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5745876d-e1d0-4152-bd0a-f3f866e6732c" containerName="payload-processing"
Jul 29 16:00:50.478964 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.478953    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5745876d-e1d0-4152-bd0a-f3f866e6732c" containerName="payload-processing"
Jul 29 16:00:50.478964 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.478965    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94ef56b0-fff6-4b6e-93a4-26633ed483b3" containerName="maas-api"
Jul 29 16:00:50.478964 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.478970    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="94ef56b0-fff6-4b6e-93a4-26633ed483b3" containerName="maas-api"
Jul 29 16:00:50.479409 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.479033    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="06e122f8-82c5-4a9a-bb7f-93b922338efe" containerName="maas-api"
Jul 29 16:00:50.479409 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.479039    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="19190c95-c5cf-47fb-95a1-fab97a9e076f" containerName="payload-pre-processing"
Jul 29 16:00:50.479409 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.479045    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d56eb4b4-2c87-4a5c-b6d7-236050c8a274" containerName="payload-pre-processing"
Jul 29 16:00:50.479409 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.479051    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="38b30f24-595f-451f-92b6-2571b74a577c" containerName="payload-processing"
Jul 29 16:00:50.479409 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.479058    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="94ef56b0-fff6-4b6e-93a4-26633ed483b3" containerName="maas-api"
Jul 29 16:00:50.479409 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.479063    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5745876d-e1d0-4152-bd0a-f3f866e6732c" containerName="payload-processing"
Jul 29 16:00:50.482106 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.482081    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"
Jul 29 16:00:50.485040 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.485010    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-31add0-serving-cert\""
Jul 29 16:00:50.499679 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.499647    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"]
Jul 29 16:00:50.585963 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.585923    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-openshift-service-ca\") pod \"maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b\" (UID: \"4bcd6f3d-3fab-4239-b99a-0d2e656b4529\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"
Jul 29 16:00:50.586215 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.586055    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9jcx\" (UniqueName: \"kubernetes.io/projected/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-kube-api-access-k9jcx\") pod \"maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b\" (UID: \"4bcd6f3d-3fab-4239-b99a-0d2e656b4529\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"
Jul 29 16:00:50.586215 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.586103    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-maas-api-tls\") pod \"maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b\" (UID: \"4bcd6f3d-3fab-4239-b99a-0d2e656b4529\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"
Jul 29 16:00:50.687426 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.687382    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-openshift-service-ca\") pod \"maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b\" (UID: \"4bcd6f3d-3fab-4239-b99a-0d2e656b4529\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"
Jul 29 16:00:50.687668 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.687514    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9jcx\" (UniqueName: \"kubernetes.io/projected/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-kube-api-access-k9jcx\") pod \"maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b\" (UID: \"4bcd6f3d-3fab-4239-b99a-0d2e656b4529\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"
Jul 29 16:00:50.687668 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.687541    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-maas-api-tls\") pod \"maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b\" (UID: \"4bcd6f3d-3fab-4239-b99a-0d2e656b4529\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"
Jul 29 16:00:50.688100 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.688069    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-openshift-service-ca\") pod \"maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b\" (UID: \"4bcd6f3d-3fab-4239-b99a-0d2e656b4529\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"
Jul 29 16:00:50.690156 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.690131    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-maas-api-tls\") pod \"maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b\" (UID: \"4bcd6f3d-3fab-4239-b99a-0d2e656b4529\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"
Jul 29 16:00:50.698102 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.698074    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9jcx\" (UniqueName: \"kubernetes.io/projected/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-kube-api-access-k9jcx\") pod \"maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b\" (UID: \"4bcd6f3d-3fab-4239-b99a-0d2e656b4529\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"
Jul 29 16:00:50.795117 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.795004    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"
Jul 29 16:00:50.964602 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:50.964568    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"]
Jul 29 16:00:50.967725 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:00:50.967597    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bcd6f3d_3fab_4239_b99a_0d2e656b4529.slice/crio-6b9c151aa08c6aee3f702621b79555edca57920561913ceba19fded7a310118f WatchSource:0}: Error finding container 6b9c151aa08c6aee3f702621b79555edca57920561913ceba19fded7a310118f: Status 404 returned error can't find the container with id 6b9c151aa08c6aee3f702621b79555edca57920561913ceba19fded7a310118f
Jul 29 16:00:51.312892 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.312845    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"]
Jul 29 16:00:51.317804 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.317777    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"
Jul 29 16:00:51.321705 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.321656    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-31add0\""
Jul 29 16:00:51.322075 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.322035    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-31add0-dockercfg-jmrkd\""
Jul 29 16:00:51.329302 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.327935    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"]
Jul 29 16:00:51.351985 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.351941    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"]
Jul 29 16:00:51.355724 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.355690    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"
Jul 29 16:00:51.365849 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.365816    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"]
Jul 29 16:00:51.395518 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.395483    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7078d65a-2a03-4d46-bd33-b482467fceba-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh\" (UID: \"7078d65a-2a03-4d46-bd33-b482467fceba\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"
Jul 29 16:00:51.395715 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.395534    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pv9p\" (UniqueName: \"kubernetes.io/projected/7078d65a-2a03-4d46-bd33-b482467fceba-kube-api-access-9pv9p\") pod \"payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh\" (UID: \"7078d65a-2a03-4d46-bd33-b482467fceba\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"
Jul 29 16:00:51.395715 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.395600    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e59d93d2-8759-40c6-a84e-acf96d27e2db-plugins-config-volume\") pod \"payload-processing-e2e-api-a-31add0-69875b54db-4k54p\" (UID: \"e59d93d2-8759-40c6-a84e-acf96d27e2db\") " pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"
Jul 29 16:00:51.395715 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.395667    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fs5l\" (UniqueName: \"kubernetes.io/projected/e59d93d2-8759-40c6-a84e-acf96d27e2db-kube-api-access-6fs5l\") pod \"payload-processing-e2e-api-a-31add0-69875b54db-4k54p\" (UID: \"e59d93d2-8759-40c6-a84e-acf96d27e2db\") " pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"
Jul 29 16:00:51.445104 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.445034    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b" event={"ID":"4bcd6f3d-3fab-4239-b99a-0d2e656b4529","Type":"ContainerStarted","Data":"6b9c151aa08c6aee3f702621b79555edca57920561913ceba19fded7a310118f"}
Jul 29 16:00:51.497224 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.497180    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7078d65a-2a03-4d46-bd33-b482467fceba-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh\" (UID: \"7078d65a-2a03-4d46-bd33-b482467fceba\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"
Jul 29 16:00:51.497794 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.497236    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pv9p\" (UniqueName: \"kubernetes.io/projected/7078d65a-2a03-4d46-bd33-b482467fceba-kube-api-access-9pv9p\") pod \"payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh\" (UID: \"7078d65a-2a03-4d46-bd33-b482467fceba\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"
Jul 29 16:00:51.497794 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.497259    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e59d93d2-8759-40c6-a84e-acf96d27e2db-plugins-config-volume\") pod \"payload-processing-e2e-api-a-31add0-69875b54db-4k54p\" (UID: \"e59d93d2-8759-40c6-a84e-acf96d27e2db\") " pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"
Jul 29 16:00:51.497794 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.497283    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fs5l\" (UniqueName: \"kubernetes.io/projected/e59d93d2-8759-40c6-a84e-acf96d27e2db-kube-api-access-6fs5l\") pod \"payload-processing-e2e-api-a-31add0-69875b54db-4k54p\" (UID: \"e59d93d2-8759-40c6-a84e-acf96d27e2db\") " pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"
Jul 29 16:00:51.497971 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.497949    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7078d65a-2a03-4d46-bd33-b482467fceba-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh\" (UID: \"7078d65a-2a03-4d46-bd33-b482467fceba\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"
Jul 29 16:00:51.498025 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.498005    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e59d93d2-8759-40c6-a84e-acf96d27e2db-plugins-config-volume\") pod \"payload-processing-e2e-api-a-31add0-69875b54db-4k54p\" (UID: \"e59d93d2-8759-40c6-a84e-acf96d27e2db\") " pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"
Jul 29 16:00:51.507660 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.507605    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pv9p\" (UniqueName: \"kubernetes.io/projected/7078d65a-2a03-4d46-bd33-b482467fceba-kube-api-access-9pv9p\") pod \"payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh\" (UID: \"7078d65a-2a03-4d46-bd33-b482467fceba\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"
Jul 29 16:00:51.507660 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.507615    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fs5l\" (UniqueName: \"kubernetes.io/projected/e59d93d2-8759-40c6-a84e-acf96d27e2db-kube-api-access-6fs5l\") pod \"payload-processing-e2e-api-a-31add0-69875b54db-4k54p\" (UID: \"e59d93d2-8759-40c6-a84e-acf96d27e2db\") " pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"
Jul 29 16:00:51.634202 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.634148    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"
Jul 29 16:00:51.668979 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.668944    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"
Jul 29 16:00:51.812472 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.812435    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"]
Jul 29 16:00:51.814784 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:00:51.814723    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode59d93d2_8759_40c6_a84e_acf96d27e2db.slice/crio-ef595e8ddece9e80b730370b887f7d858910706de0150cba0e2385bed02e0a99 WatchSource:0}: Error finding container ef595e8ddece9e80b730370b887f7d858910706de0150cba0e2385bed02e0a99: Status 404 returned error can't find the container with id ef595e8ddece9e80b730370b887f7d858910706de0150cba0e2385bed02e0a99
Jul 29 16:00:51.847052 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:51.846950    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"]
Jul 29 16:00:51.850280 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:00:51.850247    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7078d65a_2a03_4d46_bd33_b482467fceba.slice/crio-e6a6105b3003dc092693cb7b665ae3e83e69deb8814bbf663b739848e4b5ba61 WatchSource:0}: Error finding container e6a6105b3003dc092693cb7b665ae3e83e69deb8814bbf663b739848e4b5ba61: Status 404 returned error can't find the container with id e6a6105b3003dc092693cb7b665ae3e83e69deb8814bbf663b739848e4b5ba61
Jul 29 16:00:52.453495 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:52.453447    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh" event={"ID":"7078d65a-2a03-4d46-bd33-b482467fceba","Type":"ContainerStarted","Data":"c6462034a207135bb1d508a8e6af11869bdfc15cc1a7dd866bf6110dcd5963d7"}
Jul 29 16:00:52.453756 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:52.453525    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"
Jul 29 16:00:52.453756 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:52.453555    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh" event={"ID":"7078d65a-2a03-4d46-bd33-b482467fceba","Type":"ContainerStarted","Data":"e6a6105b3003dc092693cb7b665ae3e83e69deb8814bbf663b739848e4b5ba61"}
Jul 29 16:00:52.455506 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:52.455469    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b" event={"ID":"4bcd6f3d-3fab-4239-b99a-0d2e656b4529","Type":"ContainerStarted","Data":"6fe5c578210abf1e46bdfc431240105f45cf8b11e44aa6d652ee6b7384e43ec0"}
Jul 29 16:00:52.455705 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:52.455537    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"
Jul 29 16:00:52.457208 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:52.457179    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p" event={"ID":"e59d93d2-8759-40c6-a84e-acf96d27e2db","Type":"ContainerStarted","Data":"18ff7bede08a98a8d02e742a06732eb3d39a8144177a94f20c337162a637d3e1"}
Jul 29 16:00:52.457371 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:52.457232    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p" event={"ID":"e59d93d2-8759-40c6-a84e-acf96d27e2db","Type":"ContainerStarted","Data":"ef595e8ddece9e80b730370b887f7d858910706de0150cba0e2385bed02e0a99"}
Jul 29 16:00:52.457371 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:52.457261    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"
Jul 29 16:00:52.470962 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:52.470906    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh" podStartSLOduration=1.470887546 podStartE2EDuration="1.470887546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:00:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:00:52.468880589 +0000 UTC m=+2985.618174493" watchObservedRunningTime="2026-07-29 16:00:52.470887546 +0000 UTC m=+2985.620181426"
Jul 29 16:00:52.485462 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:52.485405    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p" podStartSLOduration=1.485387872 podStartE2EDuration="1.485387872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:00:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:00:52.48366846 +0000 UTC m=+2985.632962361" watchObservedRunningTime="2026-07-29 16:00:52.485387872 +0000 UTC m=+2985.634681771"
Jul 29 16:00:52.502642 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:52.502578    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b" podStartSLOduration=1.954371723 podStartE2EDuration="2.502559845s" totalImagesPullingTime="548.188122ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:00:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:00:52.501452966 +0000 UTC m=+2985.650746880" watchObservedRunningTime="2026-07-29 16:00:52.502559845 +0000 UTC m=+2985.651853745"
Jul 29 16:00:58.468843 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:00:58.468806    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"
Jul 29 16:01:03.465471 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:03.465435    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"
Jul 29 16:01:03.466036 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:03.465522    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"
Jul 29 16:01:07.469729 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:07.469689    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 16:01:07.477636 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:07.477603    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 16:01:07.479827 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:07.479800    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 16:01:07.486920 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:07.486893    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 16:01:19.889047 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:19.889002    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"]
Jul 29 16:01:19.893217 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:19.893189    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"
Jul 29 16:01:19.895555 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:19.895517    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-e5a566-serving-cert\""
Jul 29 16:01:19.906516 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:19.906489    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"]
Jul 29 16:01:19.989674 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:19.989617    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3c8da16-fb28-4172-8a52-b12269472dab-openshift-service-ca\") pod \"maas-api-e2e-api-b-e5a566-6456c8456b-9l465\" (UID: \"c3c8da16-fb28-4172-8a52-b12269472dab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"
Jul 29 16:01:19.989674 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:19.989666    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkwrt\" (UniqueName: \"kubernetes.io/projected/c3c8da16-fb28-4172-8a52-b12269472dab-kube-api-access-mkwrt\") pod \"maas-api-e2e-api-b-e5a566-6456c8456b-9l465\" (UID: \"c3c8da16-fb28-4172-8a52-b12269472dab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"
Jul 29 16:01:19.990032 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:19.989699    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3c8da16-fb28-4172-8a52-b12269472dab-maas-api-tls\") pod \"maas-api-e2e-api-b-e5a566-6456c8456b-9l465\" (UID: \"c3c8da16-fb28-4172-8a52-b12269472dab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"
Jul 29 16:01:20.090922 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:20.090872    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3c8da16-fb28-4172-8a52-b12269472dab-maas-api-tls\") pod \"maas-api-e2e-api-b-e5a566-6456c8456b-9l465\" (UID: \"c3c8da16-fb28-4172-8a52-b12269472dab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"
Jul 29 16:01:20.091163 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:20.090998    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3c8da16-fb28-4172-8a52-b12269472dab-openshift-service-ca\") pod \"maas-api-e2e-api-b-e5a566-6456c8456b-9l465\" (UID: \"c3c8da16-fb28-4172-8a52-b12269472dab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"
Jul 29 16:01:20.091163 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:20.091025    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkwrt\" (UniqueName: \"kubernetes.io/projected/c3c8da16-fb28-4172-8a52-b12269472dab-kube-api-access-mkwrt\") pod \"maas-api-e2e-api-b-e5a566-6456c8456b-9l465\" (UID: \"c3c8da16-fb28-4172-8a52-b12269472dab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"
Jul 29 16:01:20.091790 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:20.091762    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3c8da16-fb28-4172-8a52-b12269472dab-openshift-service-ca\") pod \"maas-api-e2e-api-b-e5a566-6456c8456b-9l465\" (UID: \"c3c8da16-fb28-4172-8a52-b12269472dab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"
Jul 29 16:01:20.093627 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:20.093601    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3c8da16-fb28-4172-8a52-b12269472dab-maas-api-tls\") pod \"maas-api-e2e-api-b-e5a566-6456c8456b-9l465\" (UID: \"c3c8da16-fb28-4172-8a52-b12269472dab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"
Jul 29 16:01:20.105885 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:20.105857    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkwrt\" (UniqueName: \"kubernetes.io/projected/c3c8da16-fb28-4172-8a52-b12269472dab-kube-api-access-mkwrt\") pod \"maas-api-e2e-api-b-e5a566-6456c8456b-9l465\" (UID: \"c3c8da16-fb28-4172-8a52-b12269472dab\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"
Jul 29 16:01:20.205834 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:20.205714    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"
Jul 29 16:01:20.375694 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:20.375665    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"]
Jul 29 16:01:20.378596 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:01:20.378544    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3c8da16_fb28_4172_8a52_b12269472dab.slice/crio-e3c393d2c4786ea9df7373a5ee25dc1b2719316ef5ee1e4bdcaa21b346d4dff5 WatchSource:0}: Error finding container e3c393d2c4786ea9df7373a5ee25dc1b2719316ef5ee1e4bdcaa21b346d4dff5: Status 404 returned error can't find the container with id e3c393d2c4786ea9df7373a5ee25dc1b2719316ef5ee1e4bdcaa21b346d4dff5
Jul 29 16:01:20.617089 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:20.617023    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465" event={"ID":"c3c8da16-fb28-4172-8a52-b12269472dab","Type":"ContainerStarted","Data":"e3c393d2c4786ea9df7373a5ee25dc1b2719316ef5ee1e4bdcaa21b346d4dff5"}
Jul 29 16:01:20.944850 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:20.944751    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"]
Jul 29 16:01:20.948783 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:20.948756    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"
Jul 29 16:01:20.951265 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:20.951238    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-e5a566\""
Jul 29 16:01:20.951446 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:20.951238    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-e5a566-dockercfg-6pgkg\""
Jul 29 16:01:20.961220 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:20.961177    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"]
Jul 29 16:01:20.999627 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:20.999590    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzlrf\" (UniqueName: \"kubernetes.io/projected/dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba-kube-api-access-lzlrf\") pod \"payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j\" (UID: \"dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba\") " pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"
Jul 29 16:01:20.999855 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:20.999644    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j\" (UID: \"dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba\") " pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"
Jul 29 16:01:21.101111 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:21.101061    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzlrf\" (UniqueName: \"kubernetes.io/projected/dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba-kube-api-access-lzlrf\") pod \"payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j\" (UID: \"dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba\") " pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"
Jul 29 16:01:21.101395 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:21.101129    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j\" (UID: \"dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba\") " pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"
Jul 29 16:01:21.102159 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:21.102129    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba-plugins-config-volume\") pod \"payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j\" (UID: \"dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba\") " pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"
Jul 29 16:01:21.110923 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:21.110892    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzlrf\" (UniqueName: \"kubernetes.io/projected/dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba-kube-api-access-lzlrf\") pod \"payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j\" (UID: \"dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba\") " pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"
Jul 29 16:01:21.263545 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:21.263426    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"
Jul 29 16:01:21.429618 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:21.429580    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"]
Jul 29 16:01:21.432531 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:01:21.432482    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbbe6a95_2e4e_4caf_aeb0_9f5c534eecba.slice/crio-6660db257481ca4d653fc965b39c56edb4daa3438ae69d17e1ee7515fef09e81 WatchSource:0}: Error finding container 6660db257481ca4d653fc965b39c56edb4daa3438ae69d17e1ee7515fef09e81: Status 404 returned error can't find the container with id 6660db257481ca4d653fc965b39c56edb4daa3438ae69d17e1ee7515fef09e81
Jul 29 16:01:21.626427 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:21.626370    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465" event={"ID":"c3c8da16-fb28-4172-8a52-b12269472dab","Type":"ContainerStarted","Data":"60e694596581cd0c788a9e32bd05e67cb4ede88e60b2703365c9542df75900b0"}
Jul 29 16:01:21.626710 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:21.626491    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"
Jul 29 16:01:21.628431 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:21.628390    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j" event={"ID":"dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba","Type":"ContainerStarted","Data":"332f90584da99f518939f31f3d72d599835b1fcdc49d55ca80cb7818235a1f33"}
Jul 29 16:01:21.628614 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:21.628469    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"
Jul 29 16:01:21.628614 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:21.628500    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j" event={"ID":"dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba","Type":"ContainerStarted","Data":"6660db257481ca4d653fc965b39c56edb4daa3438ae69d17e1ee7515fef09e81"}
Jul 29 16:01:21.645645 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:21.645569    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465" podStartSLOduration=2.059509727 podStartE2EDuration="2.645542344s" totalImagesPullingTime="586.032617ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:01:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:01:21.645233093 +0000 UTC m=+3014.794526993" watchObservedRunningTime="2026-07-29 16:01:21.645542344 +0000 UTC m=+3014.794836246"
Jul 29 16:01:21.663364 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:21.663289    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j" podStartSLOduration=1.663269951 podStartE2EDuration="1.663269951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:01:21.66126262 +0000 UTC m=+3014.810556521" watchObservedRunningTime="2026-07-29 16:01:21.663269951 +0000 UTC m=+3014.812563850"
Jul 29 16:01:27.641355 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:27.641290    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"
Jul 29 16:01:32.637501 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:01:32.637469    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"
Jul 29 16:02:03.241917 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.241798    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"]
Jul 29 16:02:03.242562 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.242132    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b" podUID="4bcd6f3d-3fab-4239-b99a-0d2e656b4529" containerName="maas-api" containerID="cri-o://6fe5c578210abf1e46bdfc431240105f45cf8b11e44aa6d652ee6b7384e43ec0" gracePeriod=30
Jul 29 16:02:03.315921 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.315828    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"]
Jul 29 16:02:03.316654 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.316355    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p" podUID="e59d93d2-8759-40c6-a84e-acf96d27e2db" containerName="payload-processing" containerID="cri-o://18ff7bede08a98a8d02e742a06732eb3d39a8144177a94f20c337162a637d3e1" gracePeriod=30
Jul 29 16:02:03.324922 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.324615    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"]
Jul 29 16:02:03.325190 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.324958    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh" podUID="7078d65a-2a03-4d46-bd33-b482467fceba" containerName="payload-pre-processing" containerID="cri-o://c6462034a207135bb1d508a8e6af11869bdfc15cc1a7dd866bf6110dcd5963d7" gracePeriod=30
Jul 29 16:02:03.464141 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.463710    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh" podUID="7078d65a-2a03-4d46-bd33-b482467fceba" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.73:9004: connect: connection refused"
Jul 29 16:02:03.464141 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.463771    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p" podUID="e59d93d2-8759-40c6-a84e-acf96d27e2db" containerName="payload-processing" probeResult="failure" output="dial tcp 10.133.0.72:9004: connect: connection refused"
Jul 29 16:02:03.464642 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.464453    2575 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b" podUID="4bcd6f3d-3fab-4239-b99a-0d2e656b4529" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.71:8443/health\": dial tcp 10.133.0.71:8443: connect: connection refused"
Jul 29 16:02:03.599644 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.599604    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"
Jul 29 16:02:03.630972 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.630923    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9jcx\" (UniqueName: \"kubernetes.io/projected/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-kube-api-access-k9jcx\") pod \"4bcd6f3d-3fab-4239-b99a-0d2e656b4529\" (UID: \"4bcd6f3d-3fab-4239-b99a-0d2e656b4529\") "
Jul 29 16:02:03.631437 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.631409    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-openshift-service-ca\") pod \"4bcd6f3d-3fab-4239-b99a-0d2e656b4529\" (UID: \"4bcd6f3d-3fab-4239-b99a-0d2e656b4529\") "
Jul 29 16:02:03.631638 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.631619    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-maas-api-tls\") pod \"4bcd6f3d-3fab-4239-b99a-0d2e656b4529\" (UID: \"4bcd6f3d-3fab-4239-b99a-0d2e656b4529\") "
Jul 29 16:02:03.632119 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.632087    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4bcd6f3d-3fab-4239-b99a-0d2e656b4529" (UID: "4bcd6f3d-3fab-4239-b99a-0d2e656b4529"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:02:03.635008 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.634976    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-kube-api-access-k9jcx" (OuterVolumeSpecName: "kube-api-access-k9jcx") pod "4bcd6f3d-3fab-4239-b99a-0d2e656b4529" (UID: "4bcd6f3d-3fab-4239-b99a-0d2e656b4529"). InnerVolumeSpecName "kube-api-access-k9jcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:02:03.635683 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.635652    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4bcd6f3d-3fab-4239-b99a-0d2e656b4529" (UID: "4bcd6f3d-3fab-4239-b99a-0d2e656b4529"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:02:03.655406 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.655312    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"
Jul 29 16:02:03.659314 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.659289    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"
Jul 29 16:02:03.732876 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.732834    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e59d93d2-8759-40c6-a84e-acf96d27e2db-plugins-config-volume\") pod \"e59d93d2-8759-40c6-a84e-acf96d27e2db\" (UID: \"e59d93d2-8759-40c6-a84e-acf96d27e2db\") "
Jul 29 16:02:03.733088 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.732943    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fs5l\" (UniqueName: \"kubernetes.io/projected/e59d93d2-8759-40c6-a84e-acf96d27e2db-kube-api-access-6fs5l\") pod \"e59d93d2-8759-40c6-a84e-acf96d27e2db\" (UID: \"e59d93d2-8759-40c6-a84e-acf96d27e2db\") "
Jul 29 16:02:03.733088 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.732992    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pv9p\" (UniqueName: \"kubernetes.io/projected/7078d65a-2a03-4d46-bd33-b482467fceba-kube-api-access-9pv9p\") pod \"7078d65a-2a03-4d46-bd33-b482467fceba\" (UID: \"7078d65a-2a03-4d46-bd33-b482467fceba\") "
Jul 29 16:02:03.733088 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.733022    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7078d65a-2a03-4d46-bd33-b482467fceba-plugins-config-volume\") pod \"7078d65a-2a03-4d46-bd33-b482467fceba\" (UID: \"7078d65a-2a03-4d46-bd33-b482467fceba\") "
Jul 29 16:02:03.733380 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.733355    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e59d93d2-8759-40c6-a84e-acf96d27e2db-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e59d93d2-8759-40c6-a84e-acf96d27e2db" (UID: "e59d93d2-8759-40c6-a84e-acf96d27e2db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:02:03.733483 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.733458    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9jcx\" (UniqueName: \"kubernetes.io/projected/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-kube-api-access-k9jcx\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:02:03.733532 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.733496    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e59d93d2-8759-40c6-a84e-acf96d27e2db-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:02:03.733532 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.733517    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-openshift-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:02:03.733606 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.733530    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bcd6f3d-3fab-4239-b99a-0d2e656b4529-maas-api-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:02:03.733606 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.733544    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7078d65a-2a03-4d46-bd33-b482467fceba-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7078d65a-2a03-4d46-bd33-b482467fceba" (UID: "7078d65a-2a03-4d46-bd33-b482467fceba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:02:03.735599 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.735573    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e59d93d2-8759-40c6-a84e-acf96d27e2db-kube-api-access-6fs5l" (OuterVolumeSpecName: "kube-api-access-6fs5l") pod "e59d93d2-8759-40c6-a84e-acf96d27e2db" (UID: "e59d93d2-8759-40c6-a84e-acf96d27e2db"). InnerVolumeSpecName "kube-api-access-6fs5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:02:03.735698 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.735609    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7078d65a-2a03-4d46-bd33-b482467fceba-kube-api-access-9pv9p" (OuterVolumeSpecName: "kube-api-access-9pv9p") pod "7078d65a-2a03-4d46-bd33-b482467fceba" (UID: "7078d65a-2a03-4d46-bd33-b482467fceba"). InnerVolumeSpecName "kube-api-access-9pv9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:02:03.834640 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.834528    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fs5l\" (UniqueName: \"kubernetes.io/projected/e59d93d2-8759-40c6-a84e-acf96d27e2db-kube-api-access-6fs5l\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:02:03.834640 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.834559    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pv9p\" (UniqueName: \"kubernetes.io/projected/7078d65a-2a03-4d46-bd33-b482467fceba-kube-api-access-9pv9p\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:02:03.834640 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.834568    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7078d65a-2a03-4d46-bd33-b482467fceba-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:02:03.862484 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.862441    2575 generic.go:356] "Generic (PLEG): container finished" podID="4bcd6f3d-3fab-4239-b99a-0d2e656b4529" containerID="6fe5c578210abf1e46bdfc431240105f45cf8b11e44aa6d652ee6b7384e43ec0" exitCode=0
Jul 29 16:02:03.862718 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.862501    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"
Jul 29 16:02:03.862718 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.862525    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b" event={"ID":"4bcd6f3d-3fab-4239-b99a-0d2e656b4529","Type":"ContainerDied","Data":"6fe5c578210abf1e46bdfc431240105f45cf8b11e44aa6d652ee6b7384e43ec0"}
Jul 29 16:02:03.862718 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.862593    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b" event={"ID":"4bcd6f3d-3fab-4239-b99a-0d2e656b4529","Type":"ContainerDied","Data":"6b9c151aa08c6aee3f702621b79555edca57920561913ceba19fded7a310118f"}
Jul 29 16:02:03.862718 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.862621    2575 scope.go:117] "RemoveContainer" containerID="6fe5c578210abf1e46bdfc431240105f45cf8b11e44aa6d652ee6b7384e43ec0"
Jul 29 16:02:03.864215 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.864187    2575 generic.go:356] "Generic (PLEG): container finished" podID="e59d93d2-8759-40c6-a84e-acf96d27e2db" containerID="18ff7bede08a98a8d02e742a06732eb3d39a8144177a94f20c337162a637d3e1" exitCode=0
Jul 29 16:02:03.864356 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.864228    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p" event={"ID":"e59d93d2-8759-40c6-a84e-acf96d27e2db","Type":"ContainerDied","Data":"18ff7bede08a98a8d02e742a06732eb3d39a8144177a94f20c337162a637d3e1"}
Jul 29 16:02:03.864356 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.864249    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"
Jul 29 16:02:03.864356 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.864288    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p" event={"ID":"e59d93d2-8759-40c6-a84e-acf96d27e2db","Type":"ContainerDied","Data":"ef595e8ddece9e80b730370b887f7d858910706de0150cba0e2385bed02e0a99"}
Jul 29 16:02:03.865935 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.865902    2575 generic.go:356] "Generic (PLEG): container finished" podID="7078d65a-2a03-4d46-bd33-b482467fceba" containerID="c6462034a207135bb1d508a8e6af11869bdfc15cc1a7dd866bf6110dcd5963d7" exitCode=0
Jul 29 16:02:03.866048 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.865952    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"
Jul 29 16:02:03.866048 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.865957    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh" event={"ID":"7078d65a-2a03-4d46-bd33-b482467fceba","Type":"ContainerDied","Data":"c6462034a207135bb1d508a8e6af11869bdfc15cc1a7dd866bf6110dcd5963d7"}
Jul 29 16:02:03.866048 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.866003    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh" event={"ID":"7078d65a-2a03-4d46-bd33-b482467fceba","Type":"ContainerDied","Data":"e6a6105b3003dc092693cb7b665ae3e83e69deb8814bbf663b739848e4b5ba61"}
Jul 29 16:02:03.875377 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.875329    2575 scope.go:117] "RemoveContainer" containerID="6fe5c578210abf1e46bdfc431240105f45cf8b11e44aa6d652ee6b7384e43ec0"
Jul 29 16:02:03.875752 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:02:03.875728    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fe5c578210abf1e46bdfc431240105f45cf8b11e44aa6d652ee6b7384e43ec0\": container with ID starting with 6fe5c578210abf1e46bdfc431240105f45cf8b11e44aa6d652ee6b7384e43ec0 not found: ID does not exist" containerID="6fe5c578210abf1e46bdfc431240105f45cf8b11e44aa6d652ee6b7384e43ec0"
Jul 29 16:02:03.875843 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.875762    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fe5c578210abf1e46bdfc431240105f45cf8b11e44aa6d652ee6b7384e43ec0"} err="failed to get container status \"6fe5c578210abf1e46bdfc431240105f45cf8b11e44aa6d652ee6b7384e43ec0\": rpc error: code = NotFound desc = could not find container \"6fe5c578210abf1e46bdfc431240105f45cf8b11e44aa6d652ee6b7384e43ec0\": container with ID starting with 6fe5c578210abf1e46bdfc431240105f45cf8b11e44aa6d652ee6b7384e43ec0 not found: ID does not exist"
Jul 29 16:02:03.875843 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.875782    2575 scope.go:117] "RemoveContainer" containerID="18ff7bede08a98a8d02e742a06732eb3d39a8144177a94f20c337162a637d3e1"
Jul 29 16:02:03.885154 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.885128    2575 scope.go:117] "RemoveContainer" containerID="18ff7bede08a98a8d02e742a06732eb3d39a8144177a94f20c337162a637d3e1"
Jul 29 16:02:03.885564 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:02:03.885540    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18ff7bede08a98a8d02e742a06732eb3d39a8144177a94f20c337162a637d3e1\": container with ID starting with 18ff7bede08a98a8d02e742a06732eb3d39a8144177a94f20c337162a637d3e1 not found: ID does not exist" containerID="18ff7bede08a98a8d02e742a06732eb3d39a8144177a94f20c337162a637d3e1"
Jul 29 16:02:03.885677 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.885578    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18ff7bede08a98a8d02e742a06732eb3d39a8144177a94f20c337162a637d3e1"} err="failed to get container status \"18ff7bede08a98a8d02e742a06732eb3d39a8144177a94f20c337162a637d3e1\": rpc error: code = NotFound desc = could not find container \"18ff7bede08a98a8d02e742a06732eb3d39a8144177a94f20c337162a637d3e1\": container with ID starting with 18ff7bede08a98a8d02e742a06732eb3d39a8144177a94f20c337162a637d3e1 not found: ID does not exist"
Jul 29 16:02:03.885677 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.885606    2575 scope.go:117] "RemoveContainer" containerID="c6462034a207135bb1d508a8e6af11869bdfc15cc1a7dd866bf6110dcd5963d7"
Jul 29 16:02:03.895753 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.895687    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"]
Jul 29 16:02:03.897161 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.897126    2575 scope.go:117] "RemoveContainer" containerID="c6462034a207135bb1d508a8e6af11869bdfc15cc1a7dd866bf6110dcd5963d7"
Jul 29 16:02:03.897596 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:02:03.897575    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6462034a207135bb1d508a8e6af11869bdfc15cc1a7dd866bf6110dcd5963d7\": container with ID starting with c6462034a207135bb1d508a8e6af11869bdfc15cc1a7dd866bf6110dcd5963d7 not found: ID does not exist" containerID="c6462034a207135bb1d508a8e6af11869bdfc15cc1a7dd866bf6110dcd5963d7"
Jul 29 16:02:03.897694 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.897611    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6462034a207135bb1d508a8e6af11869bdfc15cc1a7dd866bf6110dcd5963d7"} err="failed to get container status \"c6462034a207135bb1d508a8e6af11869bdfc15cc1a7dd866bf6110dcd5963d7\": rpc error: code = NotFound desc = could not find container \"c6462034a207135bb1d508a8e6af11869bdfc15cc1a7dd866bf6110dcd5963d7\": container with ID starting with c6462034a207135bb1d508a8e6af11869bdfc15cc1a7dd866bf6110dcd5963d7 not found: ID does not exist"
Jul 29 16:02:03.900905 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.900874    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-31add0-6b75d7558d-b4n8b"]
Jul 29 16:02:03.914165 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.914127    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"]
Jul 29 16:02:03.918332 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.918287    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-31add0-69875b54db-4k54p"]
Jul 29 16:02:03.930721 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.930685    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"]
Jul 29 16:02:03.936071 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:03.936034    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-31add0-5fb889dc4d-6ngkh"]
Jul 29 16:02:05.296535 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:05.296495    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bcd6f3d-3fab-4239-b99a-0d2e656b4529" path="/var/lib/kubelet/pods/4bcd6f3d-3fab-4239-b99a-0d2e656b4529/volumes"
Jul 29 16:02:05.297006 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:05.296882    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7078d65a-2a03-4d46-bd33-b482467fceba" path="/var/lib/kubelet/pods/7078d65a-2a03-4d46-bd33-b482467fceba/volumes"
Jul 29 16:02:05.297211 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:05.297195    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e59d93d2-8759-40c6-a84e-acf96d27e2db" path="/var/lib/kubelet/pods/e59d93d2-8759-40c6-a84e-acf96d27e2db/volumes"
Jul 29 16:02:10.786347 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:10.786265    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2"]
Jul 29 16:02:10.786919 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:10.786790    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7078d65a-2a03-4d46-bd33-b482467fceba" containerName="payload-pre-processing"
Jul 29 16:02:10.786919 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:10.786806    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="7078d65a-2a03-4d46-bd33-b482467fceba" containerName="payload-pre-processing"
Jul 29 16:02:10.786919 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:10.786820    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bcd6f3d-3fab-4239-b99a-0d2e656b4529" containerName="maas-api"
Jul 29 16:02:10.786919 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:10.786825    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bcd6f3d-3fab-4239-b99a-0d2e656b4529" containerName="maas-api"
Jul 29 16:02:10.786919 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:10.786846    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e59d93d2-8759-40c6-a84e-acf96d27e2db" containerName="payload-processing"
Jul 29 16:02:10.786919 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:10.786853    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e59d93d2-8759-40c6-a84e-acf96d27e2db" containerName="payload-processing"
Jul 29 16:02:10.787260 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:10.786931    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bcd6f3d-3fab-4239-b99a-0d2e656b4529" containerName="maas-api"
Jul 29 16:02:10.787260 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:10.786947    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e59d93d2-8759-40c6-a84e-acf96d27e2db" containerName="payload-processing"
Jul 29 16:02:10.787260 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:10.786958    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="7078d65a-2a03-4d46-bd33-b482467fceba" containerName="payload-pre-processing"
Jul 29 16:02:10.792807 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:10.792779    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2"
Jul 29 16:02:10.795473 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:10.795441    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgrq5\""
Jul 29 16:02:10.797184 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:10.797149    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2"]
Jul 29 16:02:10.907731 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:10.907681    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85e94c40-d67a-4107-8b14-abe5b3bb581d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2\" (UID: \"85e94c40-d67a-4107-8b14-abe5b3bb581d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2"
Jul 29 16:02:11.008886 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:11.008839    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85e94c40-d67a-4107-8b14-abe5b3bb581d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2\" (UID: \"85e94c40-d67a-4107-8b14-abe5b3bb581d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2"
Jul 29 16:02:11.009667 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:11.009640    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85e94c40-d67a-4107-8b14-abe5b3bb581d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2\" (UID: \"85e94c40-d67a-4107-8b14-abe5b3bb581d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2"
Jul 29 16:02:11.105079 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:11.104973    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2"
Jul 29 16:02:11.266925 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:11.266893    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2"]
Jul 29 16:02:11.270492 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:02:11.270453    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85e94c40_d67a_4107_8b14_abe5b3bb581d.slice/crio-a6e98fef501afb5ec4b2e5bbfa6d6cc3a42f757f538f58910b5d2fa454f11fb9 WatchSource:0}: Error finding container a6e98fef501afb5ec4b2e5bbfa6d6cc3a42f757f538f58910b5d2fa454f11fb9: Status 404 returned error can't find the container with id a6e98fef501afb5ec4b2e5bbfa6d6cc3a42f757f538f58910b5d2fa454f11fb9
Jul 29 16:02:11.914278 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:11.914231    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2" event={"ID":"85e94c40-d67a-4107-8b14-abe5b3bb581d","Type":"ContainerStarted","Data":"a6e98fef501afb5ec4b2e5bbfa6d6cc3a42f757f538f58910b5d2fa454f11fb9"}
Jul 29 16:02:14.935752 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:14.935707    2575 generic.go:356] "Generic (PLEG): container finished" podID="85e94c40-d67a-4107-8b14-abe5b3bb581d" containerID="05b5f92f75579fb5b0bf79ba18a3fcb09793f7d650f09a660018eb0446dd9e45" exitCode=0
Jul 29 16:02:14.936308 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:14.935793    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2" event={"ID":"85e94c40-d67a-4107-8b14-abe5b3bb581d","Type":"ContainerDied","Data":"05b5f92f75579fb5b0bf79ba18a3fcb09793f7d650f09a660018eb0446dd9e45"}
Jul 29 16:02:16.092036 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:16.091998    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2"
Jul 29 16:02:16.166461 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:16.166408    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85e94c40-d67a-4107-8b14-abe5b3bb581d-maas-api-service-ca\") pod \"85e94c40-d67a-4107-8b14-abe5b3bb581d\" (UID: \"85e94c40-d67a-4107-8b14-abe5b3bb581d\") "
Jul 29 16:02:16.166913 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:16.166887    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85e94c40-d67a-4107-8b14-abe5b3bb581d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "85e94c40-d67a-4107-8b14-abe5b3bb581d" (UID: "85e94c40-d67a-4107-8b14-abe5b3bb581d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:02:16.267420 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:16.267298    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85e94c40-d67a-4107-8b14-abe5b3bb581d-maas-api-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:02:16.948999 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:16.948959    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2"
Jul 29 16:02:16.949222 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:16.948960    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2" event={"ID":"85e94c40-d67a-4107-8b14-abe5b3bb581d","Type":"ContainerDied","Data":"a6e98fef501afb5ec4b2e5bbfa6d6cc3a42f757f538f58910b5d2fa454f11fb9"}
Jul 29 16:02:16.949222 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:16.949093    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a6e98fef501afb5ec4b2e5bbfa6d6cc3a42f757f538f58910b5d2fa454f11fb9"
Jul 29 16:02:26.312148 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.312101    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"]
Jul 29 16:02:26.313106 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.313062    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465" podUID="c3c8da16-fb28-4172-8a52-b12269472dab" containerName="maas-api" containerID="cri-o://60e694596581cd0c788a9e32bd05e67cb4ede88e60b2703365c9542df75900b0" gracePeriod=30
Jul 29 16:02:26.362939 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:02:26.362895    2575 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3c8da16_fb28_4172_8a52_b12269472dab.slice/crio-60e694596581cd0c788a9e32bd05e67cb4ede88e60b2703365c9542df75900b0.scope\": RecentStats: unable to find data in memory cache]"
Jul 29 16:02:26.387746 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.387695    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"]
Jul 29 16:02:26.388048 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.387984    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j" podUID="dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba" containerName="payload-processing" containerID="cri-o://332f90584da99f518939f31f3d72d599835b1fcdc49d55ca80cb7818235a1f33" gracePeriod=30
Jul 29 16:02:26.566163 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.566072    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2"]
Jul 29 16:02:26.570678 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.570605    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-e5a566-lwfx2"]
Jul 29 16:02:26.654554 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.654526    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"
Jul 29 16:02:26.690031 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.690000    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"
Jul 29 16:02:26.770744 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.770714    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3c8da16-fb28-4172-8a52-b12269472dab-maas-api-tls\") pod \"c3c8da16-fb28-4172-8a52-b12269472dab\" (UID: \"c3c8da16-fb28-4172-8a52-b12269472dab\") "
Jul 29 16:02:26.770744 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.770749    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkwrt\" (UniqueName: \"kubernetes.io/projected/c3c8da16-fb28-4172-8a52-b12269472dab-kube-api-access-mkwrt\") pod \"c3c8da16-fb28-4172-8a52-b12269472dab\" (UID: \"c3c8da16-fb28-4172-8a52-b12269472dab\") "
Jul 29 16:02:26.770952 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.770774    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba-plugins-config-volume\") pod \"dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba\" (UID: \"dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba\") "
Jul 29 16:02:26.770952 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.770861    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3c8da16-fb28-4172-8a52-b12269472dab-openshift-service-ca\") pod \"c3c8da16-fb28-4172-8a52-b12269472dab\" (UID: \"c3c8da16-fb28-4172-8a52-b12269472dab\") "
Jul 29 16:02:26.770952 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.770889    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzlrf\" (UniqueName: \"kubernetes.io/projected/dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba-kube-api-access-lzlrf\") pod \"dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba\" (UID: \"dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba\") "
Jul 29 16:02:26.771273 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.771218    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba" (UID: "dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:02:26.771273 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.771231    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3c8da16-fb28-4172-8a52-b12269472dab-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c3c8da16-fb28-4172-8a52-b12269472dab" (UID: "c3c8da16-fb28-4172-8a52-b12269472dab"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:02:26.774010 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.773943    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3c8da16-fb28-4172-8a52-b12269472dab-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c3c8da16-fb28-4172-8a52-b12269472dab" (UID: "c3c8da16-fb28-4172-8a52-b12269472dab"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:02:26.774010 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.773943    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3c8da16-fb28-4172-8a52-b12269472dab-kube-api-access-mkwrt" (OuterVolumeSpecName: "kube-api-access-mkwrt") pod "c3c8da16-fb28-4172-8a52-b12269472dab" (UID: "c3c8da16-fb28-4172-8a52-b12269472dab"). InnerVolumeSpecName "kube-api-access-mkwrt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:02:26.774278 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.773975    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba-kube-api-access-lzlrf" (OuterVolumeSpecName: "kube-api-access-lzlrf") pod "dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba" (UID: "dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba"). InnerVolumeSpecName "kube-api-access-lzlrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:02:26.872228 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.872186    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzlrf\" (UniqueName: \"kubernetes.io/projected/dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba-kube-api-access-lzlrf\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:02:26.872228 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.872220    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3c8da16-fb28-4172-8a52-b12269472dab-maas-api-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:02:26.872228 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.872230    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkwrt\" (UniqueName: \"kubernetes.io/projected/c3c8da16-fb28-4172-8a52-b12269472dab-kube-api-access-mkwrt\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:02:26.872529 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.872238    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:02:26.872529 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:26.872248    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3c8da16-fb28-4172-8a52-b12269472dab-openshift-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:02:27.004923 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.004885    2575 generic.go:356] "Generic (PLEG): container finished" podID="dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba" containerID="332f90584da99f518939f31f3d72d599835b1fcdc49d55ca80cb7818235a1f33" exitCode=0
Jul 29 16:02:27.005144 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.004937    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"
Jul 29 16:02:27.005144 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.004970    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j" event={"ID":"dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba","Type":"ContainerDied","Data":"332f90584da99f518939f31f3d72d599835b1fcdc49d55ca80cb7818235a1f33"}
Jul 29 16:02:27.005144 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.005030    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j" event={"ID":"dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba","Type":"ContainerDied","Data":"6660db257481ca4d653fc965b39c56edb4daa3438ae69d17e1ee7515fef09e81"}
Jul 29 16:02:27.005144 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.005059    2575 scope.go:117] "RemoveContainer" containerID="332f90584da99f518939f31f3d72d599835b1fcdc49d55ca80cb7818235a1f33"
Jul 29 16:02:27.006690 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.006659    2575 generic.go:356] "Generic (PLEG): container finished" podID="c3c8da16-fb28-4172-8a52-b12269472dab" containerID="60e694596581cd0c788a9e32bd05e67cb4ede88e60b2703365c9542df75900b0" exitCode=0
Jul 29 16:02:27.006837 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.006703    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"
Jul 29 16:02:27.006914 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.006700    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465" event={"ID":"c3c8da16-fb28-4172-8a52-b12269472dab","Type":"ContainerDied","Data":"60e694596581cd0c788a9e32bd05e67cb4ede88e60b2703365c9542df75900b0"}
Jul 29 16:02:27.006976 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.006949    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465" event={"ID":"c3c8da16-fb28-4172-8a52-b12269472dab","Type":"ContainerDied","Data":"e3c393d2c4786ea9df7373a5ee25dc1b2719316ef5ee1e4bdcaa21b346d4dff5"}
Jul 29 16:02:27.015384 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.015358    2575 scope.go:117] "RemoveContainer" containerID="332f90584da99f518939f31f3d72d599835b1fcdc49d55ca80cb7818235a1f33"
Jul 29 16:02:27.015736 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:02:27.015711    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"332f90584da99f518939f31f3d72d599835b1fcdc49d55ca80cb7818235a1f33\": container with ID starting with 332f90584da99f518939f31f3d72d599835b1fcdc49d55ca80cb7818235a1f33 not found: ID does not exist" containerID="332f90584da99f518939f31f3d72d599835b1fcdc49d55ca80cb7818235a1f33"
Jul 29 16:02:27.015844 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.015747    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"332f90584da99f518939f31f3d72d599835b1fcdc49d55ca80cb7818235a1f33"} err="failed to get container status \"332f90584da99f518939f31f3d72d599835b1fcdc49d55ca80cb7818235a1f33\": rpc error: code = NotFound desc = could not find container \"332f90584da99f518939f31f3d72d599835b1fcdc49d55ca80cb7818235a1f33\": container with ID starting with 332f90584da99f518939f31f3d72d599835b1fcdc49d55ca80cb7818235a1f33 not found: ID does not exist"
Jul 29 16:02:27.015844 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.015770    2575 scope.go:117] "RemoveContainer" containerID="60e694596581cd0c788a9e32bd05e67cb4ede88e60b2703365c9542df75900b0"
Jul 29 16:02:27.030239 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.030072    2575 scope.go:117] "RemoveContainer" containerID="60e694596581cd0c788a9e32bd05e67cb4ede88e60b2703365c9542df75900b0"
Jul 29 16:02:27.030542 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:02:27.030519    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60e694596581cd0c788a9e32bd05e67cb4ede88e60b2703365c9542df75900b0\": container with ID starting with 60e694596581cd0c788a9e32bd05e67cb4ede88e60b2703365c9542df75900b0 not found: ID does not exist" containerID="60e694596581cd0c788a9e32bd05e67cb4ede88e60b2703365c9542df75900b0"
Jul 29 16:02:27.030630 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.030549    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60e694596581cd0c788a9e32bd05e67cb4ede88e60b2703365c9542df75900b0"} err="failed to get container status \"60e694596581cd0c788a9e32bd05e67cb4ede88e60b2703365c9542df75900b0\": rpc error: code = NotFound desc = could not find container \"60e694596581cd0c788a9e32bd05e67cb4ede88e60b2703365c9542df75900b0\": container with ID starting with 60e694596581cd0c788a9e32bd05e67cb4ede88e60b2703365c9542df75900b0 not found: ID does not exist"
Jul 29 16:02:27.032329 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.032282    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"]
Jul 29 16:02:27.035699 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.035629    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-e5a566-666fd844cd-xz94j"]
Jul 29 16:02:27.047595 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.047558    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"]
Jul 29 16:02:27.053484 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.053449    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e5a566-6456c8456b-9l465"]
Jul 29 16:02:27.295693 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.295599    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85e94c40-d67a-4107-8b14-abe5b3bb581d" path="/var/lib/kubelet/pods/85e94c40-d67a-4107-8b14-abe5b3bb581d/volumes"
Jul 29 16:02:27.296010 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.295995    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3c8da16-fb28-4172-8a52-b12269472dab" path="/var/lib/kubelet/pods/c3c8da16-fb28-4172-8a52-b12269472dab/volumes"
Jul 29 16:02:27.296331 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:27.296308    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba" path="/var/lib/kubelet/pods/dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba/volumes"
Jul 29 16:02:37.989775 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:37.989738    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"]
Jul 29 16:02:37.990486 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:37.990427    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3c8da16-fb28-4172-8a52-b12269472dab" containerName="maas-api"
Jul 29 16:02:37.990486 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:37.990452    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3c8da16-fb28-4172-8a52-b12269472dab" containerName="maas-api"
Jul 29 16:02:37.990619 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:37.990493    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba" containerName="payload-processing"
Jul 29 16:02:37.990619 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:37.990504    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba" containerName="payload-processing"
Jul 29 16:02:37.990619 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:37.990524    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85e94c40-d67a-4107-8b14-abe5b3bb581d" containerName="revoke-keys"
Jul 29 16:02:37.990619 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:37.990534    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="85e94c40-d67a-4107-8b14-abe5b3bb581d" containerName="revoke-keys"
Jul 29 16:02:37.990876 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:37.990644    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3c8da16-fb28-4172-8a52-b12269472dab" containerName="maas-api"
Jul 29 16:02:37.990876 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:37.990659    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="85e94c40-d67a-4107-8b14-abe5b3bb581d" containerName="revoke-keys"
Jul 29 16:02:37.990876 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:37.990671    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbbe6a95-2e4e-4caf-aeb0-9f5c534eecba" containerName="payload-processing"
Jul 29 16:02:37.995462 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:37.995429    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"
Jul 29 16:02:37.998511 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:37.998482    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-02add2-serving-cert\""
Jul 29 16:02:38.007861 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.007787    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"]
Jul 29 16:02:38.079587 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.079554    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa588d7c-e56d-4039-8643-fcaec33edc80-maas-api-tls\") pod \"maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5\" (UID: \"aa588d7c-e56d-4039-8643-fcaec33edc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"
Jul 29 16:02:38.079826 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.079614    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b4dn\" (UniqueName: \"kubernetes.io/projected/aa588d7c-e56d-4039-8643-fcaec33edc80-kube-api-access-6b4dn\") pod \"maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5\" (UID: \"aa588d7c-e56d-4039-8643-fcaec33edc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"
Jul 29 16:02:38.079826 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.079649    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa588d7c-e56d-4039-8643-fcaec33edc80-openshift-service-ca\") pod \"maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5\" (UID: \"aa588d7c-e56d-4039-8643-fcaec33edc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"
Jul 29 16:02:38.180750 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.180703    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa588d7c-e56d-4039-8643-fcaec33edc80-maas-api-tls\") pod \"maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5\" (UID: \"aa588d7c-e56d-4039-8643-fcaec33edc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"
Jul 29 16:02:38.181021 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.180759    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b4dn\" (UniqueName: \"kubernetes.io/projected/aa588d7c-e56d-4039-8643-fcaec33edc80-kube-api-access-6b4dn\") pod \"maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5\" (UID: \"aa588d7c-e56d-4039-8643-fcaec33edc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"
Jul 29 16:02:38.181021 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.180785    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa588d7c-e56d-4039-8643-fcaec33edc80-openshift-service-ca\") pod \"maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5\" (UID: \"aa588d7c-e56d-4039-8643-fcaec33edc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"
Jul 29 16:02:38.181021 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:02:38.180891    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-serving-cert: secret "maas-api-e2e-shared-a-02add2-serving-cert" not found
Jul 29 16:02:38.181021 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:02:38.180987    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa588d7c-e56d-4039-8643-fcaec33edc80-maas-api-tls podName:aa588d7c-e56d-4039-8643-fcaec33edc80 nodeName:}" failed. No retries permitted until 2026-07-29 16:02:38.680961736 +0000 UTC m=+3091.830255614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aa588d7c-e56d-4039-8643-fcaec33edc80-maas-api-tls") pod "maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5" (UID: "aa588d7c-e56d-4039-8643-fcaec33edc80") : secret "maas-api-e2e-shared-a-02add2-serving-cert" not found
Jul 29 16:02:38.181456 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.181440    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa588d7c-e56d-4039-8643-fcaec33edc80-openshift-service-ca\") pod \"maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5\" (UID: \"aa588d7c-e56d-4039-8643-fcaec33edc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"
Jul 29 16:02:38.191700 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.191664    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b4dn\" (UniqueName: \"kubernetes.io/projected/aa588d7c-e56d-4039-8643-fcaec33edc80-kube-api-access-6b4dn\") pod \"maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5\" (UID: \"aa588d7c-e56d-4039-8643-fcaec33edc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"
Jul 29 16:02:38.686821 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.686774    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa588d7c-e56d-4039-8643-fcaec33edc80-maas-api-tls\") pod \"maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5\" (UID: \"aa588d7c-e56d-4039-8643-fcaec33edc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"
Jul 29 16:02:38.689857 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.689821    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa588d7c-e56d-4039-8643-fcaec33edc80-maas-api-tls\") pod \"maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5\" (UID: \"aa588d7c-e56d-4039-8643-fcaec33edc80\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"
Jul 29 16:02:38.910433 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.910386    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"
Jul 29 16:02:38.929947 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.929910    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"]
Jul 29 16:02:38.935003 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.934937    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"
Jul 29 16:02:38.938161 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.938064    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-02add2\""
Jul 29 16:02:38.938161 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.938116    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-02add2-dockercfg-2d9rx\""
Jul 29 16:02:38.958643 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.958605    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"]
Jul 29 16:02:38.990633 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.990431    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94hlf\" (UniqueName: \"kubernetes.io/projected/6140907c-df02-4bd7-8267-116fc800f6ae-kube-api-access-94hlf\") pod \"payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl\" (UID: \"6140907c-df02-4bd7-8267-116fc800f6ae\") " pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"
Jul 29 16:02:38.990633 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:38.990498    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6140907c-df02-4bd7-8267-116fc800f6ae-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl\" (UID: \"6140907c-df02-4bd7-8267-116fc800f6ae\") " pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"
Jul 29 16:02:39.088475 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:39.088447    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"]
Jul 29 16:02:39.091195 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:39.091155    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94hlf\" (UniqueName: \"kubernetes.io/projected/6140907c-df02-4bd7-8267-116fc800f6ae-kube-api-access-94hlf\") pod \"payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl\" (UID: \"6140907c-df02-4bd7-8267-116fc800f6ae\") " pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"
Jul 29 16:02:39.091411 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:39.091214    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6140907c-df02-4bd7-8267-116fc800f6ae-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl\" (UID: \"6140907c-df02-4bd7-8267-116fc800f6ae\") " pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"
Jul 29 16:02:39.091411 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:02:39.091249    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa588d7c_e56d_4039_8643_fcaec33edc80.slice/crio-4cc7c3ea8a824f0dcbe1e76f8c4ce8e91b7554220e2b1f4324f8041ae8ff628c WatchSource:0}: Error finding container 4cc7c3ea8a824f0dcbe1e76f8c4ce8e91b7554220e2b1f4324f8041ae8ff628c: Status 404 returned error can't find the container with id 4cc7c3ea8a824f0dcbe1e76f8c4ce8e91b7554220e2b1f4324f8041ae8ff628c
Jul 29 16:02:39.092079 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:39.092019    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6140907c-df02-4bd7-8267-116fc800f6ae-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl\" (UID: \"6140907c-df02-4bd7-8267-116fc800f6ae\") " pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"
Jul 29 16:02:39.108363 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:39.108310    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94hlf\" (UniqueName: \"kubernetes.io/projected/6140907c-df02-4bd7-8267-116fc800f6ae-kube-api-access-94hlf\") pod \"payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl\" (UID: \"6140907c-df02-4bd7-8267-116fc800f6ae\") " pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"
Jul 29 16:02:39.254011 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:39.253898    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"
Jul 29 16:02:39.414817 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:39.414778    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"]
Jul 29 16:02:39.417369 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:02:39.417305    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6140907c_df02_4bd7_8267_116fc800f6ae.slice/crio-caf652891237b293c17c92e9cc3085ee0ddcbcf6e548246581ddc3969c2af9ef WatchSource:0}: Error finding container caf652891237b293c17c92e9cc3085ee0ddcbcf6e548246581ddc3969c2af9ef: Status 404 returned error can't find the container with id caf652891237b293c17c92e9cc3085ee0ddcbcf6e548246581ddc3969c2af9ef
Jul 29 16:02:40.077302 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:40.077261    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl" event={"ID":"6140907c-df02-4bd7-8267-116fc800f6ae","Type":"ContainerStarted","Data":"325a888b7d2e695b5fad8fa4de71aec900ff44ad1f2cda54b3e5f8b71f9fea0e"}
Jul 29 16:02:40.077818 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:40.077358    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"
Jul 29 16:02:40.077818 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:40.077386    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl" event={"ID":"6140907c-df02-4bd7-8267-116fc800f6ae","Type":"ContainerStarted","Data":"caf652891237b293c17c92e9cc3085ee0ddcbcf6e548246581ddc3969c2af9ef"}
Jul 29 16:02:40.079034 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:40.078996    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5" event={"ID":"aa588d7c-e56d-4039-8643-fcaec33edc80","Type":"ContainerStarted","Data":"4d57605fe01719ba682c5300ded05e6b7aea7264802d5730ea74cbb0c7f78954"}
Jul 29 16:02:40.079217 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:40.079050    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5" event={"ID":"aa588d7c-e56d-4039-8643-fcaec33edc80","Type":"ContainerStarted","Data":"4cc7c3ea8a824f0dcbe1e76f8c4ce8e91b7554220e2b1f4324f8041ae8ff628c"}
Jul 29 16:02:40.079217 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:40.079079    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"
Jul 29 16:02:40.093270 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:40.093172    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl" podStartSLOduration=2.093148917 podStartE2EDuration="2.093148917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:02:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:02:40.092901272 +0000 UTC m=+3093.242195173" watchObservedRunningTime="2026-07-29 16:02:40.093148917 +0000 UTC m=+3093.242442818"
Jul 29 16:02:40.110392 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:40.110315    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5" podStartSLOduration=2.724751892 podStartE2EDuration="3.110296549s" totalImagesPullingTime="385.544657ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:02:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:02:40.109214011 +0000 UTC m=+3093.258507913" watchObservedRunningTime="2026-07-29 16:02:40.110296549 +0000 UTC m=+3093.259590448"
Jul 29 16:02:46.090798 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:46.090762    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"
Jul 29 16:02:51.087087 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:02:51.087049    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"
Jul 29 16:03:07.239916 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:07.239881    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"]
Jul 29 16:03:07.243797 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:07.243773    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"
Jul 29 16:03:07.246341 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:07.246282    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-7e5daf-serving-cert\""
Jul 29 16:03:07.255102 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:07.255075    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"]
Jul 29 16:03:07.387082 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:07.387044    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-maas-api-tls\") pod \"maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5\" (UID: \"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"
Jul 29 16:03:07.387288 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:07.387094    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5\" (UID: \"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"
Jul 29 16:03:07.387288 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:07.387192    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2chj\" (UniqueName: \"kubernetes.io/projected/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-kube-api-access-n2chj\") pod \"maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5\" (UID: \"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"
Jul 29 16:03:07.488350 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:07.488281    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-maas-api-tls\") pod \"maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5\" (UID: \"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"
Jul 29 16:03:07.488565 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:07.488371    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5\" (UID: \"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"
Jul 29 16:03:07.488565 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:07.488452    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2chj\" (UniqueName: \"kubernetes.io/projected/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-kube-api-access-n2chj\") pod \"maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5\" (UID: \"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"
Jul 29 16:03:07.488565 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:03:07.488464    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-serving-cert: secret "maas-api-e2e-shared-b-7e5daf-serving-cert" not found
Jul 29 16:03:07.488565 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:03:07.488549    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-maas-api-tls podName:c3f618fa-f8f8-453c-8328-e65b1fb0b4cb nodeName:}" failed. No retries permitted until 2026-07-29 16:03:07.988529322 +0000 UTC m=+3121.137823199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-maas-api-tls") pod "maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5" (UID: "c3f618fa-f8f8-453c-8328-e65b1fb0b4cb") : secret "maas-api-e2e-shared-b-7e5daf-serving-cert" not found
Jul 29 16:03:07.489041 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:07.489022    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5\" (UID: \"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"
Jul 29 16:03:07.498118 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:07.498050    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2chj\" (UniqueName: \"kubernetes.io/projected/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-kube-api-access-n2chj\") pod \"maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5\" (UID: \"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"
Jul 29 16:03:07.994224 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:07.994150    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-maas-api-tls\") pod \"maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5\" (UID: \"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"
Jul 29 16:03:07.996961 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:07.996926    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-maas-api-tls\") pod \"maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5\" (UID: \"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"
Jul 29 16:03:08.156435 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:08.156397    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"
Jul 29 16:03:08.324976 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:08.324883    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"]
Jul 29 16:03:08.329742 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:03:08.329713    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3f618fa_f8f8_453c_8328_e65b1fb0b4cb.slice/crio-4c7e9fb3619cac33bbb0c0b1a145b069b99b0326708f802c7665d50c1e728ce0 WatchSource:0}: Error finding container 4c7e9fb3619cac33bbb0c0b1a145b069b99b0326708f802c7665d50c1e728ce0: Status 404 returned error can't find the container with id 4c7e9fb3619cac33bbb0c0b1a145b069b99b0326708f802c7665d50c1e728ce0
Jul 29 16:03:08.331284 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:08.331263    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 16:03:09.244256 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:09.244221    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5" event={"ID":"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb","Type":"ContainerStarted","Data":"9f6a3fa466ee9924f01fc025c7bccbbb406cc2616aee3d90a631e1819bcb4798"}
Jul 29 16:03:09.244592 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:09.244278    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5" event={"ID":"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb","Type":"ContainerStarted","Data":"4c7e9fb3619cac33bbb0c0b1a145b069b99b0326708f802c7665d50c1e728ce0"}
Jul 29 16:03:09.244592 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:09.244310    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"
Jul 29 16:03:09.263778 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:09.263727    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5" podStartSLOduration=1.82699589 podStartE2EDuration="2.263706983s" totalImagesPullingTime="436.711093ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:03:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:03:09.263314644 +0000 UTC m=+3122.412608545" watchObservedRunningTime="2026-07-29 16:03:09.263706983 +0000 UTC m=+3122.413000883"
Jul 29 16:03:15.255105 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:03:15.255071    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"
Jul 29 16:06:07.500652 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:06:07.500625    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 16:06:07.507947 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:06:07.507924    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 16:06:07.511224 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:06:07.511198    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 16:06:07.517629 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:06:07.517614    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 16:09:07.754056 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:09:07.754024    2575 scope.go:117] "RemoveContainer" containerID="05b5f92f75579fb5b0bf79ba18a3fcb09793f7d650f09a660018eb0446dd9e45"
Jul 29 16:11:07.531353 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:11:07.531308    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 16:11:07.541835 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:11:07.541811    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 16:11:07.546617 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:11:07.546594    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 16:11:07.553590 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:11:07.553571    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 16:14:30.700867 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:30.700831    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"]
Jul 29 16:14:30.704559 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:30.704534    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"
Jul 29 16:14:30.706999 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:30.706977    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-6fccfc-serving-cert\""
Jul 29 16:14:30.715762 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:30.715737    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"]
Jul 29 16:14:30.809809 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:30.809776    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxjx2\" (UniqueName: \"kubernetes.io/projected/c2c4c1d2-a01a-419a-b9f2-6e049a868084-kube-api-access-qxjx2\") pod \"maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4\" (UID: \"c2c4c1d2-a01a-419a-b9f2-6e049a868084\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"
Jul 29 16:14:30.809991 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:30.809865    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2c4c1d2-a01a-419a-b9f2-6e049a868084-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4\" (UID: \"c2c4c1d2-a01a-419a-b9f2-6e049a868084\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"
Jul 29 16:14:30.809991 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:30.809887    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2c4c1d2-a01a-419a-b9f2-6e049a868084-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4\" (UID: \"c2c4c1d2-a01a-419a-b9f2-6e049a868084\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"
Jul 29 16:14:30.910560 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:30.910527    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxjx2\" (UniqueName: \"kubernetes.io/projected/c2c4c1d2-a01a-419a-b9f2-6e049a868084-kube-api-access-qxjx2\") pod \"maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4\" (UID: \"c2c4c1d2-a01a-419a-b9f2-6e049a868084\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"
Jul 29 16:14:30.910754 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:30.910588    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2c4c1d2-a01a-419a-b9f2-6e049a868084-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4\" (UID: \"c2c4c1d2-a01a-419a-b9f2-6e049a868084\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"
Jul 29 16:14:30.910754 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:30.910715    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2c4c1d2-a01a-419a-b9f2-6e049a868084-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4\" (UID: \"c2c4c1d2-a01a-419a-b9f2-6e049a868084\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"
Jul 29 16:14:30.911372 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:30.911349    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2c4c1d2-a01a-419a-b9f2-6e049a868084-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4\" (UID: \"c2c4c1d2-a01a-419a-b9f2-6e049a868084\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"
Jul 29 16:14:30.913081 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:30.913063    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2c4c1d2-a01a-419a-b9f2-6e049a868084-maas-api-tls\") pod \"maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4\" (UID: \"c2c4c1d2-a01a-419a-b9f2-6e049a868084\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"
Jul 29 16:14:30.921232 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:30.921208    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxjx2\" (UniqueName: \"kubernetes.io/projected/c2c4c1d2-a01a-419a-b9f2-6e049a868084-kube-api-access-qxjx2\") pod \"maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4\" (UID: \"c2c4c1d2-a01a-419a-b9f2-6e049a868084\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"
Jul 29 16:14:31.017339 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:31.017241    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"
Jul 29 16:14:31.169064 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:31.169032    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"]
Jul 29 16:14:31.171837 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:14:31.171807    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2c4c1d2_a01a_419a_b9f2_6e049a868084.slice/crio-29144d237238c0e47d82c4eaf9bc3ab5fe6a4edb9348b0d0ae9f08c68aa6bfbd WatchSource:0}: Error finding container 29144d237238c0e47d82c4eaf9bc3ab5fe6a4edb9348b0d0ae9f08c68aa6bfbd: Status 404 returned error can't find the container with id 29144d237238c0e47d82c4eaf9bc3ab5fe6a4edb9348b0d0ae9f08c68aa6bfbd
Jul 29 16:14:31.173135 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:31.173120    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 16:14:31.451147 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:31.451115    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"]
Jul 29 16:14:31.455937 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:31.455913    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"
Jul 29 16:14:31.458008 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:31.457991    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-6fccfc\""
Jul 29 16:14:31.467798 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:31.467773    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"]
Jul 29 16:14:31.516421 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:31.516365    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e409d06-e557-4f2b-9e0e-51ce670d4e4a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd\" (UID: \"9e409d06-e557-4f2b-9e0e-51ce670d4e4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"
Jul 29 16:14:31.516614 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:31.516487    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbcnk\" (UniqueName: \"kubernetes.io/projected/9e409d06-e557-4f2b-9e0e-51ce670d4e4a-kube-api-access-jbcnk\") pod \"payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd\" (UID: \"9e409d06-e557-4f2b-9e0e-51ce670d4e4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"
Jul 29 16:14:31.535784 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:31.535740    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4" event={"ID":"c2c4c1d2-a01a-419a-b9f2-6e049a868084","Type":"ContainerStarted","Data":"29144d237238c0e47d82c4eaf9bc3ab5fe6a4edb9348b0d0ae9f08c68aa6bfbd"}
Jul 29 16:14:31.617178 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:31.617144    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbcnk\" (UniqueName: \"kubernetes.io/projected/9e409d06-e557-4f2b-9e0e-51ce670d4e4a-kube-api-access-jbcnk\") pod \"payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd\" (UID: \"9e409d06-e557-4f2b-9e0e-51ce670d4e4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"
Jul 29 16:14:31.617396 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:31.617202    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e409d06-e557-4f2b-9e0e-51ce670d4e4a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd\" (UID: \"9e409d06-e557-4f2b-9e0e-51ce670d4e4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"
Jul 29 16:14:31.617885 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:31.617862    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e409d06-e557-4f2b-9e0e-51ce670d4e4a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd\" (UID: \"9e409d06-e557-4f2b-9e0e-51ce670d4e4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"
Jul 29 16:14:31.627843 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:31.627815    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbcnk\" (UniqueName: \"kubernetes.io/projected/9e409d06-e557-4f2b-9e0e-51ce670d4e4a-kube-api-access-jbcnk\") pod \"payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd\" (UID: \"9e409d06-e557-4f2b-9e0e-51ce670d4e4a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"
Jul 29 16:14:31.768691 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:31.768666    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"
Jul 29 16:14:31.917882 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:31.917845    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"]
Jul 29 16:14:31.922070 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:14:31.921835    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e409d06_e557_4f2b_9e0e_51ce670d4e4a.slice/crio-4acaad4b3a00e4b6316ac5ebc7e9328aedf855a08ba044833a1f88d5536f4e47 WatchSource:0}: Error finding container 4acaad4b3a00e4b6316ac5ebc7e9328aedf855a08ba044833a1f88d5536f4e47: Status 404 returned error can't find the container with id 4acaad4b3a00e4b6316ac5ebc7e9328aedf855a08ba044833a1f88d5536f4e47
Jul 29 16:14:32.542651 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:32.542617    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd" event={"ID":"9e409d06-e557-4f2b-9e0e-51ce670d4e4a","Type":"ContainerStarted","Data":"0550771095f7c8e950c062ff435ca7137e18c2727c644bec0d3a95da7deaf2c7"}
Jul 29 16:14:32.542857 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:32.542771    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"
Jul 29 16:14:32.542857 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:32.542806    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd" event={"ID":"9e409d06-e557-4f2b-9e0e-51ce670d4e4a","Type":"ContainerStarted","Data":"4acaad4b3a00e4b6316ac5ebc7e9328aedf855a08ba044833a1f88d5536f4e47"}
Jul 29 16:14:32.544110 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:32.544086    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4" event={"ID":"c2c4c1d2-a01a-419a-b9f2-6e049a868084","Type":"ContainerStarted","Data":"f2d0165a3b22bc4ef05d7e25107212cb4d8078149c7368c853d5cc8897b42d14"}
Jul 29 16:14:32.544215 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:32.544135    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"
Jul 29 16:14:32.560238 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:32.560183    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd" podStartSLOduration=1.560165854 podStartE2EDuration="1.560165854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:14:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:14:32.560090605 +0000 UTC m=+3805.709384506" watchObservedRunningTime="2026-07-29 16:14:32.560165854 +0000 UTC m=+3805.709459755"
Jul 29 16:14:32.580108 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:32.580066    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4" podStartSLOduration=1.9979501750000002 podStartE2EDuration="2.580053463s" totalImagesPullingTime="582.103288ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:14:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:14:32.579215563 +0000 UTC m=+3805.728509463" watchObservedRunningTime="2026-07-29 16:14:32.580053463 +0000 UTC m=+3805.729347362"
Jul 29 16:14:38.554294 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:38.554261    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"
Jul 29 16:14:43.551076 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:43.551043    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"
Jul 29 16:14:59.782128 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:59.782091    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"]
Jul 29 16:14:59.786427 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:59.786400    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"
Jul 29 16:14:59.788758 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:59.788734    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-16a6d7-serving-cert\""
Jul 29 16:14:59.794929 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:59.794414    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"]
Jul 29 16:14:59.881147 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:59.881118    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxdv2\" (UniqueName: \"kubernetes.io/projected/c146e230-ba56-4b73-9992-fe629115197d-kube-api-access-rxdv2\") pod \"maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h\" (UID: \"c146e230-ba56-4b73-9992-fe629115197d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"
Jul 29 16:14:59.881317 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:59.881170    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c146e230-ba56-4b73-9992-fe629115197d-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h\" (UID: \"c146e230-ba56-4b73-9992-fe629115197d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"
Jul 29 16:14:59.881317 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:59.881258    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c146e230-ba56-4b73-9992-fe629115197d-maas-api-tls\") pod \"maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h\" (UID: \"c146e230-ba56-4b73-9992-fe629115197d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"
Jul 29 16:14:59.982599 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:59.982561    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxdv2\" (UniqueName: \"kubernetes.io/projected/c146e230-ba56-4b73-9992-fe629115197d-kube-api-access-rxdv2\") pod \"maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h\" (UID: \"c146e230-ba56-4b73-9992-fe629115197d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"
Jul 29 16:14:59.982767 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:59.982614    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c146e230-ba56-4b73-9992-fe629115197d-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h\" (UID: \"c146e230-ba56-4b73-9992-fe629115197d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"
Jul 29 16:14:59.982767 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:59.982649    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c146e230-ba56-4b73-9992-fe629115197d-maas-api-tls\") pod \"maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h\" (UID: \"c146e230-ba56-4b73-9992-fe629115197d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"
Jul 29 16:14:59.982767 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:14:59.982761    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-serving-cert: secret "maas-api-e2e-ipp-b-16a6d7-serving-cert" not found
Jul 29 16:14:59.982903 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:14:59.982815    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c146e230-ba56-4b73-9992-fe629115197d-maas-api-tls podName:c146e230-ba56-4b73-9992-fe629115197d nodeName:}" failed. No retries permitted until 2026-07-29 16:15:00.482798457 +0000 UTC m=+3833.632092336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c146e230-ba56-4b73-9992-fe629115197d-maas-api-tls") pod "maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h" (UID: "c146e230-ba56-4b73-9992-fe629115197d") : secret "maas-api-e2e-ipp-b-16a6d7-serving-cert" not found
Jul 29 16:14:59.983374 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:59.983353    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c146e230-ba56-4b73-9992-fe629115197d-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h\" (UID: \"c146e230-ba56-4b73-9992-fe629115197d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"
Jul 29 16:14:59.991991 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:14:59.991971    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxdv2\" (UniqueName: \"kubernetes.io/projected/c146e230-ba56-4b73-9992-fe629115197d-kube-api-access-rxdv2\") pod \"maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h\" (UID: \"c146e230-ba56-4b73-9992-fe629115197d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"
Jul 29 16:15:00.488555 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:00.488518    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c146e230-ba56-4b73-9992-fe629115197d-maas-api-tls\") pod \"maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h\" (UID: \"c146e230-ba56-4b73-9992-fe629115197d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"
Jul 29 16:15:00.490924 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:00.490890    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c146e230-ba56-4b73-9992-fe629115197d-maas-api-tls\") pod \"maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h\" (UID: \"c146e230-ba56-4b73-9992-fe629115197d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"
Jul 29 16:15:00.699296 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:00.699265    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"
Jul 29 16:15:00.746798 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:00.746725    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"]
Jul 29 16:15:00.750636 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:00.750614    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"
Jul 29 16:15:00.752986 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:00.752965    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-16a6d7-dockercfg-g58dt\""
Jul 29 16:15:00.753099 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:00.753017    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-16a6d7\""
Jul 29 16:15:00.761295 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:00.761271    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"]
Jul 29 16:15:00.854153 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:00.854128    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"]
Jul 29 16:15:00.856706 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:15:00.856677    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc146e230_ba56_4b73_9992_fe629115197d.slice/crio-0a913d50bb7b7fa9f446b2436ef7be87aab44b8e2480131e4134cddcc9dc2467 WatchSource:0}: Error finding container 0a913d50bb7b7fa9f446b2436ef7be87aab44b8e2480131e4134cddcc9dc2467: Status 404 returned error can't find the container with id 0a913d50bb7b7fa9f446b2436ef7be87aab44b8e2480131e4134cddcc9dc2467
Jul 29 16:15:00.892722 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:00.892696    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0c7ebdf-dd70-4458-a413-b44f47d2b0c6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl\" (UID: \"a0c7ebdf-dd70-4458-a413-b44f47d2b0c6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"
Jul 29 16:15:00.892839 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:00.892749    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p89qf\" (UniqueName: \"kubernetes.io/projected/a0c7ebdf-dd70-4458-a413-b44f47d2b0c6-kube-api-access-p89qf\") pod \"payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl\" (UID: \"a0c7ebdf-dd70-4458-a413-b44f47d2b0c6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"
Jul 29 16:15:00.994182 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:00.994146    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0c7ebdf-dd70-4458-a413-b44f47d2b0c6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl\" (UID: \"a0c7ebdf-dd70-4458-a413-b44f47d2b0c6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"
Jul 29 16:15:00.994362 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:00.994205    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p89qf\" (UniqueName: \"kubernetes.io/projected/a0c7ebdf-dd70-4458-a413-b44f47d2b0c6-kube-api-access-p89qf\") pod \"payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl\" (UID: \"a0c7ebdf-dd70-4458-a413-b44f47d2b0c6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"
Jul 29 16:15:00.994904 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:00.994876    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0c7ebdf-dd70-4458-a413-b44f47d2b0c6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl\" (UID: \"a0c7ebdf-dd70-4458-a413-b44f47d2b0c6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"
Jul 29 16:15:01.003702 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:01.003649    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p89qf\" (UniqueName: \"kubernetes.io/projected/a0c7ebdf-dd70-4458-a413-b44f47d2b0c6-kube-api-access-p89qf\") pod \"payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl\" (UID: \"a0c7ebdf-dd70-4458-a413-b44f47d2b0c6\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"
Jul 29 16:15:01.068477 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:01.068452    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"
Jul 29 16:15:01.413679 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:01.413650    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"]
Jul 29 16:15:01.415131 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:15:01.415101    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0c7ebdf_dd70_4458_a413_b44f47d2b0c6.slice/crio-beedf4ffde13c283bb1666dab6954feff46d3e48488bbec884fc528f5c65baaa WatchSource:0}: Error finding container beedf4ffde13c283bb1666dab6954feff46d3e48488bbec884fc528f5c65baaa: Status 404 returned error can't find the container with id beedf4ffde13c283bb1666dab6954feff46d3e48488bbec884fc528f5c65baaa
Jul 29 16:15:01.693451 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:01.693411    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h" event={"ID":"c146e230-ba56-4b73-9992-fe629115197d","Type":"ContainerStarted","Data":"72da7b10be6882bf4a8269eb0fb29119e4509be204d446ecb86ff36c6cd9ca6b"}
Jul 29 16:15:01.693640 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:01.693474    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h" event={"ID":"c146e230-ba56-4b73-9992-fe629115197d","Type":"ContainerStarted","Data":"0a913d50bb7b7fa9f446b2436ef7be87aab44b8e2480131e4134cddcc9dc2467"}
Jul 29 16:15:01.693640 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:01.693618    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"
Jul 29 16:15:01.694986 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:01.694962    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl" event={"ID":"a0c7ebdf-dd70-4458-a413-b44f47d2b0c6","Type":"ContainerStarted","Data":"a29e559329cbf3b7b94a978265b439bcf9b48953008e35f35f3c26e99429d0fb"}
Jul 29 16:15:01.695094 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:01.694999    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl" event={"ID":"a0c7ebdf-dd70-4458-a413-b44f47d2b0c6","Type":"ContainerStarted","Data":"beedf4ffde13c283bb1666dab6954feff46d3e48488bbec884fc528f5c65baaa"}
Jul 29 16:15:01.695094 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:01.695081    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"
Jul 29 16:15:01.713707 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:01.713663    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h" podStartSLOduration=2.321792283 podStartE2EDuration="2.713649447s" totalImagesPullingTime="391.857164ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:14:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:15:01.712645276 +0000 UTC m=+3834.861939177" watchObservedRunningTime="2026-07-29 16:15:01.713649447 +0000 UTC m=+3834.862943347"
Jul 29 16:15:01.733160 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:01.733107    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl" podStartSLOduration=1.733093939 podStartE2EDuration="1.733093939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:15:01.730729494 +0000 UTC m=+3834.880023395" watchObservedRunningTime="2026-07-29 16:15:01.733093939 +0000 UTC m=+3834.882387840"
Jul 29 16:15:07.705300 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:07.705271    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"
Jul 29 16:15:12.702168 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:12.702135    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"
Jul 29 16:15:31.586884 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:31.586806    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-knc5p_6fb56df1-ee67-48c1-be21-5b9faa36351e/payload-processing/0.log"
Jul 29 16:15:31.715383 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:15:31.715292    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl_a0c7ebdf-dd70-4458-a413-b44f47d2b0c6/payload-processing/0.log"
Jul 29 16:16:07.566087 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:07.566057    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 16:16:07.573180 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:07.573159    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 16:16:07.579236 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:07.579218    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 16:16:07.585901 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:07.585880    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ksmm5_a4d1a578-61d7-4a7d-b4c1-7389cb969155/ovn-acl-logging/0.log"
Jul 29 16:16:39.407334 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:39.407290    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-knc5p_6fb56df1-ee67-48c1-be21-5b9faa36351e/payload-processing/0.log"
Jul 29 16:16:39.524938 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:39.524909    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl_a0c7ebdf-dd70-4458-a413-b44f47d2b0c6/payload-processing/0.log"
Jul 29 16:16:43.384860 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:43.384822    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"]
Jul 29 16:16:43.389031 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:43.389009    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"
Jul 29 16:16:43.391095 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:43.391072    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-86568b-serving-cert\""
Jul 29 16:16:43.398641 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:43.398616    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"]
Jul 29 16:16:43.430383 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:43.430354    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnt99\" (UniqueName: \"kubernetes.io/projected/4beb0a48-540f-4cdc-9a45-37736856716c-kube-api-access-nnt99\") pod \"maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq\" (UID: \"4beb0a48-540f-4cdc-9a45-37736856716c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"
Jul 29 16:16:43.430598 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:43.430425    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4beb0a48-540f-4cdc-9a45-37736856716c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq\" (UID: \"4beb0a48-540f-4cdc-9a45-37736856716c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"
Jul 29 16:16:43.430598 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:43.430470    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4beb0a48-540f-4cdc-9a45-37736856716c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq\" (UID: \"4beb0a48-540f-4cdc-9a45-37736856716c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"
Jul 29 16:16:43.531096 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:43.531048    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnt99\" (UniqueName: \"kubernetes.io/projected/4beb0a48-540f-4cdc-9a45-37736856716c-kube-api-access-nnt99\") pod \"maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq\" (UID: \"4beb0a48-540f-4cdc-9a45-37736856716c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"
Jul 29 16:16:43.531355 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:43.531113    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4beb0a48-540f-4cdc-9a45-37736856716c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq\" (UID: \"4beb0a48-540f-4cdc-9a45-37736856716c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"
Jul 29 16:16:43.531355 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:43.531147    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4beb0a48-540f-4cdc-9a45-37736856716c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq\" (UID: \"4beb0a48-540f-4cdc-9a45-37736856716c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"
Jul 29 16:16:43.531554 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:16:43.531532    2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-serving-cert: secret "maas-api-e2e-ipp-cleanup-86568b-serving-cert" not found
Jul 29 16:16:43.531631 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:16:43.531618    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4beb0a48-540f-4cdc-9a45-37736856716c-maas-api-tls podName:4beb0a48-540f-4cdc-9a45-37736856716c nodeName:}" failed. No retries permitted until 2026-07-29 16:16:44.031596017 +0000 UTC m=+3937.180889912 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4beb0a48-540f-4cdc-9a45-37736856716c-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq" (UID: "4beb0a48-540f-4cdc-9a45-37736856716c") : secret "maas-api-e2e-ipp-cleanup-86568b-serving-cert" not found
Jul 29 16:16:43.531937 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:43.531918    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4beb0a48-540f-4cdc-9a45-37736856716c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq\" (UID: \"4beb0a48-540f-4cdc-9a45-37736856716c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"
Jul 29 16:16:43.543120 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:43.543084    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnt99\" (UniqueName: \"kubernetes.io/projected/4beb0a48-540f-4cdc-9a45-37736856716c-kube-api-access-nnt99\") pod \"maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq\" (UID: \"4beb0a48-540f-4cdc-9a45-37736856716c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"
Jul 29 16:16:44.034460 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:44.034388    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4beb0a48-540f-4cdc-9a45-37736856716c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq\" (UID: \"4beb0a48-540f-4cdc-9a45-37736856716c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"
Jul 29 16:16:44.036909 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:44.036873    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4beb0a48-540f-4cdc-9a45-37736856716c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq\" (UID: \"4beb0a48-540f-4cdc-9a45-37736856716c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"
Jul 29 16:16:44.300480 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:44.300382    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"
Jul 29 16:16:44.451096 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:44.451058    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"]
Jul 29 16:16:44.454122 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:16:44.454091    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4beb0a48_540f_4cdc_9a45_37736856716c.slice/crio-65ab848b8539e75d4fe309f3631390bfbe45d99ecea404fb0705d8e90832e84b WatchSource:0}: Error finding container 65ab848b8539e75d4fe309f3631390bfbe45d99ecea404fb0705d8e90832e84b: Status 404 returned error can't find the container with id 65ab848b8539e75d4fe309f3631390bfbe45d99ecea404fb0705d8e90832e84b
Jul 29 16:16:45.235725 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:45.235637    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq" event={"ID":"4beb0a48-540f-4cdc-9a45-37736856716c","Type":"ContainerStarted","Data":"b76e121158b525bb32bbd9241138be6cc6000414633d43ac8b0aead3308b9fe7"}
Jul 29 16:16:45.235725 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:45.235687    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq" event={"ID":"4beb0a48-540f-4cdc-9a45-37736856716c","Type":"ContainerStarted","Data":"65ab848b8539e75d4fe309f3631390bfbe45d99ecea404fb0705d8e90832e84b"}
Jul 29 16:16:45.235939 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:45.235726    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"
Jul 29 16:16:45.252673 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:45.252618    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq" podStartSLOduration=1.7836653 podStartE2EDuration="2.252599021s" totalImagesPullingTime="468.933721ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:16:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:16:45.251952775 +0000 UTC m=+3938.401246675" watchObservedRunningTime="2026-07-29 16:16:45.252599021 +0000 UTC m=+3938.401892922"
Jul 29 16:16:51.246792 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:16:51.246763    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"
Jul 29 16:17:08.931272 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:08.931161    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c"]
Jul 29 16:17:08.935550 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:08.935529    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c"
Jul 29 16:17:08.939111 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:08.939087    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgrq5\""
Jul 29 16:17:08.946162 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:08.946129    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c"]
Jul 29 16:17:09.063656 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:09.063622    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c3281ea-050f-431f-a26e-3559a0a74431-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c\" (UID: \"3c3281ea-050f-431f-a26e-3559a0a74431\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c"
Jul 29 16:17:09.165195 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:09.165163    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c3281ea-050f-431f-a26e-3559a0a74431-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c\" (UID: \"3c3281ea-050f-431f-a26e-3559a0a74431\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c"
Jul 29 16:17:09.165799 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:09.165779    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c3281ea-050f-431f-a26e-3559a0a74431-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c\" (UID: \"3c3281ea-050f-431f-a26e-3559a0a74431\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c"
Jul 29 16:17:09.247274 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:09.247204    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c"
Jul 29 16:17:09.387459 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:09.387430    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c"]
Jul 29 16:17:09.390349 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:17:09.390305    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c3281ea_050f_431f_a26e_3559a0a74431.slice/crio-cafe1b5a6ea018daf43a3050421ecbc6c5d61d0a22d1ffe9938dcd7f132551d6 WatchSource:0}: Error finding container cafe1b5a6ea018daf43a3050421ecbc6c5d61d0a22d1ffe9938dcd7f132551d6: Status 404 returned error can't find the container with id cafe1b5a6ea018daf43a3050421ecbc6c5d61d0a22d1ffe9938dcd7f132551d6
Jul 29 16:17:10.372603 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:10.372570    2575 generic.go:356] "Generic (PLEG): container finished" podID="3c3281ea-050f-431f-a26e-3559a0a74431" containerID="294bf1561dfbeafd88feb37e5471c1f44d9e3fa181141f36905a1b95d2d10856" exitCode=0
Jul 29 16:17:10.373089 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:10.372658    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c" event={"ID":"3c3281ea-050f-431f-a26e-3559a0a74431","Type":"ContainerDied","Data":"294bf1561dfbeafd88feb37e5471c1f44d9e3fa181141f36905a1b95d2d10856"}
Jul 29 16:17:10.373089 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:10.372720    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c" event={"ID":"3c3281ea-050f-431f-a26e-3559a0a74431","Type":"ContainerStarted","Data":"cafe1b5a6ea018daf43a3050421ecbc6c5d61d0a22d1ffe9938dcd7f132551d6"}
Jul 29 16:17:11.521043 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:11.521021    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c"
Jul 29 16:17:11.690391 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:11.690278    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c3281ea-050f-431f-a26e-3559a0a74431-maas-api-service-ca\") pod \"3c3281ea-050f-431f-a26e-3559a0a74431\" (UID: \"3c3281ea-050f-431f-a26e-3559a0a74431\") "
Jul 29 16:17:11.690615 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:11.690592    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c3281ea-050f-431f-a26e-3559a0a74431-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3c3281ea-050f-431f-a26e-3559a0a74431" (UID: "3c3281ea-050f-431f-a26e-3559a0a74431"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:17:11.790965 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:11.790933    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c3281ea-050f-431f-a26e-3559a0a74431-maas-api-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:17:12.383714 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:12.383670    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c" event={"ID":"3c3281ea-050f-431f-a26e-3559a0a74431","Type":"ContainerDied","Data":"cafe1b5a6ea018daf43a3050421ecbc6c5d61d0a22d1ffe9938dcd7f132551d6"}
Jul 29 16:17:12.383714 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:12.383696    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c"
Jul 29 16:17:12.383928 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:12.383737    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cafe1b5a6ea018daf43a3050421ecbc6c5d61d0a22d1ffe9938dcd7f132551d6"
Jul 29 16:17:14.062333 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.062286    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"]
Jul 29 16:17:14.062789 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.062592    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq" podUID="4beb0a48-540f-4cdc-9a45-37736856716c" containerName="maas-api" containerID="cri-o://b76e121158b525bb32bbd9241138be6cc6000414633d43ac8b0aead3308b9fe7" gracePeriod=30
Jul 29 16:17:14.295530 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.295502    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c"]
Jul 29 16:17:14.298681 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.298659    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-86568b-j674c"]
Jul 29 16:17:14.318672 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.318609    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"
Jul 29 16:17:14.398804 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.398771    2575 generic.go:356] "Generic (PLEG): container finished" podID="4beb0a48-540f-4cdc-9a45-37736856716c" containerID="b76e121158b525bb32bbd9241138be6cc6000414633d43ac8b0aead3308b9fe7" exitCode=0
Jul 29 16:17:14.398988 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.398823    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"
Jul 29 16:17:14.398988 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.398858    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq" event={"ID":"4beb0a48-540f-4cdc-9a45-37736856716c","Type":"ContainerDied","Data":"b76e121158b525bb32bbd9241138be6cc6000414633d43ac8b0aead3308b9fe7"}
Jul 29 16:17:14.398988 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.398917    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq" event={"ID":"4beb0a48-540f-4cdc-9a45-37736856716c","Type":"ContainerDied","Data":"65ab848b8539e75d4fe309f3631390bfbe45d99ecea404fb0705d8e90832e84b"}
Jul 29 16:17:14.398988 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.398940    2575 scope.go:117] "RemoveContainer" containerID="b76e121158b525bb32bbd9241138be6cc6000414633d43ac8b0aead3308b9fe7"
Jul 29 16:17:14.409084 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.409064    2575 scope.go:117] "RemoveContainer" containerID="b76e121158b525bb32bbd9241138be6cc6000414633d43ac8b0aead3308b9fe7"
Jul 29 16:17:14.409379 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:17:14.409356    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b76e121158b525bb32bbd9241138be6cc6000414633d43ac8b0aead3308b9fe7\": container with ID starting with b76e121158b525bb32bbd9241138be6cc6000414633d43ac8b0aead3308b9fe7 not found: ID does not exist" containerID="b76e121158b525bb32bbd9241138be6cc6000414633d43ac8b0aead3308b9fe7"
Jul 29 16:17:14.409481 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.409387    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b76e121158b525bb32bbd9241138be6cc6000414633d43ac8b0aead3308b9fe7"} err="failed to get container status \"b76e121158b525bb32bbd9241138be6cc6000414633d43ac8b0aead3308b9fe7\": rpc error: code = NotFound desc = could not find container \"b76e121158b525bb32bbd9241138be6cc6000414633d43ac8b0aead3308b9fe7\": container with ID starting with b76e121158b525bb32bbd9241138be6cc6000414633d43ac8b0aead3308b9fe7 not found: ID does not exist"
Jul 29 16:17:14.413055 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.413039    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nnt99\" (UniqueName: \"kubernetes.io/projected/4beb0a48-540f-4cdc-9a45-37736856716c-kube-api-access-nnt99\") pod \"4beb0a48-540f-4cdc-9a45-37736856716c\" (UID: \"4beb0a48-540f-4cdc-9a45-37736856716c\") "
Jul 29 16:17:14.413143 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.413092    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4beb0a48-540f-4cdc-9a45-37736856716c-maas-api-tls\") pod \"4beb0a48-540f-4cdc-9a45-37736856716c\" (UID: \"4beb0a48-540f-4cdc-9a45-37736856716c\") "
Jul 29 16:17:14.413185 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.413176    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4beb0a48-540f-4cdc-9a45-37736856716c-openshift-service-ca\") pod \"4beb0a48-540f-4cdc-9a45-37736856716c\" (UID: \"4beb0a48-540f-4cdc-9a45-37736856716c\") "
Jul 29 16:17:14.413769 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.413525    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4beb0a48-540f-4cdc-9a45-37736856716c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4beb0a48-540f-4cdc-9a45-37736856716c" (UID: "4beb0a48-540f-4cdc-9a45-37736856716c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:17:14.415173 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.415151    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4beb0a48-540f-4cdc-9a45-37736856716c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4beb0a48-540f-4cdc-9a45-37736856716c" (UID: "4beb0a48-540f-4cdc-9a45-37736856716c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:17:14.415458 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.415438    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4beb0a48-540f-4cdc-9a45-37736856716c-kube-api-access-nnt99" (OuterVolumeSpecName: "kube-api-access-nnt99") pod "4beb0a48-540f-4cdc-9a45-37736856716c" (UID: "4beb0a48-540f-4cdc-9a45-37736856716c"). InnerVolumeSpecName "kube-api-access-nnt99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:17:14.514732 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.514695    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4beb0a48-540f-4cdc-9a45-37736856716c-maas-api-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:17:14.514732 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.514731    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4beb0a48-540f-4cdc-9a45-37736856716c-openshift-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:17:14.514732 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.514741    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnt99\" (UniqueName: \"kubernetes.io/projected/4beb0a48-540f-4cdc-9a45-37736856716c-kube-api-access-nnt99\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:17:14.722752 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.722722    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"]
Jul 29 16:17:14.727681 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:14.727654    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-86568b-6bc6b95f86-kljmq"]
Jul 29 16:17:15.295577 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:15.295537    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c3281ea-050f-431f-a26e-3559a0a74431" path="/var/lib/kubelet/pods/3c3281ea-050f-431f-a26e-3559a0a74431/volumes"
Jul 29 16:17:15.296108 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:15.296088    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4beb0a48-540f-4cdc-9a45-37736856716c" path="/var/lib/kubelet/pods/4beb0a48-540f-4cdc-9a45-37736856716c/volumes"
Jul 29 16:17:40.379934 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.379895    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"]
Jul 29 16:17:40.380473 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.380123    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4" podUID="c2c4c1d2-a01a-419a-b9f2-6e049a868084" containerName="maas-api" containerID="cri-o://f2d0165a3b22bc4ef05d7e25107212cb4d8078149c7368c853d5cc8897b42d14" gracePeriod=30
Jul 29 16:17:40.455687 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.455653    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"]
Jul 29 16:17:40.455906 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.455859    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd" podUID="9e409d06-e557-4f2b-9e0e-51ce670d4e4a" containerName="payload-pre-processing" containerID="cri-o://0550771095f7c8e950c062ff435ca7137e18c2727c644bec0d3a95da7deaf2c7" gracePeriod=30
Jul 29 16:17:40.537713 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.537685    2575 generic.go:356] "Generic (PLEG): container finished" podID="c2c4c1d2-a01a-419a-b9f2-6e049a868084" containerID="f2d0165a3b22bc4ef05d7e25107212cb4d8078149c7368c853d5cc8897b42d14" exitCode=0
Jul 29 16:17:40.537878 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.537752    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4" event={"ID":"c2c4c1d2-a01a-419a-b9f2-6e049a868084","Type":"ContainerDied","Data":"f2d0165a3b22bc4ef05d7e25107212cb4d8078149c7368c853d5cc8897b42d14"}
Jul 29 16:17:40.724055 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.723971    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"
Jul 29 16:17:40.751841 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.751818    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"
Jul 29 16:17:40.849257 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.849224    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxjx2\" (UniqueName: \"kubernetes.io/projected/c2c4c1d2-a01a-419a-b9f2-6e049a868084-kube-api-access-qxjx2\") pod \"c2c4c1d2-a01a-419a-b9f2-6e049a868084\" (UID: \"c2c4c1d2-a01a-419a-b9f2-6e049a868084\") "
Jul 29 16:17:40.849679 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.849296    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2c4c1d2-a01a-419a-b9f2-6e049a868084-maas-api-tls\") pod \"c2c4c1d2-a01a-419a-b9f2-6e049a868084\" (UID: \"c2c4c1d2-a01a-419a-b9f2-6e049a868084\") "
Jul 29 16:17:40.849679 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.849361    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbcnk\" (UniqueName: \"kubernetes.io/projected/9e409d06-e557-4f2b-9e0e-51ce670d4e4a-kube-api-access-jbcnk\") pod \"9e409d06-e557-4f2b-9e0e-51ce670d4e4a\" (UID: \"9e409d06-e557-4f2b-9e0e-51ce670d4e4a\") "
Jul 29 16:17:40.849679 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.849424    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2c4c1d2-a01a-419a-b9f2-6e049a868084-openshift-service-ca\") pod \"c2c4c1d2-a01a-419a-b9f2-6e049a868084\" (UID: \"c2c4c1d2-a01a-419a-b9f2-6e049a868084\") "
Jul 29 16:17:40.849679 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.849480    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e409d06-e557-4f2b-9e0e-51ce670d4e4a-plugins-config-volume\") pod \"9e409d06-e557-4f2b-9e0e-51ce670d4e4a\" (UID: \"9e409d06-e557-4f2b-9e0e-51ce670d4e4a\") "
Jul 29 16:17:40.849976 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.849742    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2c4c1d2-a01a-419a-b9f2-6e049a868084-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c2c4c1d2-a01a-419a-b9f2-6e049a868084" (UID: "c2c4c1d2-a01a-419a-b9f2-6e049a868084"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:17:40.849976 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.849951    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e409d06-e557-4f2b-9e0e-51ce670d4e4a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9e409d06-e557-4f2b-9e0e-51ce670d4e4a" (UID: "9e409d06-e557-4f2b-9e0e-51ce670d4e4a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:17:40.851671 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.851647    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2c4c1d2-a01a-419a-b9f2-6e049a868084-kube-api-access-qxjx2" (OuterVolumeSpecName: "kube-api-access-qxjx2") pod "c2c4c1d2-a01a-419a-b9f2-6e049a868084" (UID: "c2c4c1d2-a01a-419a-b9f2-6e049a868084"). InnerVolumeSpecName "kube-api-access-qxjx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:17:40.851757 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.851652    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2c4c1d2-a01a-419a-b9f2-6e049a868084-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c2c4c1d2-a01a-419a-b9f2-6e049a868084" (UID: "c2c4c1d2-a01a-419a-b9f2-6e049a868084"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:17:40.851757 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.851653    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e409d06-e557-4f2b-9e0e-51ce670d4e4a-kube-api-access-jbcnk" (OuterVolumeSpecName: "kube-api-access-jbcnk") pod "9e409d06-e557-4f2b-9e0e-51ce670d4e4a" (UID: "9e409d06-e557-4f2b-9e0e-51ce670d4e4a"). InnerVolumeSpecName "kube-api-access-jbcnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:17:40.950801 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.950717    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2c4c1d2-a01a-419a-b9f2-6e049a868084-openshift-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:17:40.950801 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.950747    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e409d06-e557-4f2b-9e0e-51ce670d4e4a-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:17:40.950801 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.950757    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxjx2\" (UniqueName: \"kubernetes.io/projected/c2c4c1d2-a01a-419a-b9f2-6e049a868084-kube-api-access-qxjx2\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:17:40.950801 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.950767    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2c4c1d2-a01a-419a-b9f2-6e049a868084-maas-api-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:17:40.950801 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:40.950776    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbcnk\" (UniqueName: \"kubernetes.io/projected/9e409d06-e557-4f2b-9e0e-51ce670d4e4a-kube-api-access-jbcnk\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:17:41.543639 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:41.543604    2575 generic.go:356] "Generic (PLEG): container finished" podID="9e409d06-e557-4f2b-9e0e-51ce670d4e4a" containerID="0550771095f7c8e950c062ff435ca7137e18c2727c644bec0d3a95da7deaf2c7" exitCode=0
Jul 29 16:17:41.544207 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:41.543637    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd" event={"ID":"9e409d06-e557-4f2b-9e0e-51ce670d4e4a","Type":"ContainerDied","Data":"0550771095f7c8e950c062ff435ca7137e18c2727c644bec0d3a95da7deaf2c7"}
Jul 29 16:17:41.544207 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:41.543661    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"
Jul 29 16:17:41.544207 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:41.543701    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd" event={"ID":"9e409d06-e557-4f2b-9e0e-51ce670d4e4a","Type":"ContainerDied","Data":"4acaad4b3a00e4b6316ac5ebc7e9328aedf855a08ba044833a1f88d5536f4e47"}
Jul 29 16:17:41.544207 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:41.543738    2575 scope.go:117] "RemoveContainer" containerID="0550771095f7c8e950c062ff435ca7137e18c2727c644bec0d3a95da7deaf2c7"
Jul 29 16:17:41.545304 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:41.545244    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4" event={"ID":"c2c4c1d2-a01a-419a-b9f2-6e049a868084","Type":"ContainerDied","Data":"29144d237238c0e47d82c4eaf9bc3ab5fe6a4edb9348b0d0ae9f08c68aa6bfbd"}
Jul 29 16:17:41.545304 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:41.545257    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"
Jul 29 16:17:41.553497 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:41.553482    2575 scope.go:117] "RemoveContainer" containerID="0550771095f7c8e950c062ff435ca7137e18c2727c644bec0d3a95da7deaf2c7"
Jul 29 16:17:41.553785 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:17:41.553747    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0550771095f7c8e950c062ff435ca7137e18c2727c644bec0d3a95da7deaf2c7\": container with ID starting with 0550771095f7c8e950c062ff435ca7137e18c2727c644bec0d3a95da7deaf2c7 not found: ID does not exist" containerID="0550771095f7c8e950c062ff435ca7137e18c2727c644bec0d3a95da7deaf2c7"
Jul 29 16:17:41.553785 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:41.553769    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0550771095f7c8e950c062ff435ca7137e18c2727c644bec0d3a95da7deaf2c7"} err="failed to get container status \"0550771095f7c8e950c062ff435ca7137e18c2727c644bec0d3a95da7deaf2c7\": rpc error: code = NotFound desc = could not find container \"0550771095f7c8e950c062ff435ca7137e18c2727c644bec0d3a95da7deaf2c7\": container with ID starting with 0550771095f7c8e950c062ff435ca7137e18c2727c644bec0d3a95da7deaf2c7 not found: ID does not exist"
Jul 29 16:17:41.553785 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:41.553784    2575 scope.go:117] "RemoveContainer" containerID="f2d0165a3b22bc4ef05d7e25107212cb4d8078149c7368c853d5cc8897b42d14"
Jul 29 16:17:41.568543 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:41.568518    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"]
Jul 29 16:17:41.573572 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:41.573550    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-6fccfc-746fc5b6bf-4zmh4"]
Jul 29 16:17:41.584958 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:41.584927    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"]
Jul 29 16:17:41.589816 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:41.589794    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-6fccfc-744448c584-8wjrd"]
Jul 29 16:17:43.295313 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:43.295283    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e409d06-e557-4f2b-9e0e-51ce670d4e4a" path="/var/lib/kubelet/pods/9e409d06-e557-4f2b-9e0e-51ce670d4e4a/volumes"
Jul 29 16:17:43.295718 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:17:43.295704    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2c4c1d2-a01a-419a-b9f2-6e049a868084" path="/var/lib/kubelet/pods/c2c4c1d2-a01a-419a-b9f2-6e049a868084/volumes"
Jul 29 16:18:35.007452 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.007359    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"]
Jul 29 16:18:35.007907 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.007652    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h" podUID="c146e230-ba56-4b73-9992-fe629115197d" containerName="maas-api" containerID="cri-o://72da7b10be6882bf4a8269eb0fb29119e4509be204d446ecb86ff36c6cd9ca6b" gracePeriod=30
Jul 29 16:18:35.084464 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.084431    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"]
Jul 29 16:18:35.084708 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.084678    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl" podUID="a0c7ebdf-dd70-4458-a413-b44f47d2b0c6" containerName="payload-processing" containerID="cri-o://a29e559329cbf3b7b94a978265b439bcf9b48953008e35f35f3c26e99429d0fb" gracePeriod=30
Jul 29 16:18:35.369574 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.369545    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"
Jul 29 16:18:35.409224 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.409203    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"
Jul 29 16:18:35.437566 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.437538    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c146e230-ba56-4b73-9992-fe629115197d-openshift-service-ca\") pod \"c146e230-ba56-4b73-9992-fe629115197d\" (UID: \"c146e230-ba56-4b73-9992-fe629115197d\") "
Jul 29 16:18:35.437723 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.437613    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxdv2\" (UniqueName: \"kubernetes.io/projected/c146e230-ba56-4b73-9992-fe629115197d-kube-api-access-rxdv2\") pod \"c146e230-ba56-4b73-9992-fe629115197d\" (UID: \"c146e230-ba56-4b73-9992-fe629115197d\") "
Jul 29 16:18:35.437723 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.437639    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c146e230-ba56-4b73-9992-fe629115197d-maas-api-tls\") pod \"c146e230-ba56-4b73-9992-fe629115197d\" (UID: \"c146e230-ba56-4b73-9992-fe629115197d\") "
Jul 29 16:18:35.437835 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.437737    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0c7ebdf-dd70-4458-a413-b44f47d2b0c6-plugins-config-volume\") pod \"a0c7ebdf-dd70-4458-a413-b44f47d2b0c6\" (UID: \"a0c7ebdf-dd70-4458-a413-b44f47d2b0c6\") "
Jul 29 16:18:35.437835 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.437767    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p89qf\" (UniqueName: \"kubernetes.io/projected/a0c7ebdf-dd70-4458-a413-b44f47d2b0c6-kube-api-access-p89qf\") pod \"a0c7ebdf-dd70-4458-a413-b44f47d2b0c6\" (UID: \"a0c7ebdf-dd70-4458-a413-b44f47d2b0c6\") "
Jul 29 16:18:35.438083 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.438051    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c146e230-ba56-4b73-9992-fe629115197d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c146e230-ba56-4b73-9992-fe629115197d" (UID: "c146e230-ba56-4b73-9992-fe629115197d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:18:35.438171 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.438147    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0c7ebdf-dd70-4458-a413-b44f47d2b0c6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a0c7ebdf-dd70-4458-a413-b44f47d2b0c6" (UID: "a0c7ebdf-dd70-4458-a413-b44f47d2b0c6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:18:35.439911 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.439879    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c146e230-ba56-4b73-9992-fe629115197d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c146e230-ba56-4b73-9992-fe629115197d" (UID: "c146e230-ba56-4b73-9992-fe629115197d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:18:35.440055 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.440031    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c146e230-ba56-4b73-9992-fe629115197d-kube-api-access-rxdv2" (OuterVolumeSpecName: "kube-api-access-rxdv2") pod "c146e230-ba56-4b73-9992-fe629115197d" (UID: "c146e230-ba56-4b73-9992-fe629115197d"). InnerVolumeSpecName "kube-api-access-rxdv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:18:35.440370 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.440349    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0c7ebdf-dd70-4458-a413-b44f47d2b0c6-kube-api-access-p89qf" (OuterVolumeSpecName: "kube-api-access-p89qf") pod "a0c7ebdf-dd70-4458-a413-b44f47d2b0c6" (UID: "a0c7ebdf-dd70-4458-a413-b44f47d2b0c6"). InnerVolumeSpecName "kube-api-access-p89qf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:18:35.538860 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.538826    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a0c7ebdf-dd70-4458-a413-b44f47d2b0c6-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:18:35.538860 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.538859    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p89qf\" (UniqueName: \"kubernetes.io/projected/a0c7ebdf-dd70-4458-a413-b44f47d2b0c6-kube-api-access-p89qf\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:18:35.539047 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.538871    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c146e230-ba56-4b73-9992-fe629115197d-openshift-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:18:35.539047 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.538884    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxdv2\" (UniqueName: \"kubernetes.io/projected/c146e230-ba56-4b73-9992-fe629115197d-kube-api-access-rxdv2\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:18:35.539047 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.538896    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c146e230-ba56-4b73-9992-fe629115197d-maas-api-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:18:35.825511 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.825475    2575 generic.go:356] "Generic (PLEG): container finished" podID="c146e230-ba56-4b73-9992-fe629115197d" containerID="72da7b10be6882bf4a8269eb0fb29119e4509be204d446ecb86ff36c6cd9ca6b" exitCode=0
Jul 29 16:18:35.825685 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.825522    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"
Jul 29 16:18:35.825685 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.825567    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h" event={"ID":"c146e230-ba56-4b73-9992-fe629115197d","Type":"ContainerDied","Data":"72da7b10be6882bf4a8269eb0fb29119e4509be204d446ecb86ff36c6cd9ca6b"}
Jul 29 16:18:35.825685 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.825621    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h" event={"ID":"c146e230-ba56-4b73-9992-fe629115197d","Type":"ContainerDied","Data":"0a913d50bb7b7fa9f446b2436ef7be87aab44b8e2480131e4134cddcc9dc2467"}
Jul 29 16:18:35.825685 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.825645    2575 scope.go:117] "RemoveContainer" containerID="72da7b10be6882bf4a8269eb0fb29119e4509be204d446ecb86ff36c6cd9ca6b"
Jul 29 16:18:35.827247 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.827220    2575 generic.go:356] "Generic (PLEG): container finished" podID="a0c7ebdf-dd70-4458-a413-b44f47d2b0c6" containerID="a29e559329cbf3b7b94a978265b439bcf9b48953008e35f35f3c26e99429d0fb" exitCode=0
Jul 29 16:18:35.827394 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.827282    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"
Jul 29 16:18:35.827394 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.827298    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl" event={"ID":"a0c7ebdf-dd70-4458-a413-b44f47d2b0c6","Type":"ContainerDied","Data":"a29e559329cbf3b7b94a978265b439bcf9b48953008e35f35f3c26e99429d0fb"}
Jul 29 16:18:35.827394 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.827369    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl" event={"ID":"a0c7ebdf-dd70-4458-a413-b44f47d2b0c6","Type":"ContainerDied","Data":"beedf4ffde13c283bb1666dab6954feff46d3e48488bbec884fc528f5c65baaa"}
Jul 29 16:18:35.837130 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.837113    2575 scope.go:117] "RemoveContainer" containerID="72da7b10be6882bf4a8269eb0fb29119e4509be204d446ecb86ff36c6cd9ca6b"
Jul 29 16:18:35.837433 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:18:35.837400    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72da7b10be6882bf4a8269eb0fb29119e4509be204d446ecb86ff36c6cd9ca6b\": container with ID starting with 72da7b10be6882bf4a8269eb0fb29119e4509be204d446ecb86ff36c6cd9ca6b not found: ID does not exist" containerID="72da7b10be6882bf4a8269eb0fb29119e4509be204d446ecb86ff36c6cd9ca6b"
Jul 29 16:18:35.837564 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.837443    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72da7b10be6882bf4a8269eb0fb29119e4509be204d446ecb86ff36c6cd9ca6b"} err="failed to get container status \"72da7b10be6882bf4a8269eb0fb29119e4509be204d446ecb86ff36c6cd9ca6b\": rpc error: code = NotFound desc = could not find container \"72da7b10be6882bf4a8269eb0fb29119e4509be204d446ecb86ff36c6cd9ca6b\": container with ID starting with 72da7b10be6882bf4a8269eb0fb29119e4509be204d446ecb86ff36c6cd9ca6b not found: ID does not exist"
Jul 29 16:18:35.837564 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.837465    2575 scope.go:117] "RemoveContainer" containerID="a29e559329cbf3b7b94a978265b439bcf9b48953008e35f35f3c26e99429d0fb"
Jul 29 16:18:35.846536 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.846510    2575 scope.go:117] "RemoveContainer" containerID="a29e559329cbf3b7b94a978265b439bcf9b48953008e35f35f3c26e99429d0fb"
Jul 29 16:18:35.846801 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:18:35.846781    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a29e559329cbf3b7b94a978265b439bcf9b48953008e35f35f3c26e99429d0fb\": container with ID starting with a29e559329cbf3b7b94a978265b439bcf9b48953008e35f35f3c26e99429d0fb not found: ID does not exist" containerID="a29e559329cbf3b7b94a978265b439bcf9b48953008e35f35f3c26e99429d0fb"
Jul 29 16:18:35.846874 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.846806    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a29e559329cbf3b7b94a978265b439bcf9b48953008e35f35f3c26e99429d0fb"} err="failed to get container status \"a29e559329cbf3b7b94a978265b439bcf9b48953008e35f35f3c26e99429d0fb\": rpc error: code = NotFound desc = could not find container \"a29e559329cbf3b7b94a978265b439bcf9b48953008e35f35f3c26e99429d0fb\": container with ID starting with a29e559329cbf3b7b94a978265b439bcf9b48953008e35f35f3c26e99429d0fb not found: ID does not exist"
Jul 29 16:18:35.856955 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.856930    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"]
Jul 29 16:18:35.864836 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.864815    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-16a6d7-7cd5ff779d-www2h"]
Jul 29 16:18:35.878505 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.878484    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"]
Jul 29 16:18:35.883569 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:35.883547    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-16a6d7-76c8bf86bb-x67pl"]
Jul 29 16:18:37.295291 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:37.295261    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0c7ebdf-dd70-4458-a413-b44f47d2b0c6" path="/var/lib/kubelet/pods/a0c7ebdf-dd70-4458-a413-b44f47d2b0c6/volumes"
Jul 29 16:18:37.295685 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:37.295641    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c146e230-ba56-4b73-9992-fe629115197d" path="/var/lib/kubelet/pods/c146e230-ba56-4b73-9992-fe629115197d/volumes"
Jul 29 16:18:42.783168 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783128    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7"]
Jul 29 16:18:42.783569 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783536    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0c7ebdf-dd70-4458-a413-b44f47d2b0c6" containerName="payload-processing"
Jul 29 16:18:42.783569 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783548    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0c7ebdf-dd70-4458-a413-b44f47d2b0c6" containerName="payload-processing"
Jul 29 16:18:42.783569 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783560    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c3281ea-050f-431f-a26e-3559a0a74431" containerName="revoke-keys"
Jul 29 16:18:42.783569 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783567    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c3281ea-050f-431f-a26e-3559a0a74431" containerName="revoke-keys"
Jul 29 16:18:42.783707 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783580    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e409d06-e557-4f2b-9e0e-51ce670d4e4a" containerName="payload-pre-processing"
Jul 29 16:18:42.783707 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783586    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e409d06-e557-4f2b-9e0e-51ce670d4e4a" containerName="payload-pre-processing"
Jul 29 16:18:42.783707 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783592    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c146e230-ba56-4b73-9992-fe629115197d" containerName="maas-api"
Jul 29 16:18:42.783707 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783597    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c146e230-ba56-4b73-9992-fe629115197d" containerName="maas-api"
Jul 29 16:18:42.783707 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783602    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4beb0a48-540f-4cdc-9a45-37736856716c" containerName="maas-api"
Jul 29 16:18:42.783707 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783608    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4beb0a48-540f-4cdc-9a45-37736856716c" containerName="maas-api"
Jul 29 16:18:42.783707 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783621    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2c4c1d2-a01a-419a-b9f2-6e049a868084" containerName="maas-api"
Jul 29 16:18:42.783707 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783626    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2c4c1d2-a01a-419a-b9f2-6e049a868084" containerName="maas-api"
Jul 29 16:18:42.783707 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783681    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c3281ea-050f-431f-a26e-3559a0a74431" containerName="revoke-keys"
Jul 29 16:18:42.783707 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783689    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c146e230-ba56-4b73-9992-fe629115197d" containerName="maas-api"
Jul 29 16:18:42.783707 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783696    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e409d06-e557-4f2b-9e0e-51ce670d4e4a" containerName="payload-pre-processing"
Jul 29 16:18:42.783707 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783703    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0c7ebdf-dd70-4458-a413-b44f47d2b0c6" containerName="payload-processing"
Jul 29 16:18:42.783707 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783710    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2c4c1d2-a01a-419a-b9f2-6e049a868084" containerName="maas-api"
Jul 29 16:18:42.784096 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.783715    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4beb0a48-540f-4cdc-9a45-37736856716c" containerName="maas-api"
Jul 29 16:18:42.788238 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.788220    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7"
Jul 29 16:18:42.790539 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.790517    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgrq5\""
Jul 29 16:18:42.795791 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.795555    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7"]
Jul 29 16:18:42.907622 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:42.907582    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7\" (UID: \"4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7"
Jul 29 16:18:43.008661 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:43.008620    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7\" (UID: \"4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7"
Jul 29 16:18:43.009346 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:43.009300    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7\" (UID: \"4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7"
Jul 29 16:18:43.100723 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:43.100643    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7"
Jul 29 16:18:43.246791 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:43.246769    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7"]
Jul 29 16:18:43.249554 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:18:43.249524    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cc2473a_c4d6_4ac1_ad43_63a1571c3aa7.slice/crio-06c6207ff57608a5e551f99206d7086b2ace14f65b79ff5babade1076d0b6660 WatchSource:0}: Error finding container 06c6207ff57608a5e551f99206d7086b2ace14f65b79ff5babade1076d0b6660: Status 404 returned error can't find the container with id 06c6207ff57608a5e551f99206d7086b2ace14f65b79ff5babade1076d0b6660
Jul 29 16:18:43.868861 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:43.868819    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7" event={"ID":"4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7","Type":"ContainerStarted","Data":"f39e0d068da984c6a6112673a43ff8f18ccc78a1e46ffae95e80f6cd44cd4eb2"}
Jul 29 16:18:43.869245 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:43.868921    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7" event={"ID":"4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7","Type":"ContainerStarted","Data":"06c6207ff57608a5e551f99206d7086b2ace14f65b79ff5babade1076d0b6660"}
Jul 29 16:18:43.892432 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:43.892384    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7" podStartSLOduration=1.8923691790000001 podStartE2EDuration="1.892369179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 16:18:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:18:43.890851816 +0000 UTC m=+4057.040145717" watchObservedRunningTime="2026-07-29 16:18:43.892369179 +0000 UTC m=+4057.041663079"
Jul 29 16:18:44.875830 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:44.875795    2575 generic.go:356] "Generic (PLEG): container finished" podID="4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7" containerID="f39e0d068da984c6a6112673a43ff8f18ccc78a1e46ffae95e80f6cd44cd4eb2" exitCode=0
Jul 29 16:18:44.876277 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:44.875884    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7" event={"ID":"4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7","Type":"ContainerDied","Data":"f39e0d068da984c6a6112673a43ff8f18ccc78a1e46ffae95e80f6cd44cd4eb2"}
Jul 29 16:18:46.019543 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:46.019515    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7"
Jul 29 16:18:46.138931 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:46.138850    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7-maas-api-service-ca\") pod \"4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7\" (UID: \"4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7\") "
Jul 29 16:18:46.139192 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:46.139163    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7" (UID: "4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:18:46.240052 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:46.240020    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7-maas-api-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:18:46.887706 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:46.887679    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7"
Jul 29 16:18:46.887913 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:46.887741    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7" event={"ID":"4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7","Type":"ContainerDied","Data":"06c6207ff57608a5e551f99206d7086b2ace14f65b79ff5babade1076d0b6660"}
Jul 29 16:18:46.887913 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:46.887788    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06c6207ff57608a5e551f99206d7086b2ace14f65b79ff5babade1076d0b6660"
Jul 29 16:18:53.435297 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.435251    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"]
Jul 29 16:18:53.436166 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.436129    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5" podUID="aa588d7c-e56d-4039-8643-fcaec33edc80" containerName="maas-api" containerID="cri-o://4d57605fe01719ba682c5300ded05e6b7aea7264802d5730ea74cbb0c7f78954" gracePeriod=30
Jul 29 16:18:53.522410 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.522370    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"]
Jul 29 16:18:53.523154 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.523112    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl" podUID="6140907c-df02-4bd7-8267-116fc800f6ae" containerName="payload-processing" containerID="cri-o://325a888b7d2e695b5fad8fa4de71aec900ff44ad1f2cda54b3e5f8b71f9fea0e" gracePeriod=30
Jul 29 16:18:53.729451 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.729417    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7"]
Jul 29 16:18:53.731975 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.731952    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-02add2-dw2b7"]
Jul 29 16:18:53.776391 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.776367    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"
Jul 29 16:18:53.807168 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.807145    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"
Jul 29 16:18:53.909581 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.909550    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94hlf\" (UniqueName: \"kubernetes.io/projected/6140907c-df02-4bd7-8267-116fc800f6ae-kube-api-access-94hlf\") pod \"6140907c-df02-4bd7-8267-116fc800f6ae\" (UID: \"6140907c-df02-4bd7-8267-116fc800f6ae\") "
Jul 29 16:18:53.909733 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.909642    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa588d7c-e56d-4039-8643-fcaec33edc80-openshift-service-ca\") pod \"aa588d7c-e56d-4039-8643-fcaec33edc80\" (UID: \"aa588d7c-e56d-4039-8643-fcaec33edc80\") "
Jul 29 16:18:53.909733 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.909674    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa588d7c-e56d-4039-8643-fcaec33edc80-maas-api-tls\") pod \"aa588d7c-e56d-4039-8643-fcaec33edc80\" (UID: \"aa588d7c-e56d-4039-8643-fcaec33edc80\") "
Jul 29 16:18:53.909733 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.909712    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6b4dn\" (UniqueName: \"kubernetes.io/projected/aa588d7c-e56d-4039-8643-fcaec33edc80-kube-api-access-6b4dn\") pod \"aa588d7c-e56d-4039-8643-fcaec33edc80\" (UID: \"aa588d7c-e56d-4039-8643-fcaec33edc80\") "
Jul 29 16:18:53.909733 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.909729    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6140907c-df02-4bd7-8267-116fc800f6ae-plugins-config-volume\") pod \"6140907c-df02-4bd7-8267-116fc800f6ae\" (UID: \"6140907c-df02-4bd7-8267-116fc800f6ae\") "
Jul 29 16:18:53.910042 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.910012    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa588d7c-e56d-4039-8643-fcaec33edc80-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aa588d7c-e56d-4039-8643-fcaec33edc80" (UID: "aa588d7c-e56d-4039-8643-fcaec33edc80"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:18:53.910234 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.910209    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6140907c-df02-4bd7-8267-116fc800f6ae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6140907c-df02-4bd7-8267-116fc800f6ae" (UID: "6140907c-df02-4bd7-8267-116fc800f6ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:18:53.911931 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.911908    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa588d7c-e56d-4039-8643-fcaec33edc80-kube-api-access-6b4dn" (OuterVolumeSpecName: "kube-api-access-6b4dn") pod "aa588d7c-e56d-4039-8643-fcaec33edc80" (UID: "aa588d7c-e56d-4039-8643-fcaec33edc80"). InnerVolumeSpecName "kube-api-access-6b4dn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:18:53.912030 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.911933    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa588d7c-e56d-4039-8643-fcaec33edc80-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aa588d7c-e56d-4039-8643-fcaec33edc80" (UID: "aa588d7c-e56d-4039-8643-fcaec33edc80"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:18:53.912030 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.911936    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6140907c-df02-4bd7-8267-116fc800f6ae-kube-api-access-94hlf" (OuterVolumeSpecName: "kube-api-access-94hlf") pod "6140907c-df02-4bd7-8267-116fc800f6ae" (UID: "6140907c-df02-4bd7-8267-116fc800f6ae"). InnerVolumeSpecName "kube-api-access-94hlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:18:53.923665 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.923638    2575 generic.go:356] "Generic (PLEG): container finished" podID="aa588d7c-e56d-4039-8643-fcaec33edc80" containerID="4d57605fe01719ba682c5300ded05e6b7aea7264802d5730ea74cbb0c7f78954" exitCode=0
Jul 29 16:18:53.923791 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.923680    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5" event={"ID":"aa588d7c-e56d-4039-8643-fcaec33edc80","Type":"ContainerDied","Data":"4d57605fe01719ba682c5300ded05e6b7aea7264802d5730ea74cbb0c7f78954"}
Jul 29 16:18:53.923791 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.923737    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5" event={"ID":"aa588d7c-e56d-4039-8643-fcaec33edc80","Type":"ContainerDied","Data":"4cc7c3ea8a824f0dcbe1e76f8c4ce8e91b7554220e2b1f4324f8041ae8ff628c"}
Jul 29 16:18:53.923791 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.923759    2575 scope.go:117] "RemoveContainer" containerID="4d57605fe01719ba682c5300ded05e6b7aea7264802d5730ea74cbb0c7f78954"
Jul 29 16:18:53.923791 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.923690    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"
Jul 29 16:18:53.925423 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.925403    2575 generic.go:356] "Generic (PLEG): container finished" podID="6140907c-df02-4bd7-8267-116fc800f6ae" containerID="325a888b7d2e695b5fad8fa4de71aec900ff44ad1f2cda54b3e5f8b71f9fea0e" exitCode=0
Jul 29 16:18:53.925538 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.925433    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl" event={"ID":"6140907c-df02-4bd7-8267-116fc800f6ae","Type":"ContainerDied","Data":"325a888b7d2e695b5fad8fa4de71aec900ff44ad1f2cda54b3e5f8b71f9fea0e"}
Jul 29 16:18:53.925538 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.925458    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"
Jul 29 16:18:53.925538 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.925485    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl" event={"ID":"6140907c-df02-4bd7-8267-116fc800f6ae","Type":"ContainerDied","Data":"caf652891237b293c17c92e9cc3085ee0ddcbcf6e548246581ddc3969c2af9ef"}
Jul 29 16:18:53.935017 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.935001    2575 scope.go:117] "RemoveContainer" containerID="4d57605fe01719ba682c5300ded05e6b7aea7264802d5730ea74cbb0c7f78954"
Jul 29 16:18:53.935309 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:18:53.935246    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d57605fe01719ba682c5300ded05e6b7aea7264802d5730ea74cbb0c7f78954\": container with ID starting with 4d57605fe01719ba682c5300ded05e6b7aea7264802d5730ea74cbb0c7f78954 not found: ID does not exist" containerID="4d57605fe01719ba682c5300ded05e6b7aea7264802d5730ea74cbb0c7f78954"
Jul 29 16:18:53.935309 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.935271    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d57605fe01719ba682c5300ded05e6b7aea7264802d5730ea74cbb0c7f78954"} err="failed to get container status \"4d57605fe01719ba682c5300ded05e6b7aea7264802d5730ea74cbb0c7f78954\": rpc error: code = NotFound desc = could not find container \"4d57605fe01719ba682c5300ded05e6b7aea7264802d5730ea74cbb0c7f78954\": container with ID starting with 4d57605fe01719ba682c5300ded05e6b7aea7264802d5730ea74cbb0c7f78954 not found: ID does not exist"
Jul 29 16:18:53.935309 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.935292    2575 scope.go:117] "RemoveContainer" containerID="325a888b7d2e695b5fad8fa4de71aec900ff44ad1f2cda54b3e5f8b71f9fea0e"
Jul 29 16:18:53.942988 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.942970    2575 scope.go:117] "RemoveContainer" containerID="325a888b7d2e695b5fad8fa4de71aec900ff44ad1f2cda54b3e5f8b71f9fea0e"
Jul 29 16:18:53.943227 ip-10-0-135-116 kubenswrapper[2575]: E0729 16:18:53.943208    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"325a888b7d2e695b5fad8fa4de71aec900ff44ad1f2cda54b3e5f8b71f9fea0e\": container with ID starting with 325a888b7d2e695b5fad8fa4de71aec900ff44ad1f2cda54b3e5f8b71f9fea0e not found: ID does not exist" containerID="325a888b7d2e695b5fad8fa4de71aec900ff44ad1f2cda54b3e5f8b71f9fea0e"
Jul 29 16:18:53.943292 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.943230    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"325a888b7d2e695b5fad8fa4de71aec900ff44ad1f2cda54b3e5f8b71f9fea0e"} err="failed to get container status \"325a888b7d2e695b5fad8fa4de71aec900ff44ad1f2cda54b3e5f8b71f9fea0e\": rpc error: code = NotFound desc = could not find container \"325a888b7d2e695b5fad8fa4de71aec900ff44ad1f2cda54b3e5f8b71f9fea0e\": container with ID starting with 325a888b7d2e695b5fad8fa4de71aec900ff44ad1f2cda54b3e5f8b71f9fea0e not found: ID does not exist"
Jul 29 16:18:53.952910 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.952888    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"]
Jul 29 16:18:53.956082 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.956063    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-02add2-6454c4ff95-wsgl5"]
Jul 29 16:18:53.971674 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.971655    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"]
Jul 29 16:18:53.977117 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:53.977100    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-02add2-6d5894478b-psgxl"]
Jul 29 16:18:54.010786 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:54.010761    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa588d7c-e56d-4039-8643-fcaec33edc80-openshift-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:18:54.010893 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:54.010790    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa588d7c-e56d-4039-8643-fcaec33edc80-maas-api-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:18:54.010893 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:54.010806    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6b4dn\" (UniqueName: \"kubernetes.io/projected/aa588d7c-e56d-4039-8643-fcaec33edc80-kube-api-access-6b4dn\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:18:54.010893 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:54.010818    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6140907c-df02-4bd7-8267-116fc800f6ae-plugins-config-volume\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:18:54.010893 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:54.010831    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94hlf\" (UniqueName: \"kubernetes.io/projected/6140907c-df02-4bd7-8267-116fc800f6ae-kube-api-access-94hlf\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:18:55.295976 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:55.295944    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7" path="/var/lib/kubelet/pods/4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7/volumes"
Jul 29 16:18:55.296355 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:55.296298    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6140907c-df02-4bd7-8267-116fc800f6ae" path="/var/lib/kubelet/pods/6140907c-df02-4bd7-8267-116fc800f6ae/volumes"
Jul 29 16:18:55.296659 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:18:55.296646    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa588d7c-e56d-4039-8643-fcaec33edc80" path="/var/lib/kubelet/pods/aa588d7c-e56d-4039-8643-fcaec33edc80/volumes"
Jul 29 16:19:01.283405 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.283374    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj"]
Jul 29 16:19:01.283792 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.283749    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7" containerName="revoke-keys"
Jul 29 16:19:01.283792 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.283761    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7" containerName="revoke-keys"
Jul 29 16:19:01.283792 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.283781    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6140907c-df02-4bd7-8267-116fc800f6ae" containerName="payload-processing"
Jul 29 16:19:01.283792 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.283787    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6140907c-df02-4bd7-8267-116fc800f6ae" containerName="payload-processing"
Jul 29 16:19:01.283921 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.283798    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa588d7c-e56d-4039-8643-fcaec33edc80" containerName="maas-api"
Jul 29 16:19:01.283921 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.283804    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa588d7c-e56d-4039-8643-fcaec33edc80" containerName="maas-api"
Jul 29 16:19:01.283921 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.283861    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6140907c-df02-4bd7-8267-116fc800f6ae" containerName="payload-processing"
Jul 29 16:19:01.283921 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.283868    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cc2473a-c4d6-4ac1-ad43-63a1571c3aa7" containerName="revoke-keys"
Jul 29 16:19:01.283921 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.283879    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa588d7c-e56d-4039-8643-fcaec33edc80" containerName="maas-api"
Jul 29 16:19:01.287974 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.287958    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj"
Jul 29 16:19:01.289956 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.289928    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgrq5\""
Jul 29 16:19:01.296580 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.296554    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj"]
Jul 29 16:19:01.377619 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.377584    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/333e4ca6-db43-4bc1-871b-a89315b29307-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj\" (UID: \"333e4ca6-db43-4bc1-871b-a89315b29307\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj"
Jul 29 16:19:01.478478 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.478437    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/333e4ca6-db43-4bc1-871b-a89315b29307-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj\" (UID: \"333e4ca6-db43-4bc1-871b-a89315b29307\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj"
Jul 29 16:19:01.479219 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.479199    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/333e4ca6-db43-4bc1-871b-a89315b29307-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj\" (UID: \"333e4ca6-db43-4bc1-871b-a89315b29307\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj"
Jul 29 16:19:01.599877 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.599785    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj"
Jul 29 16:19:01.742226 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.742194    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj"]
Jul 29 16:19:01.744429 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:19:01.744397    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod333e4ca6_db43_4bc1_871b_a89315b29307.slice/crio-ff6987f768eaf77063ea383e2c84065322b6fece116437e5a426c7dc4aeabdea WatchSource:0}: Error finding container ff6987f768eaf77063ea383e2c84065322b6fece116437e5a426c7dc4aeabdea: Status 404 returned error can't find the container with id ff6987f768eaf77063ea383e2c84065322b6fece116437e5a426c7dc4aeabdea
Jul 29 16:19:01.968425 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:01.968389    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj" event={"ID":"333e4ca6-db43-4bc1-871b-a89315b29307","Type":"ContainerStarted","Data":"ff6987f768eaf77063ea383e2c84065322b6fece116437e5a426c7dc4aeabdea"}
Jul 29 16:19:02.975392 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:02.975359    2575 generic.go:356] "Generic (PLEG): container finished" podID="333e4ca6-db43-4bc1-871b-a89315b29307" containerID="beff9b2ff8572a2dee04f16730ae46b51bd92fa98efcddaeddd46ab30aac54cc" exitCode=0
Jul 29 16:19:02.975794 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:02.975405    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj" event={"ID":"333e4ca6-db43-4bc1-871b-a89315b29307","Type":"ContainerDied","Data":"beff9b2ff8572a2dee04f16730ae46b51bd92fa98efcddaeddd46ab30aac54cc"}
Jul 29 16:19:04.117316 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:04.117291    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj"
Jul 29 16:19:04.209034 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:04.208999    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/333e4ca6-db43-4bc1-871b-a89315b29307-maas-api-service-ca\") pod \"333e4ca6-db43-4bc1-871b-a89315b29307\" (UID: \"333e4ca6-db43-4bc1-871b-a89315b29307\") "
Jul 29 16:19:04.209400 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:04.209375    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/333e4ca6-db43-4bc1-871b-a89315b29307-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "333e4ca6-db43-4bc1-871b-a89315b29307" (UID: "333e4ca6-db43-4bc1-871b-a89315b29307"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:19:04.310166 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:04.310081    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/333e4ca6-db43-4bc1-871b-a89315b29307-maas-api-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:19:04.987068 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:04.987038    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj"
Jul 29 16:19:04.987228 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:04.987064    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj" event={"ID":"333e4ca6-db43-4bc1-871b-a89315b29307","Type":"ContainerDied","Data":"ff6987f768eaf77063ea383e2c84065322b6fece116437e5a426c7dc4aeabdea"}
Jul 29 16:19:04.987228 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:04.987110    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff6987f768eaf77063ea383e2c84065322b6fece116437e5a426c7dc4aeabdea"
Jul 29 16:19:11.776657 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:11.776621    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"]
Jul 29 16:19:11.777109 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:11.776870    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5" podUID="c3f618fa-f8f8-453c-8328-e65b1fb0b4cb" containerName="maas-api" containerID="cri-o://9f6a3fa466ee9924f01fc025c7bccbbb406cc2616aee3d90a631e1819bcb4798" gracePeriod=30
Jul 29 16:19:12.025381 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:12.025155    2575 generic.go:356] "Generic (PLEG): container finished" podID="c3f618fa-f8f8-453c-8328-e65b1fb0b4cb" containerID="9f6a3fa466ee9924f01fc025c7bccbbb406cc2616aee3d90a631e1819bcb4798" exitCode=0
Jul 29 16:19:12.025381 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:12.025264    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5" event={"ID":"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb","Type":"ContainerDied","Data":"9f6a3fa466ee9924f01fc025c7bccbbb406cc2616aee3d90a631e1819bcb4798"}
Jul 29 16:19:12.040443 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:12.040380    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"
Jul 29 16:19:12.055967 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:12.055929    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj"]
Jul 29 16:19:12.063351 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:12.063301    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-7e5daf-wq7sj"]
Jul 29 16:19:12.076905 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:12.076882    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-maas-api-tls\") pod \"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb\" (UID: \"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb\") "
Jul 29 16:19:12.077017 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:12.076951    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2chj\" (UniqueName: \"kubernetes.io/projected/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-kube-api-access-n2chj\") pod \"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb\" (UID: \"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb\") "
Jul 29 16:19:12.077017 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:12.076976    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-openshift-service-ca\") pod \"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb\" (UID: \"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb\") "
Jul 29 16:19:12.077385 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:12.077356    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c3f618fa-f8f8-453c-8328-e65b1fb0b4cb" (UID: "c3f618fa-f8f8-453c-8328-e65b1fb0b4cb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 16:19:12.078931 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:12.078911    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-kube-api-access-n2chj" (OuterVolumeSpecName: "kube-api-access-n2chj") pod "c3f618fa-f8f8-453c-8328-e65b1fb0b4cb" (UID: "c3f618fa-f8f8-453c-8328-e65b1fb0b4cb"). InnerVolumeSpecName "kube-api-access-n2chj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 16:19:12.079011 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:12.078997    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c3f618fa-f8f8-453c-8328-e65b1fb0b4cb" (UID: "c3f618fa-f8f8-453c-8328-e65b1fb0b4cb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 16:19:12.178020 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:12.177987    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-maas-api-tls\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:19:12.178020 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:12.178016    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2chj\" (UniqueName: \"kubernetes.io/projected/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-kube-api-access-n2chj\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:19:12.178020 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:12.178027    2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb-openshift-service-ca\") on node \"ip-10-0-135-116.ec2.internal\" DevicePath \"\""
Jul 29 16:19:13.031389 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:13.031359    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"
Jul 29 16:19:13.031911 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:13.031357    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5" event={"ID":"c3f618fa-f8f8-453c-8328-e65b1fb0b4cb","Type":"ContainerDied","Data":"4c7e9fb3619cac33bbb0c0b1a145b069b99b0326708f802c7665d50c1e728ce0"}
Jul 29 16:19:13.031911 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:13.031493    2575 scope.go:117] "RemoveContainer" containerID="9f6a3fa466ee9924f01fc025c7bccbbb406cc2616aee3d90a631e1819bcb4798"
Jul 29 16:19:13.054084 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:13.054061    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"]
Jul 29 16:19:13.059481 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:13.059449    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-7e5daf-f4b7f574c-9rtm5"]
Jul 29 16:19:13.294811 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:13.294730    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="333e4ca6-db43-4bc1-871b-a89315b29307" path="/var/lib/kubelet/pods/333e4ca6-db43-4bc1-871b-a89315b29307/volumes"
Jul 29 16:19:13.295109 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:13.295097    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3f618fa-f8f8-453c-8328-e65b1fb0b4cb" path="/var/lib/kubelet/pods/c3f618fa-f8f8-453c-8328-e65b1fb0b4cb/volumes"
Jul 29 16:19:25.241625 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:25.241594    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gjvg9_e5cb24fc-8cca-4ff2-bf3c-3269c685b3df/manager/0.log"
Jul 29 16:19:25.370422 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:25.370389    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-hs8s2_54c563b3-de7c-47de-aa91-a909bcad32a4/manager/0.log"
Jul 29 16:19:25.501069 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:25.500993    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8497979899-wgmwr_19b9c0bc-a3b2-4694-afad-3e5c35d074d8/manager/0.log"
Jul 29 16:19:25.626034 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:25.626006    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-57wds_59fd0b51-1e06-4c85-a9e0-154b7a178bb8/server/0.log"
Jul 29 16:19:25.749805 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:25.749778    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-vspkj_146aecc1-dbef-4fd8-9ba9-f9425d7f91c0/manager/0.log"
Jul 29 16:19:26.140700 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:26.140674    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85b6fb79c5-rfhp8_539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0/copy-manifests/0.log"
Jul 29 16:19:26.155623 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:26.155598    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85b6fb79c5-rfhp8_539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0/manager/0.log"
Jul 29 16:19:26.527481 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:26.527405    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-847fd45f6b-52k77_d8b103fd-382c-45cb-bce6-71f59a5f33ef/maas-api/0.log"
Jul 29 16:19:27.000842 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:27.000814    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-z4vnp_6c8373a2-3b2a-4b90-abfc-a063ab89ce84/postgres/0.log"
Jul 29 16:19:28.505552 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:28.505519    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-27hfh_010096a5-22e8-4f05-bdf6-7616da899e6d/manager/0.log"
Jul 29 16:19:28.741420 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:28.741392    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-bspg8_748b38d9-4a8b-42a9-9769-20f041c64d29/kuadrant-console-plugin/0.log"
Jul 29 16:19:28.865541 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:28.865517    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-5zjdr_6693fd43-9bf1-41f2-87c6-37f1b8a4981d/registry-server/0.log"
Jul 29 16:19:29.000233 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:29.000203    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-kvt52_e0de3ccd-e482-46ac-9b8f-7aa6c1173147/manager/0.log"
Jul 29 16:19:29.241175 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:29.241100    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-zhfmp_ff3181c2-66af-4c2c-bc41-63cfcb90f40d/manager/0.log"
Jul 29 16:19:29.990464 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:29.990433    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-688955555-v6nlm_4142a983-ca2f-476f-806d-21efb849144e/kube-auth-proxy/0.log"
Jul 29 16:19:30.241491 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:30.241409    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-n8z9v_244016a2-b3e4-4f13-a387-6197b07e8be2/payload-pre-processing/0.log"
Jul 29 16:19:30.365222 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:30.365191    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-knc5p_6fb56df1-ee67-48c1-be21-5b9faa36351e/payload-processing/0.log"
Jul 29 16:19:30.849116 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:30.849088    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-mp87m_fd94d310-9226-4c39-adf9-3c5d8a11e129/main/0.log"
Jul 29 16:19:31.212390 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:31.212353    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f44rhc_14d20cc8-0ed5-4f5b-a3b9-f95d7022e336/main/0.log"
Jul 29 16:19:40.398300 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:40.398271    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rnrfc_a2f60801-5312-4da2-a438-6103942ccbd7/global-pull-secret-syncer/0.log"
Jul 29 16:19:40.606895 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:40.606866    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zwf85_b36df5c1-83d8-4ac9-8be0-2c7aa7086502/konnectivity-agent/0.log"
Jul 29 16:19:40.659368 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:40.659274    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-116.ec2.internal_5da59de62b315dbd806daa423c5548a3/haproxy/0.log"
Jul 29 16:19:45.660665 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:45.660578    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-27hfh_010096a5-22e8-4f05-bdf6-7616da899e6d/manager/0.log"
Jul 29 16:19:45.735958 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:45.735934    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-bspg8_748b38d9-4a8b-42a9-9769-20f041c64d29/kuadrant-console-plugin/0.log"
Jul 29 16:19:45.789512 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:45.789487    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-5zjdr_6693fd43-9bf1-41f2-87c6-37f1b8a4981d/registry-server/0.log"
Jul 29 16:19:45.874086 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:45.874058    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-kvt52_e0de3ccd-e482-46ac-9b8f-7aa6c1173147/manager/0.log"
Jul 29 16:19:45.998573 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:45.998496    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-zhfmp_ff3181c2-66af-4c2c-bc41-63cfcb90f40d/manager/0.log"
Jul 29 16:19:48.232252 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:48.232227    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-4kcv6_640f7311-506d-4d5c-bb8c-59facc070d1a/monitoring-plugin/0.log"
Jul 29 16:19:48.355627 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:48.355603    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kfrcd_5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e/node-exporter/0.log"
Jul 29 16:19:48.378748 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:48.378721    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kfrcd_5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e/kube-rbac-proxy/0.log"
Jul 29 16:19:48.406562 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:48.406539    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kfrcd_5d01ae0c-9529-4b6c-9f7d-0ab0d3e7507e/init-textfile/0.log"
Jul 29 16:19:48.628177 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:48.628097    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b55a053e-d8e1-4d15-afbf-76e928fc5d47/prometheus/0.log"
Jul 29 16:19:48.655040 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:48.655017    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b55a053e-d8e1-4d15-afbf-76e928fc5d47/config-reloader/0.log"
Jul 29 16:19:48.683250 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:48.683227    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b55a053e-d8e1-4d15-afbf-76e928fc5d47/thanos-sidecar/0.log"
Jul 29 16:19:48.713124 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:48.713102    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b55a053e-d8e1-4d15-afbf-76e928fc5d47/kube-rbac-proxy-web/0.log"
Jul 29 16:19:48.741008 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:48.740991    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b55a053e-d8e1-4d15-afbf-76e928fc5d47/kube-rbac-proxy/0.log"
Jul 29 16:19:48.766447 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:48.766425    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b55a053e-d8e1-4d15-afbf-76e928fc5d47/kube-rbac-proxy-thanos/0.log"
Jul 29 16:19:48.792676 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:48.792657    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b55a053e-d8e1-4d15-afbf-76e928fc5d47/init-config-reloader/0.log"
Jul 29 16:19:50.880400 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:50.880361    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"]
Jul 29 16:19:50.880930 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:50.880909    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="333e4ca6-db43-4bc1-871b-a89315b29307" containerName="revoke-keys"
Jul 29 16:19:50.881014 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:50.880934    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="333e4ca6-db43-4bc1-871b-a89315b29307" containerName="revoke-keys"
Jul 29 16:19:50.881014 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:50.880953    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3f618fa-f8f8-453c-8328-e65b1fb0b4cb" containerName="maas-api"
Jul 29 16:19:50.881014 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:50.880961    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3f618fa-f8f8-453c-8328-e65b1fb0b4cb" containerName="maas-api"
Jul 29 16:19:50.881166 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:50.881070    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3f618fa-f8f8-453c-8328-e65b1fb0b4cb" containerName="maas-api"
Jul 29 16:19:50.881166 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:50.881085    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="333e4ca6-db43-4bc1-871b-a89315b29307" containerName="revoke-keys"
Jul 29 16:19:50.884464 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:50.884442    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:50.886669 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:50.886645    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6xxwd\"/\"kube-root-ca.crt\""
Jul 29 16:19:50.886779 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:50.886714    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6xxwd\"/\"openshift-service-ca.crt\""
Jul 29 16:19:50.887384 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:50.887362    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6xxwd\"/\"default-dockercfg-hmbm7\""
Jul 29 16:19:50.892577 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:50.892556    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"]
Jul 29 16:19:51.025195 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.025155    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79g5k\" (UniqueName: \"kubernetes.io/projected/1923b6d0-c3d9-445e-94ae-bbefc85a6f13-kube-api-access-79g5k\") pod \"perf-node-gather-daemonset-kpdm8\" (UID: \"1923b6d0-c3d9-445e-94ae-bbefc85a6f13\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:51.025439 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.025208    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1923b6d0-c3d9-445e-94ae-bbefc85a6f13-lib-modules\") pod \"perf-node-gather-daemonset-kpdm8\" (UID: \"1923b6d0-c3d9-445e-94ae-bbefc85a6f13\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:51.025439 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.025233    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1923b6d0-c3d9-445e-94ae-bbefc85a6f13-proc\") pod \"perf-node-gather-daemonset-kpdm8\" (UID: \"1923b6d0-c3d9-445e-94ae-bbefc85a6f13\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:51.025439 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.025370    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1923b6d0-c3d9-445e-94ae-bbefc85a6f13-podres\") pod \"perf-node-gather-daemonset-kpdm8\" (UID: \"1923b6d0-c3d9-445e-94ae-bbefc85a6f13\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:51.025439 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.025423    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1923b6d0-c3d9-445e-94ae-bbefc85a6f13-sys\") pod \"perf-node-gather-daemonset-kpdm8\" (UID: \"1923b6d0-c3d9-445e-94ae-bbefc85a6f13\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:51.082158 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.082100    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/2.log"
Jul 29 16:19:51.085949 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.085928    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-ztzjg_f268c925-4fbb-42bd-9d51-b2c74d55c34e/console-operator/3.log"
Jul 29 16:19:51.126761 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.126736    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1923b6d0-c3d9-445e-94ae-bbefc85a6f13-podres\") pod \"perf-node-gather-daemonset-kpdm8\" (UID: \"1923b6d0-c3d9-445e-94ae-bbefc85a6f13\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:51.126862 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.126773    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1923b6d0-c3d9-445e-94ae-bbefc85a6f13-sys\") pod \"perf-node-gather-daemonset-kpdm8\" (UID: \"1923b6d0-c3d9-445e-94ae-bbefc85a6f13\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:51.126862 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.126819    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79g5k\" (UniqueName: \"kubernetes.io/projected/1923b6d0-c3d9-445e-94ae-bbefc85a6f13-kube-api-access-79g5k\") pod \"perf-node-gather-daemonset-kpdm8\" (UID: \"1923b6d0-c3d9-445e-94ae-bbefc85a6f13\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:51.126991 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.126862    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1923b6d0-c3d9-445e-94ae-bbefc85a6f13-lib-modules\") pod \"perf-node-gather-daemonset-kpdm8\" (UID: \"1923b6d0-c3d9-445e-94ae-bbefc85a6f13\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:51.126991 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.126908    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1923b6d0-c3d9-445e-94ae-bbefc85a6f13-podres\") pod \"perf-node-gather-daemonset-kpdm8\" (UID: \"1923b6d0-c3d9-445e-94ae-bbefc85a6f13\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:51.126991 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.126936    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1923b6d0-c3d9-445e-94ae-bbefc85a6f13-sys\") pod \"perf-node-gather-daemonset-kpdm8\" (UID: \"1923b6d0-c3d9-445e-94ae-bbefc85a6f13\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:51.126991 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.126956    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1923b6d0-c3d9-445e-94ae-bbefc85a6f13-lib-modules\") pod \"perf-node-gather-daemonset-kpdm8\" (UID: \"1923b6d0-c3d9-445e-94ae-bbefc85a6f13\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:51.126991 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.126982    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1923b6d0-c3d9-445e-94ae-bbefc85a6f13-proc\") pod \"perf-node-gather-daemonset-kpdm8\" (UID: \"1923b6d0-c3d9-445e-94ae-bbefc85a6f13\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:51.127155 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.127061    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1923b6d0-c3d9-445e-94ae-bbefc85a6f13-proc\") pod \"perf-node-gather-daemonset-kpdm8\" (UID: \"1923b6d0-c3d9-445e-94ae-bbefc85a6f13\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:51.140784 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.140725    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79g5k\" (UniqueName: \"kubernetes.io/projected/1923b6d0-c3d9-445e-94ae-bbefc85a6f13-kube-api-access-79g5k\") pod \"perf-node-gather-daemonset-kpdm8\" (UID: \"1923b6d0-c3d9-445e-94ae-bbefc85a6f13\") " pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:51.194581 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.194558    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:51.328723 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:51.328682    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"]
Jul 29 16:19:51.331090 ip-10-0-135-116 kubenswrapper[2575]: W0729 16:19:51.331049    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1923b6d0_c3d9_445e_94ae_bbefc85a6f13.slice/crio-c13acd18202d3dbd19658f8807073719ff4be4be02ae38ef53fc130796180501 WatchSource:0}: Error finding container c13acd18202d3dbd19658f8807073719ff4be4be02ae38ef53fc130796180501: Status 404 returned error can't find the container with id c13acd18202d3dbd19658f8807073719ff4be4be02ae38ef53fc130796180501
Jul 29 16:19:52.228276 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:52.228221    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8" event={"ID":"1923b6d0-c3d9-445e-94ae-bbefc85a6f13","Type":"ContainerStarted","Data":"f76ad9b001fe2e4134d7cc66f3988cc7d0e4afde8676ad61ec8ece9b71546d2a"}
Jul 29 16:19:52.228757 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:52.228291    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8" event={"ID":"1923b6d0-c3d9-445e-94ae-bbefc85a6f13","Type":"ContainerStarted","Data":"c13acd18202d3dbd19658f8807073719ff4be4be02ae38ef53fc130796180501"}
Jul 29 16:19:52.228757 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:52.228428    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:52.253181 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:52.253134    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8" podStartSLOduration=2.25311906 podStartE2EDuration="2.25311906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 16:19:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 16:19:52.25173442 +0000 UTC m=+4125.401028321" watchObservedRunningTime="2026-07-29 16:19:52.25311906 +0000 UTC m=+4125.402412959"
Jul 29 16:19:53.594143 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:53.594114    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zvjbc_4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e/dns/0.log"
Jul 29 16:19:53.618005 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:53.617982    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zvjbc_4c9df7c8-57b4-4ec0-b1fb-85149f5ec60e/kube-rbac-proxy/0.log"
Jul 29 16:19:53.702826 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:53.702798    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8xf5b_68f9c6d7-78ed-4086-8e7f-9244b109c1d7/dns-node-resolver/0.log"
Jul 29 16:19:54.392122 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:54.392097    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m6bcg_52c1f3d7-1c12-497f-82ed-3d5439d7dfbc/node-ca/0.log"
Jul 29 16:19:55.720004 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:55.719974    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-688955555-v6nlm_4142a983-ca2f-476f-806d-21efb849144e/kube-auth-proxy/0.log"
Jul 29 16:19:55.852621 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:55.852593    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-n8z9v_244016a2-b3e4-4f13-a387-6197b07e8be2/payload-pre-processing/0.log"
Jul 29 16:19:55.885075 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:55.885049    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-knc5p_6fb56df1-ee67-48c1-be21-5b9faa36351e/payload-processing/0.log"
Jul 29 16:19:56.796176 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:56.796146    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tps4w_4a897ee8-304b-48a3-8afe-c09656ab3028/serve-healthcheck-canary/0.log"
Jul 29 16:19:57.279017 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:57.278990    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-zfj9l_d3ffac2a-3208-473f-9d7f-ca7819f1ecff/insights-operator/0.log"
Jul 29 16:19:57.280908 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:57.280892    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-zfj9l_d3ffac2a-3208-473f-9d7f-ca7819f1ecff/insights-operator/1.log"
Jul 29 16:19:57.403015 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:57.402986    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fsfps_3ce93a47-6362-4a83-b069-a12163c25159/kube-rbac-proxy/0.log"
Jul 29 16:19:57.427304 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:57.427276    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fsfps_3ce93a47-6362-4a83-b069-a12163c25159/exporter/0.log"
Jul 29 16:19:57.451078 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:57.451056    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fsfps_3ce93a47-6362-4a83-b069-a12163c25159/extractor/0.log"
Jul 29 16:19:58.243254 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:58.243223    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6xxwd/perf-node-gather-daemonset-kpdm8"
Jul 29 16:19:59.865630 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:19:59.865602    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gjvg9_e5cb24fc-8cca-4ff2-bf3c-3269c685b3df/manager/0.log"
Jul 29 16:20:00.096755 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:20:00.096662    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-hs8s2_54c563b3-de7c-47de-aa91-a909bcad32a4/manager/0.log"
Jul 29 16:20:00.270484 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:20:00.270457    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8497979899-wgmwr_19b9c0bc-a3b2-4694-afad-3e5c35d074d8/manager/0.log"
Jul 29 16:20:00.299954 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:20:00.299929    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-57wds_59fd0b51-1e06-4c85-a9e0-154b7a178bb8/server/0.log"
Jul 29 16:20:00.392332 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:20:00.392297    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-vspkj_146aecc1-dbef-4fd8-9ba9-f9425d7f91c0/manager/0.log"
Jul 29 16:20:00.609991 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:20:00.609959    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85b6fb79c5-rfhp8_539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0/manager/0.log"
Jul 29 16:20:00.630745 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:20:00.630723    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85b6fb79c5-rfhp8_539a12a3-ac2e-4ce2-abb6-ebc1d6a1a1d0/copy-manifests/0.log"
Jul 29 16:20:01.665819 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:20:01.665783    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-688955555-v6nlm_4142a983-ca2f-476f-806d-21efb849144e/kube-auth-proxy/0.log"
Jul 29 16:20:01.767553 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:20:01.767526    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-n8z9v_244016a2-b3e4-4f13-a387-6197b07e8be2/payload-pre-processing/0.log"
Jul 29 16:20:01.788007 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:20:01.787983    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-knc5p_6fb56df1-ee67-48c1-be21-5b9faa36351e/payload-processing/0.log"
Jul 29 16:20:03.590975 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:20:03.590946    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-688955555-v6nlm_4142a983-ca2f-476f-806d-21efb849144e/kube-auth-proxy/0.log"
Jul 29 16:20:03.695372 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:20:03.695338    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-n8z9v_244016a2-b3e4-4f13-a387-6197b07e8be2/payload-pre-processing/0.log"
Jul 29 16:20:03.724843 ip-10-0-135-116 kubenswrapper[2575]: I0729 16:20:03.724813    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-knc5p_6fb56df1-ee67-48c1-be21-5b9faa36351e/payload-processing/0.log"