Jul 24 15:36:10.444048 ip-10-0-135-164 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 15:36:10.748563 ip-10-0-135-164 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 15:36:10.748563 ip-10-0-135-164 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 15:36:10.748563 ip-10-0-135-164 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 15:36:10.748563 ip-10-0-135-164 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 15:36:10.748563 ip-10-0-135-164 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 15:36:10.750057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.749945    2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 15:36:10.755174 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755158    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:36:10.755212 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755176    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:36:10.755212 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755180    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:36:10.755212 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755183    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:36:10.755212 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755187    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:36:10.755212 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755191    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:36:10.755212 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755193    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:36:10.755212 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755196    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:36:10.755212 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755198    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:36:10.755212 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755201    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:36:10.755212 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755203    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:36:10.755212 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755206    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:36:10.755212 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755210    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:36:10.755212 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755213    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:36:10.755212 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755216    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:36:10.755212 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755219    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755222    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755225    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755228    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755230    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755233    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755237    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755239    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755242    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755244    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755247    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755249    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755252    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755255    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755257    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755260    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755262    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755264    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755267    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755269    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:36:10.755601 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755272    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755274    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755277    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755279    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755281    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755284    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755286    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755289    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755292    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755295    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755297    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755300    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755303    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755306    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755308    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755312    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755316    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755319    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755322    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755325    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:36:10.756085 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755327    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755330    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755332    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755335    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755337    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755340    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755342    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755344    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755347    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755349    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755352    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755354    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755357    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755359    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755362    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755364    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755368    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755370    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755374    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755376    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:36:10.756569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755379    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755382    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755384    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755387    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755390    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755393    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755395    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755398    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755401    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755403    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755405    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755408    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755410    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755413    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755415    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755419    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755422    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755425    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755428    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755431    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:36:10.757058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755433    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755438    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755441    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.755443    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756194    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756201    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756204    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756209    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756213    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756217    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756220    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756223    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756226    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756229    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756232    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756234    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756237    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756240    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756242    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756245    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:36:10.757532 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756247    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756250    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756252    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756255    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756257    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756260    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756262    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756265    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756267    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756270    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756272    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756274    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756277    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756280    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756282    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756285    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756288    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756290    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756292    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756295    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:36:10.758086 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756298    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756300    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756303    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756306    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756309    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756311    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756314    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756316    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756319    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756321    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756324    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756326    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756328    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756331    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756333    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756335    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756338    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756340    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756342    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:36:10.758580 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756345    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756347    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756349    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756352    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756354    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756357    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756359    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756362    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756364    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756366    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756369    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756372    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756377    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756380    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756383    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756386    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756389    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756392    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756394    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:36:10.759062 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756397    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756399    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756401    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756404    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756407    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756409    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756411    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756414    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756417    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756419    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756423    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756426    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756429    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756431    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756434    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756436    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756438    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756441    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756444    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756446    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:36:10.759518 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756449    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756451    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756454    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756456    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756459    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756832    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756838    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756841    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756843    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756850    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756852    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756855    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756858    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756860    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756863    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756866    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756868    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756871    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756873    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756875    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:36:10.760110 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756878    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756881    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756883    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756886    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756890    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756893    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756896    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756899    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756902    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756905    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756909    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756912    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756915    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756918    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756921    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756923    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756926    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756928    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756931    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:36:10.760588 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756933    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756936    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756939    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756942    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756944    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756947    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756949    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756951    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756954    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756956    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756959    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756961    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756963    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756966    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756968    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756971    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756973    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756975    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756978    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756980    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:36:10.761061 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756983    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756985    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756988    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756993    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756997    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.756999    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757003    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757007    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757010    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757013    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757016    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757018    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757021    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757024    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757026    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757029    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757032    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757035    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757038    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757041    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:36:10.761531 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757044    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757048    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757050    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757053    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757055    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757058    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757060    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757063    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757066    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757068    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757071    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757073    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757076    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757078    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757080    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757085    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757089    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757092    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757094    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:36:10.762060 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757097    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757099    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757102    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757104    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757107    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757109    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757180    2561 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757187    2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757193    2561 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757197    2561 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757202    2561 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757221    2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757227    2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757231    2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757234    2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757237    2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757241    2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757244    2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757248    2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757251    2561 flags.go:64] FLAG: --cgroup-root=""
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757254    2561 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757256    2561 flags.go:64] FLAG: --client-ca-file=""
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757259    2561 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 15:36:10.762501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757262    2561 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757266    2561 flags.go:64] FLAG: --cluster-domain=""
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757269    2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757272    2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757275    2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757279    2561 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757284    2561 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757287    2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757290    2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757293    2561 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757296    2561 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757299    2561 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757301    2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757304    2561 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757307    2561 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757311    2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757314    2561 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757317    2561 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757319    2561 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757322    2561 flags.go:64] FLAG: --enable-server="true"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757326    2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757330    2561 flags.go:64] FLAG: --event-burst="100"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757333    2561 flags.go:64] FLAG: --event-qps="50"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757336    2561 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 15:36:10.763057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757339    2561 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757342    2561 flags.go:64] FLAG: --eviction-hard=""
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757346    2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757349    2561 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757352    2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757355    2561 flags.go:64] FLAG: --eviction-soft=""
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757357    2561 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757360    2561 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757363    2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757366    2561 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757369    2561 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757371    2561 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757374    2561 flags.go:64] FLAG: --feature-gates=""
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757379    2561 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757383    2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757386    2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757389    2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757392    2561 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757395    2561 flags.go:64] FLAG: --help="false"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757398    2561 flags.go:64] FLAG: --hostname-override="ip-10-0-135-164.ec2.internal"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757401    2561 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757404    2561 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757406    2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757410    2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 15:36:10.763651 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757413    2561 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757416    2561 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757418    2561 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757421    2561 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757424    2561 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757427    2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757430    2561 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757433    2561 flags.go:64] FLAG: --kube-reserved=""
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757436    2561 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757438    2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757441    2561 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757444    2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757447    2561 flags.go:64] FLAG: --lock-file=""
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757450    2561 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757452    2561 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757455    2561 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757460    2561 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757463    2561 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757466    2561 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757468    2561 flags.go:64] FLAG: --logging-format="text"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757471    2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757475    2561 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757479    2561 flags.go:64] FLAG: --manifest-url=""
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757482    2561 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757486    2561 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 15:36:10.764263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757489    2561 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757493    2561 flags.go:64] FLAG: --max-pods="110"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757496    2561 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757499    2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757502    2561 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757505    2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757507    2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757510    2561 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757513    2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757520    2561 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757523    2561 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757526    2561 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757529    2561 flags.go:64] FLAG: --pod-cidr=""
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757533    2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757538    2561 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757541    2561 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757544    2561 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757547    2561 flags.go:64] FLAG: --port="10250"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757550    2561 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757553    2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f0b2e8411b542702"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757556    2561 flags.go:64] FLAG: --qos-reserved=""
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757559    2561 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757561    2561 flags.go:64] FLAG: --register-node="true"
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757564    2561 flags.go:64] FLAG: --register-with-taints=""
Jul 24 15:36:10.764884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757568    2561 flags.go:64] FLAG: --registry-burst="10"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757570    2561 flags.go:64] FLAG: --registry-qps="5"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757573    2561 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757576    2561 flags.go:64] FLAG: --reserved-memory=""
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757580    2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757583    2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757587    2561 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757590    2561 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757593    2561 flags.go:64] FLAG: --runonce="false"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757596    2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757599    2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757601    2561 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757604    2561 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757607    2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757609    2561 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757612    2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757615    2561 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757618    2561 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757621    2561 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757623    2561 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757626    2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757630    2561 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757633    2561 flags.go:64] FLAG: --system-cgroups=""
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757635    2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757640    2561 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 15:36:10.765467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757643    2561 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757646    2561 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757649    2561 flags.go:64] FLAG: --tls-min-version=""
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757652    2561 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757655    2561 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757657    2561 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757660    2561 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757663    2561 flags.go:64] FLAG: --v="2"
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757666    2561 flags.go:64] FLAG: --version="false"
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757670    2561 flags.go:64] FLAG: --vmodule=""
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757673    2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.757678    2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757797    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757803    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757807    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757810    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757814    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757817    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757820    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757822    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757825    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757827    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:36:10.766100 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757830    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757832    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757844    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757848    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757851    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757854    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757857    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757860    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757863    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757865    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757867    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757870    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757872    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757875    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757877    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757880    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757882    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757884    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757887    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:36:10.766639 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757890    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757892    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757896    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757898    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757902    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757905    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757908    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757910    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757913    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757915    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757918    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757920    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757923    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757925    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757927    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757930    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757932    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757935    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757937    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:36:10.767143 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757940    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757942    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757945    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757948    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757950    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757952    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757955    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757957    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757960    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757963    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757965    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757967    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757969    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757972    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757974    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757978    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757981    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757986    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757989    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757992    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:36:10.767592 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757994    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757997    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.757999    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758001    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758004    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758006    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758009    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758011    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758014    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758016    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758019    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758021    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758024    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758026    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758029    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758031    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758034    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758036    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758039    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758041    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:36:10.768078 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758043    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:36:10.768545 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758046    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:36:10.768545 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758048    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:36:10.768545 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758050    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:36:10.768545 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758053    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:36:10.768545 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758055    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:36:10.768545 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758059    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:36:10.768545 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758064    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:36:10.768545 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758067    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:36:10.768545 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758072    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:36:10.768545 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.758074    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:36:10.768545 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.758080    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 15:36:10.768545 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.764684    2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 15:36:10.768545 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.764698    2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 15:36:10.768545 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.765347    2561 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.765354    2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765410    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765414    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765417    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765420    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765422    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765425    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765428    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765430    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765433    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765435    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765438    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765441    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765445    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765447    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765450    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765452    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765455    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:36:10.768908 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765458    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765460    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765462    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765467    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765470    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765473    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765477    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765480    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765483    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765486    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765489    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765492    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765495    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765498    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765501    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765503    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765506    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765510    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765513    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:36:10.769354 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765516    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765519    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765522    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765526    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765528    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765531    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765534    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765536    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765539    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765542    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765544    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765546    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765549    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765551    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765554    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765556    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765559    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765561    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765563    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765566    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:36:10.769830 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765568    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765571    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765573    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765576    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765578    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765582    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765584    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765587    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765589    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765592    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765594    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765597    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765599    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765602    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765604    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765607    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765609    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765612    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765615    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765617    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:36:10.770347 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765619    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765622    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765625    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765627    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765629    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765632    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765634    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765636    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765639    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765641    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765644    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765646    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765649    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765652    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765655    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765658    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765661    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765664    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765667    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765669    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:36:10.770997 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765672    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:36:10.771569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765674    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:36:10.771569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765677    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:36:10.771569 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.765682    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 15:36:10.771569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765814    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:36:10.771569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765819    2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:36:10.771569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765822    2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:36:10.771569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765824    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:36:10.771569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765827    2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:36:10.771569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765829    2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:36:10.771569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765832    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:36:10.771569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765841    2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:36:10.771569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765845    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:36:10.771569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765848    2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:36:10.771569 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765852    2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765856    2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765859    2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765862    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765865    2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765868    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765871    2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765873    2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765876    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765879    2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765881    2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765884    2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765886    2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765889    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765891    2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765894    2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765896    2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765899    2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765901    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765904    2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:36:10.772138 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765906    2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765909    2561 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765911    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765913    2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765916    2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765918    2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765921    2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765923    2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765925    2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765928    2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765930    2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765932    2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765935    2561 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765938    2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765941    2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765944    2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765947    2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765949    2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765951    2561 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765954    2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:36:10.772769 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765956    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765958    2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765961    2561 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765963    2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765966    2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765969    2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765973    2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765975    2561 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765978    2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765981    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765983    2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765986    2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765988    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765991    2561 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765993    2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765995    2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.765998    2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766000    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766003    2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:36:10.773314 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766005    2561 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766007    2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766010    2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766013    2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766015    2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766017    2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766020    2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766022    2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766025    2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766027    2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766030    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766033    2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766035    2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766038    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766040    2561 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766042    2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766045    2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766047    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766049    2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766052    2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:36:10.773802 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766054    2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:36:10.774294 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766057    2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:36:10.774294 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766059    2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:36:10.774294 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766061    2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:36:10.774294 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766064    2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:36:10.774294 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766066    2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:36:10.774294 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766069    2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:36:10.774294 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766071    2561 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:36:10.774294 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766073    2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:36:10.774294 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:10.766076    2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:36:10.774294 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.766081    2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 15:36:10.774294 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.766184    2561 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 15:36:10.774294 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.768227    2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 15:36:10.774294 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.769427    2561 server.go:1019] "Starting client certificate rotation"
Jul 24 15:36:10.774294 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.769529    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 15:36:10.774641 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.769568    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 15:36:10.774641 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.774325    2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 15:36:10.775500 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.775485    2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 15:36:10.784821 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.784802    2561 log.go:25] "Validated CRI v1 runtime API"
Jul 24 15:36:10.788955 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.788935    2561 log.go:25] "Validated CRI v1 image API"
Jul 24 15:36:10.789718 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.789702    2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 15:36:10.791796 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.791779    2561 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 87a39e39-3180-49e3-8b6c-5466f4767c0a:/dev/nvme0n1p3 df696aaa-1622-4b15-bfa8-66a6b5bd4460:/dev/nvme0n1p4]
Jul 24 15:36:10.791864 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.791796    2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 15:36:10.796587 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.796483    2561 manager.go:216] Machine: {Timestamp:2026-07-24 15:36:10.795387648 +0000 UTC m=+0.258016839 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100841 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2feefb144faf18919ce28f1c533925 SystemUUID:ec2feefb-144f-af18-919c-e28f1c533925 BootID:2faa2ea6-1c88-4069-a967-b741c178f2b2 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c2:03:31:17:81 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c2:03:31:17:81 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:d9:df:75:0f:67 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 15:36:10.796587 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.796583    2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 15:36:10.796688 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.796658    2561 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 15:36:10.799145 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.799116    2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 15:36:10.799277 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.799150    2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-164.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 15:36:10.800227 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.800217    2561 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 15:36:10.800266 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.800230    2561 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 15:36:10.800266 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.800238    2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 15:36:10.800783 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.800772    2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 15:36:10.801678 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.801669    2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 15:36:10.802691 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.802675    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 15:36:10.804439 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.804427    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 15:36:10.804613 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.804602    2561 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 15:36:10.804654 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.804634    2561 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 15:36:10.804685 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.804659    2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 15:36:10.804712 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.804693    2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 15:36:10.804712 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.804703    2561 kubelet.go:398] "Adding apiserver pod source"
Jul 24 15:36:10.804796 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.804715    2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 15:36:10.805432 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.805404    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 15:36:10.805528 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.805445    2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 15:36:10.808804 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.808571    2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 15:36:10.810604 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.810590    2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 15:36:10.810673 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.810620    2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 15:36:10.812051 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.812033    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 15:36:10.812110 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.812057    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 15:36:10.812110 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.812066    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 15:36:10.812110 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.812073    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 15:36:10.812110 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.812083    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 15:36:10.812110 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.812090    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 15:36:10.812110 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.812099    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 15:36:10.812110 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.812105    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 15:36:10.812110 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.812113    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 15:36:10.812379 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.812119    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 15:36:10.812379 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.812129    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 15:36:10.812379 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.812135    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 15:36:10.813272 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.813258    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 15:36:10.813272 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.813269    2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 15:36:10.816439 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.816421    2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-164.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 15:36:10.816513 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:10.816446    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 15:36:10.816513 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:10.816452    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 15:36:10.816641 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.816554    2561 server.go:1268] "Started kubelet"
Jul 24 15:36:10.816686 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.816626    2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 15:36:10.816686 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.816672    2561 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 15:36:10.817261 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.817215    2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 15:36:10.817507 ip-10-0-135-164 systemd[1]: Started Kubernetes Kubelet.
Jul 24 15:36:10.819062 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.819046    2561 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 15:36:10.820694 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.820676    2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 15:36:10.821659 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:10.821643    2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 15:36:10.824632 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:10.823909    2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-164.ec2.internal.18c543cc027ede10  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-164.ec2.internal,UID:ip-10-0-135-164.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-164.ec2.internal,},FirstTimestamp:2026-07-24 15:36:10.816527888 +0000 UTC m=+0.279157078,LastTimestamp:2026-07-24 15:36:10.816527888 +0000 UTC m=+0.279157078,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-164.ec2.internal,}"
Jul 24 15:36:10.826658 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.826644    2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 15:36:10.826729 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.826656    2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 15:36:10.826807 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:10.826775    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-164.ec2.internal\" not found"
Jul 24 15:36:10.827246 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.827227    2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 15:36:10.827335 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.827228    2561 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 15:36:10.827335 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.827274    2561 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 15:36:10.827421 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.827402    2561 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 15:36:10.827421 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.827410    2561 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 15:36:10.827663 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.827516    2561 factory.go:55] Registering systemd factory
Jul 24 15:36:10.827663 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.827534    2561 factory.go:223] Registration of the systemd container factory successfully
Jul 24 15:36:10.827793 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.827757    2561 factory.go:153] Registering CRI-O factory
Jul 24 15:36:10.827793 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.827770    2561 factory.go:223] Registration of the crio container factory successfully
Jul 24 15:36:10.827877 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.827824    2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 15:36:10.827877 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.827846    2561 factory.go:103] Registering Raw factory
Jul 24 15:36:10.827877 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.827860    2561 manager.go:1196] Started watching for new ooms in manager
Jul 24 15:36:10.828192 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.828178    2561 manager.go:349] Starting recovery of all containers
Jul 24 15:36:10.828793 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.828755    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4vfh6"
Jul 24 15:36:10.829469 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:10.829407    2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 15:36:10.829547 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:10.829514    2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-164.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 15:36:10.836479 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.836452    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4vfh6"
Jul 24 15:36:10.840101 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.840084    2561 manager.go:354] Recovery completed
Jul 24 15:36:10.841394 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:10.841364    2561 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 24 15:36:10.844402 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.844387    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:36:10.846090 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.846075    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:36:10.846168 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.846108    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:36:10.846168 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.846123    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:36:10.846615 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.846600    2561 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 15:36:10.846615 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.846612    2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 15:36:10.846708 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.846629    2561 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 15:36:10.846708 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.846643    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:36:10.847262 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.847248    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:36:10.847332 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.847276    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:36:10.847332 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.847290    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:36:10.849164 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.849151    2561 policy_none.go:49] "None policy: Start"
Jul 24 15:36:10.849209 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.849171    2561 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 15:36:10.849209 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.849185    2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 15:36:10.851218 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.851207    2561 policy_none.go:47] "Start"
Jul 24 15:36:10.892506 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.892315    2561 manager.go:337] "Starting Device Plugin manager"
Jul 24 15:36:10.892603 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:10.892535    2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 15:36:10.892603 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.892546    2561 server.go:87] "Starting device plugin registration server"
Jul 24 15:36:10.892780 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.892769    2561 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 15:36:10.892841 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.892782    2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 15:36:10.892901 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.892878    2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 15:36:10.892979 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.892967    2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 15:36:10.893021 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.892980    2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 15:36:10.893482 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:10.893462    2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 15:36:10.893564 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:10.893514    2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-164.ec2.internal\" not found"
Jul 24 15:36:10.992923 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.992892    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:36:10.995151 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.993900    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:36:10.995151 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.993933    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:36:10.995151 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.993945    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:36:10.995151 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.993986    2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-164.ec2.internal"
Jul 24 15:36:10.999957 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:10.999919    2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-164.ec2.internal"
Jul 24 15:36:10.999957 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:10.999942    2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-164.ec2.internal\": node \"ip-10-0-135-164.ec2.internal\" not found"
Jul 24 15:36:11.026269 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:11.026243    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-164.ec2.internal\" not found"
Jul 24 15:36:11.046539 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.046507    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 15:36:11.047905 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.047886    2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 15:36:11.047991 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.047931    2561 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 15:36:11.047991 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.047954    2561 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 15:36:11.048093 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:11.047993    2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 15:36:11.049934 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.049915    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:36:11.127235 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:11.127199    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-164.ec2.internal\" not found"
Jul 24 15:36:11.148524 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.148501    2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-164.ec2.internal"]
Jul 24 15:36:11.148591 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.148561    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:36:11.149513 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.149494    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:36:11.149561 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.149531    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:36:11.149561 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.149545    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:36:11.151936 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.151925    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:36:11.152087 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.152059    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal"
Jul 24 15:36:11.152134 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.152109    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:36:11.152689 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.152676    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:36:11.152727 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.152707    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:36:11.152727 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.152721    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:36:11.153177 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.153163    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:36:11.153222 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.153189    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:36:11.153222 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.153198    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:36:11.155409 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.155397    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-164.ec2.internal"
Jul 24 15:36:11.155457 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.155420    2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:36:11.156104 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.156087    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:36:11.156189 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.156109    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:36:11.156189 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.156125    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:36:11.171269 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:11.171245    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-164.ec2.internal\" not found" node="ip-10-0-135-164.ec2.internal"
Jul 24 15:36:11.175175 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:11.175159    2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-164.ec2.internal\" not found" node="ip-10-0-135-164.ec2.internal"
Jul 24 15:36:11.227580 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:11.227557    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-164.ec2.internal\" not found"
Jul 24 15:36:11.229590 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.229550    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c43182eb3bfc6cb96f40919d9e144573-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal\" (UID: \"c43182eb3bfc6cb96f40919d9e144573\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal"
Jul 24 15:36:11.229654 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.229597    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c43182eb3bfc6cb96f40919d9e144573-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal\" (UID: \"c43182eb3bfc6cb96f40919d9e144573\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal"
Jul 24 15:36:11.229654 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.229614    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b4237762195c7e99a7a069493539ac56-config\") pod \"kube-apiserver-proxy-ip-10-0-135-164.ec2.internal\" (UID: \"b4237762195c7e99a7a069493539ac56\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-164.ec2.internal"
Jul 24 15:36:11.328133 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:11.328069    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-164.ec2.internal\" not found"
Jul 24 15:36:11.330226 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.330207    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c43182eb3bfc6cb96f40919d9e144573-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal\" (UID: \"c43182eb3bfc6cb96f40919d9e144573\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal"
Jul 24 15:36:11.330281 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.330234    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c43182eb3bfc6cb96f40919d9e144573-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal\" (UID: \"c43182eb3bfc6cb96f40919d9e144573\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal"
Jul 24 15:36:11.330281 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.330254    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b4237762195c7e99a7a069493539ac56-config\") pod \"kube-apiserver-proxy-ip-10-0-135-164.ec2.internal\" (UID: \"b4237762195c7e99a7a069493539ac56\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-164.ec2.internal"
Jul 24 15:36:11.330356 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.330305    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b4237762195c7e99a7a069493539ac56-config\") pod \"kube-apiserver-proxy-ip-10-0-135-164.ec2.internal\" (UID: \"b4237762195c7e99a7a069493539ac56\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-164.ec2.internal"
Jul 24 15:36:11.330356 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.330314    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c43182eb3bfc6cb96f40919d9e144573-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal\" (UID: \"c43182eb3bfc6cb96f40919d9e144573\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal"
Jul 24 15:36:11.330356 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.330314    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c43182eb3bfc6cb96f40919d9e144573-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal\" (UID: \"c43182eb3bfc6cb96f40919d9e144573\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal"
Jul 24 15:36:11.428517 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:11.428472    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-164.ec2.internal\" not found"
Jul 24 15:36:11.473809 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.473790    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal"
Jul 24 15:36:11.478936 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.478718    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-164.ec2.internal"
Jul 24 15:36:11.529174 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:11.529142    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-164.ec2.internal\" not found"
Jul 24 15:36:11.629689 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:11.629604    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-164.ec2.internal\" not found"
Jul 24 15:36:11.634737 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.634721    2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:36:11.730657 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:11.730626    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-164.ec2.internal\" not found"
Jul 24 15:36:11.768971 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.768947    2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 15:36:11.769396 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.769074    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 15:36:11.769396 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.769101    2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 15:36:11.830801 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:11.830770    2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-164.ec2.internal\" not found"
Jul 24 15:36:11.838530 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.838512    2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:36:11.842172 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.842149    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 15:31:11 +0000 UTC" deadline="2028-03-28 16:45:39.278627926 +0000 UTC"
Jul 24 15:36:11.842172 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.842172    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14713h9m27.436458601s"
Jul 24 15:36:11.860590 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.859333    2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:36:11.927546 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.927483    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-164.ec2.internal"
Jul 24 15:36:11.934962 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.934943    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 15:36:11.935697 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:11.935685    2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal"
Jul 24 15:36:12.996596 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:12.996575    2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 15:36:13.105136 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:13.105101    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc43182eb3bfc6cb96f40919d9e144573.slice/crio-7ce16baba7e820bba1f5bafb387a23c9fdc7a83aaf4df960e666aa877cd2de4d WatchSource:0}: Error finding container 7ce16baba7e820bba1f5bafb387a23c9fdc7a83aaf4df960e666aa877cd2de4d: Status 404 returned error can't find the container with id 7ce16baba7e820bba1f5bafb387a23c9fdc7a83aaf4df960e666aa877cd2de4d
Jul 24 15:36:13.105835 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:13.105813    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4237762195c7e99a7a069493539ac56.slice/crio-3867f6ae1741bab9123b2d7bbd7cf1bb4b74c05d3a3acead0f1df1f6519b3a22 WatchSource:0}: Error finding container 3867f6ae1741bab9123b2d7bbd7cf1bb4b74c05d3a3acead0f1df1f6519b3a22: Status 404 returned error can't find the container with id 3867f6ae1741bab9123b2d7bbd7cf1bb4b74c05d3a3acead0f1df1f6519b3a22
Jul 24 15:36:13.109967 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.109955    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:36:13.857275 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.857242    2561 apiserver.go:52] "Watching apiserver"
Jul 24 15:36:13.863123 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.863095    2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 15:36:13.864917 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.864882    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-lsj8p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal","openshift-network-diagnostics/network-check-target-qrsfq","openshift-network-operator/iptables-alerter-lmdw4","kube-system/global-pull-secret-syncer-j8fqw","kube-system/konnectivity-agent-lvgwz","openshift-cluster-node-tuning-operator/tuned-fqlp8","openshift-dns/node-resolver-s6wn2","openshift-multus/multus-5g4s6","openshift-multus/multus-additional-cni-plugins-l7xpv","openshift-multus/network-metrics-daemon-vmnkg","openshift-ovn-kubernetes/ovnkube-node-th7v2","kube-system/kube-apiserver-proxy-ip-10-0-135-164.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"]
Jul 24 15:36:13.867961 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.867934    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lsj8p"
Jul 24 15:36:13.869992 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.869692    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:13.869992 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:13.869777    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:13.870161 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.870112    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b9n24\""
Jul 24 15:36:13.870491 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.870471    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 15:36:13.870815 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.870796    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 15:36:13.870984 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.870968    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 15:36:13.871920 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.871904    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lmdw4"
Jul 24 15:36:13.873962 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.873941    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 15:36:13.874376 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.874359    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 15:36:13.874553 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.874537    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:13.874798 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:13.874600    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8fqw" podUID="94e9b736-6944-4781-9b40-500ec82d10e2"
Jul 24 15:36:13.874798 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.874607    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8q6l9\""
Jul 24 15:36:13.874798 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.874652    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 15:36:13.876290 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.876273    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lvgwz"
Jul 24 15:36:13.879912 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.878047    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 15:36:13.879912 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.878495    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.879912 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.879358    2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 15:36:13.880762 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.880588    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 15:36:13.880762 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.880617    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wsn8d\""
Jul 24 15:36:13.881082 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.881057    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s6wn2"
Jul 24 15:36:13.882155 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.882134    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-km2d8\""
Jul 24 15:36:13.882245 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.882208    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 15:36:13.882566 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.882550    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 15:36:13.883142 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.883127    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 15:36:13.883429 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.883407    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 15:36:13.884467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.884450    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-l7vfx\""
Jul 24 15:36:13.884811 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.884747    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.886688 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.886304    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:13.887422 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.887403    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 15:36:13.888921 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.888682    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 15:36:13.889067 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.888755    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fdrpd\""
Jul 24 15:36:13.889537 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.889506    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 15:36:13.889906 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.889891    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 15:36:13.893015 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.890094    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-host\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.893110 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.893054    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f144fc9d-a69b-4cb5-b99c-2092a7905b32-agent-certs\") pod \"konnectivity-agent-lvgwz\" (UID: \"f144fc9d-a69b-4cb5-b99c-2092a7905b32\") " pod="kube-system/konnectivity-agent-lvgwz"
Jul 24 15:36:13.893110 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.893085    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7ecbc05-a1ef-4546-8915-3ca20c060b37-tmp\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.893218 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.893115    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/076c0fb1-5f86-4a0b-8725-100ea9966c41-host\") pod \"node-ca-lsj8p\" (UID: \"076c0fb1-5f86-4a0b-8725-100ea9966c41\") " pod="openshift-image-registry/node-ca-lsj8p"
Jul 24 15:36:13.893218 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.893138    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/94e9b736-6944-4781-9b40-500ec82d10e2-dbus\") pod \"global-pull-secret-syncer-j8fqw\" (UID: \"94e9b736-6944-4781-9b40-500ec82d10e2\") " pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:13.893218 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.890757    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 15:36:13.893362 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.891171    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 15:36:13.893409 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.891946    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:13.893409 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.893402    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-kubernetes\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.893501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.893437    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-sys\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.893501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.892601    2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 15:36:13.893501 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.893476    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/076c0fb1-5f86-4a0b-8725-100ea9966c41-serviceca\") pod \"node-ca-lsj8p\" (UID: \"076c0fb1-5f86-4a0b-8725-100ea9966c41\") " pod="openshift-image-registry/node-ca-lsj8p"
Jul 24 15:36:13.893737 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.891971    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.893737 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:13.893464    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmnkg" podUID="f1958e1f-5868-4a15-847c-28d61994d060"
Jul 24 15:36:13.893737 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.891217    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-n6wnm\""
Jul 24 15:36:13.893892 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.893547    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc5b0603-f613-4f94-9c67-e5602c0ca649-host-slash\") pod \"iptables-alerter-lmdw4\" (UID: \"fc5b0603-f613-4f94-9c67-e5602c0ca649\") " pod="openshift-network-operator/iptables-alerter-lmdw4"
Jul 24 15:36:13.893892 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.893787    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/94e9b736-6944-4781-9b40-500ec82d10e2-kubelet-config\") pod \"global-pull-secret-syncer-j8fqw\" (UID: \"94e9b736-6944-4781-9b40-500ec82d10e2\") " pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:13.893892 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.893808    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-var-lib-kubelet\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.893892 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.893832    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aa3224e2-ecb7-4164-aa12-48213a6a5210-hosts-file\") pod \"node-resolver-s6wn2\" (UID: \"aa3224e2-ecb7-4164-aa12-48213a6a5210\") " pod="openshift-dns/node-resolver-s6wn2"
Jul 24 15:36:13.894080 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.893897    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-systemd\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.894080 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.893924    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f144fc9d-a69b-4cb5-b99c-2092a7905b32-konnectivity-ca\") pod \"konnectivity-agent-lvgwz\" (UID: \"f144fc9d-a69b-4cb5-b99c-2092a7905b32\") " pod="kube-system/konnectivity-agent-lvgwz"
Jul 24 15:36:13.894080 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.893942    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa3224e2-ecb7-4164-aa12-48213a6a5210-tmp-dir\") pod \"node-resolver-s6wn2\" (UID: \"aa3224e2-ecb7-4164-aa12-48213a6a5210\") " pod="openshift-dns/node-resolver-s6wn2"
Jul 24 15:36:13.894080 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.893965    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-sysctl-conf\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.894271 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.894160    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v55jn\" (UniqueName: \"kubernetes.io/projected/c7ecbc05-a1ef-4546-8915-3ca20c060b37-kube-api-access-v55jn\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.894271 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.894209    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jngr\" (UniqueName: \"kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr\") pod \"network-check-target-qrsfq\" (UID: \"ac6f7393-c091-44d0-9e64-4b58651a24b8\") " pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:13.894371 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.894279    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7b5v\" (UniqueName: \"kubernetes.io/projected/aa3224e2-ecb7-4164-aa12-48213a6a5210-kube-api-access-v7b5v\") pod \"node-resolver-s6wn2\" (UID: \"aa3224e2-ecb7-4164-aa12-48213a6a5210\") " pod="openshift-dns/node-resolver-s6wn2"
Jul 24 15:36:13.894371 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.894333    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fc5b0603-f613-4f94-9c67-e5602c0ca649-iptables-alerter-script\") pod \"iptables-alerter-lmdw4\" (UID: \"fc5b0603-f613-4f94-9c67-e5602c0ca649\") " pod="openshift-network-operator/iptables-alerter-lmdw4"
Jul 24 15:36:13.894468 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.894380    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-modprobe-d\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.894468 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.894417    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-sysconfig\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.894468 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.894446    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-run\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.894609 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.894479    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-tuned\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.894609 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.894549    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrbk9\" (UniqueName: \"kubernetes.io/projected/076c0fb1-5f86-4a0b-8725-100ea9966c41-kube-api-access-qrbk9\") pod \"node-ca-lsj8p\" (UID: \"076c0fb1-5f86-4a0b-8725-100ea9966c41\") " pod="openshift-image-registry/node-ca-lsj8p"
Jul 24 15:36:13.894609 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.894590    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b86vx\" (UniqueName: \"kubernetes.io/projected/fc5b0603-f613-4f94-9c67-e5602c0ca649-kube-api-access-b86vx\") pod \"iptables-alerter-lmdw4\" (UID: \"fc5b0603-f613-4f94-9c67-e5602c0ca649\") " pod="openshift-network-operator/iptables-alerter-lmdw4"
Jul 24 15:36:13.894857 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.894610    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret\") pod \"global-pull-secret-syncer-j8fqw\" (UID: \"94e9b736-6944-4781-9b40-500ec82d10e2\") " pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:13.894857 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.894628    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-sysctl-d\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.894857 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.894702    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-lib-modules\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.895743 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.895726    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:13.895972 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.895950    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qj62g\""
Jul 24 15:36:13.896060 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.895987    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 15:36:13.896653 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.896622    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 15:36:13.896763 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.896701    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 15:36:13.897018 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.896998    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 15:36:13.900013 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.899991    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 15:36:13.900084 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.900032    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 15:36:13.900084 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.900076    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 15:36:13.900202 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.900186    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 15:36:13.900241 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.900223    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 15:36:13.900626 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.900554    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vglbh\""
Jul 24 15:36:13.917219 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.917198    2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2czwm"
Jul 24 15:36:13.924543 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.924529    2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2czwm"
Jul 24 15:36:13.980889 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.980860    2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 15:36:13.995675 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.995631    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jngr\" (UniqueName: \"kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr\") pod \"network-check-target-qrsfq\" (UID: \"ac6f7393-c091-44d0-9e64-4b58651a24b8\") " pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:13.995819 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.995681    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7b5v\" (UniqueName: \"kubernetes.io/projected/aa3224e2-ecb7-4164-aa12-48213a6a5210-kube-api-access-v7b5v\") pod \"node-resolver-s6wn2\" (UID: \"aa3224e2-ecb7-4164-aa12-48213a6a5210\") " pod="openshift-dns/node-resolver-s6wn2"
Jul 24 15:36:13.995819 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.995702    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-modprobe-d\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.995932 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.995847    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-modprobe-d\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.995932 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.995888    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-run\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.995932 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.995914    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-tuned\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.996070 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.995939    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-etc-openvswitch\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.996070 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.995964    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-socket-dir\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:13.996070 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.995983    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-run\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.996070 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.995989    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:13.996070 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996040    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-cni-bin\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.996304 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996073    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/933739ec-2f20-4ea6-80ea-158f9cd48902-ovnkube-script-lib\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.996304 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996097    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-multus-daemon-config\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.996304 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996125    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-lib-modules\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.996304 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996167    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-host\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.996304 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996192    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxjp8\" (UniqueName: \"kubernetes.io/projected/f1958e1f-5868-4a15-847c-28d61994d060-kube-api-access-qxjp8\") pod \"network-metrics-daemon-vmnkg\" (UID: \"f1958e1f-5868-4a15-847c-28d61994d060\") " pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:13.996304 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996209    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f144fc9d-a69b-4cb5-b99c-2092a7905b32-agent-certs\") pod \"konnectivity-agent-lvgwz\" (UID: \"f144fc9d-a69b-4cb5-b99c-2092a7905b32\") " pod="kube-system/konnectivity-agent-lvgwz"
Jul 24 15:36:13.996304 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996231    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7ecbc05-a1ef-4546-8915-3ca20c060b37-tmp\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.996304 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996257    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-log-socket\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.996304 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996259    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-host\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.996304 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996274    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:13.996304 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996289    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-cnibin\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:13.996304 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996304    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/076c0fb1-5f86-4a0b-8725-100ea9966c41-host\") pod \"node-ca-lsj8p\" (UID: \"076c0fb1-5f86-4a0b-8725-100ea9966c41\") " pod="openshift-image-registry/node-ca-lsj8p"
Jul 24 15:36:13.996881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996322    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-sys\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.996881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996344    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97c7v\" (UniqueName: \"kubernetes.io/projected/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-kube-api-access-97c7v\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:13.996881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996257    2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 15:36:13.996881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996386    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-sys\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.996881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996415    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-os-release\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.996881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996444    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/076c0fb1-5f86-4a0b-8725-100ea9966c41-serviceca\") pod \"node-ca-lsj8p\" (UID: \"076c0fb1-5f86-4a0b-8725-100ea9966c41\") " pod="openshift-image-registry/node-ca-lsj8p"
Jul 24 15:36:13.996881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996470    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc5b0603-f613-4f94-9c67-e5602c0ca649-host-slash\") pod \"iptables-alerter-lmdw4\" (UID: \"fc5b0603-f613-4f94-9c67-e5602c0ca649\") " pod="openshift-network-operator/iptables-alerter-lmdw4"
Jul 24 15:36:13.996881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996492    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-run-systemd\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.996881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996516    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-etc-selinux\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:13.996881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996540    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-system-cni-dir\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:13.996881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996565    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs\") pod \"network-metrics-daemon-vmnkg\" (UID: \"f1958e1f-5868-4a15-847c-28d61994d060\") " pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:13.996881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996587    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-cni-binary-copy\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.996881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996298    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-lib-modules\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.996881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996647    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/076c0fb1-5f86-4a0b-8725-100ea9966c41-host\") pod \"node-ca-lsj8p\" (UID: \"076c0fb1-5f86-4a0b-8725-100ea9966c41\") " pod="openshift-image-registry/node-ca-lsj8p"
Jul 24 15:36:13.996881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996695    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aa3224e2-ecb7-4164-aa12-48213a6a5210-hosts-file\") pod \"node-resolver-s6wn2\" (UID: \"aa3224e2-ecb7-4164-aa12-48213a6a5210\") " pod="openshift-dns/node-resolver-s6wn2"
Jul 24 15:36:13.996881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996719    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-systemd\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.996881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996745    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/933739ec-2f20-4ea6-80ea-158f9cd48902-ovnkube-config\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.997683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996782    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-registration-dir\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:13.997683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996810    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-system-cni-dir\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.997683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996838    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-run-netns\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.997683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996871    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/933739ec-2f20-4ea6-80ea-158f9cd48902-ovn-node-metrics-cert\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.997683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996901    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa3224e2-ecb7-4164-aa12-48213a6a5210-tmp-dir\") pod \"node-resolver-s6wn2\" (UID: \"aa3224e2-ecb7-4164-aa12-48213a6a5210\") " pod="openshift-dns/node-resolver-s6wn2"
Jul 24 15:36:13.997683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996927    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-sysctl-conf\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.997683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996950    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v55jn\" (UniqueName: \"kubernetes.io/projected/c7ecbc05-a1ef-4546-8915-3ca20c060b37-kube-api-access-v55jn\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.997683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.996972    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-cni-netd\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.997683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997008    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.997683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997034    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnk5s\" (UniqueName: \"kubernetes.io/projected/47cf7e7d-f0af-4ace-b76d-240b87f40041-kube-api-access-rnk5s\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:13.997683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997059    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fc5b0603-f613-4f94-9c67-e5602c0ca649-iptables-alerter-script\") pod \"iptables-alerter-lmdw4\" (UID: \"fc5b0603-f613-4f94-9c67-e5602c0ca649\") " pod="openshift-network-operator/iptables-alerter-lmdw4"
Jul 24 15:36:13.997683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997077    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/076c0fb1-5f86-4a0b-8725-100ea9966c41-serviceca\") pod \"node-ca-lsj8p\" (UID: \"076c0fb1-5f86-4a0b-8725-100ea9966c41\") " pod="openshift-image-registry/node-ca-lsj8p"
Jul 24 15:36:13.997683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997085    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-sysconfig\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.997683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997118    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-device-dir\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:13.997683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997144    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-cni-binary-copy\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:13.997683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997150    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc5b0603-f613-4f94-9c67-e5602c0ca649-host-slash\") pod \"iptables-alerter-lmdw4\" (UID: \"fc5b0603-f613-4f94-9c67-e5602c0ca649\") " pod="openshift-network-operator/iptables-alerter-lmdw4"
Jul 24 15:36:13.997683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997167    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-cnibin\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.998265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997189    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-var-lib-kubelet\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.998265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997211    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-hostroot\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.998265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997462    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/aa3224e2-ecb7-4164-aa12-48213a6a5210-hosts-file\") pod \"node-resolver-s6wn2\" (UID: \"aa3224e2-ecb7-4164-aa12-48213a6a5210\") " pod="openshift-dns/node-resolver-s6wn2"
Jul 24 15:36:13.998265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997519    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-systemd\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.998265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997724    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6hkz\" (UniqueName: \"kubernetes.io/projected/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-kube-api-access-x6hkz\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.998265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997769    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-run-ovn\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.998265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997801    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-multus-conf-dir\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.998265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997837    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrbk9\" (UniqueName: \"kubernetes.io/projected/076c0fb1-5f86-4a0b-8725-100ea9966c41-kube-api-access-qrbk9\") pod \"node-ca-lsj8p\" (UID: \"076c0fb1-5f86-4a0b-8725-100ea9966c41\") " pod="openshift-image-registry/node-ca-lsj8p"
Jul 24 15:36:13.998265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997865    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b86vx\" (UniqueName: \"kubernetes.io/projected/fc5b0603-f613-4f94-9c67-e5602c0ca649-kube-api-access-b86vx\") pod \"iptables-alerter-lmdw4\" (UID: \"fc5b0603-f613-4f94-9c67-e5602c0ca649\") " pod="openshift-network-operator/iptables-alerter-lmdw4"
Jul 24 15:36:13.998265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997893    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret\") pod \"global-pull-secret-syncer-j8fqw\" (UID: \"94e9b736-6944-4781-9b40-500ec82d10e2\") " pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:13.998265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997921    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-sysctl-d\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.998265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997947    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-kubelet\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.998265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.997975    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:13.998265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998033    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-var-lib-cni-bin\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.998265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998061    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-systemd-units\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.998265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998091    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-run-openvswitch\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.998265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998115    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-sys-fs\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:13.998893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998176    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-var-lib-cni-multus\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.998893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998208    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/94e9b736-6944-4781-9b40-500ec82d10e2-dbus\") pod \"global-pull-secret-syncer-j8fqw\" (UID: \"94e9b736-6944-4781-9b40-500ec82d10e2\") " pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:13.998893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998233    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-kubernetes\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.998893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998261    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-multus-cni-dir\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.998893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998288    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/94e9b736-6944-4781-9b40-500ec82d10e2-kubelet-config\") pod \"global-pull-secret-syncer-j8fqw\" (UID: \"94e9b736-6944-4781-9b40-500ec82d10e2\") " pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:13.998893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998311    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-slash\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.998893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998348    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-run-ovn-kubernetes\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.998893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998358    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa3224e2-ecb7-4164-aa12-48213a6a5210-tmp-dir\") pod \"node-resolver-s6wn2\" (UID: \"aa3224e2-ecb7-4164-aa12-48213a6a5210\") " pod="openshift-dns/node-resolver-s6wn2"
Jul 24 15:36:13.998893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998376    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:13.998893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998400    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-var-lib-kubelet\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.998893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998421    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-run-netns\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.998893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998446    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-var-lib-openvswitch\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.998893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998479    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-node-log\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.998893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998500    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-sysctl-conf\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.998893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998502    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-multus-socket-dir-parent\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.998893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998539    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-etc-kubernetes\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.998893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.998590    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-sysconfig\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.999618 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:13.999078    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:36:13.999618 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.999109    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gv59\" (UniqueName: \"kubernetes.io/projected/933739ec-2f20-4ea6-80ea-158f9cd48902-kube-api-access-2gv59\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.999618 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:13.999158    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret podName:94e9b736-6944-4781-9b40-500ec82d10e2 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:14.499125621 +0000 UTC m=+2.909836239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret") pod "global-pull-secret-syncer-j8fqw" (UID: "94e9b736-6944-4781-9b40-500ec82d10e2") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:36:13.999618 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.999177    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f144fc9d-a69b-4cb5-b99c-2092a7905b32-konnectivity-ca\") pod \"konnectivity-agent-lvgwz\" (UID: \"f144fc9d-a69b-4cb5-b99c-2092a7905b32\") " pod="kube-system/konnectivity-agent-lvgwz"
Jul 24 15:36:13.999618 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.999206    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-kubernetes\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.999618 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.999205    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/933739ec-2f20-4ea6-80ea-158f9cd48902-env-overrides\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:13.999618 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.999249    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-os-release\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:13.999618 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.999264    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/94e9b736-6944-4781-9b40-500ec82d10e2-kubelet-config\") pod \"global-pull-secret-syncer-j8fqw\" (UID: \"94e9b736-6944-4781-9b40-500ec82d10e2\") " pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:13.999618 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.999274    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-run-k8s-cni-cncf-io\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.999618 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.999308    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-run-multus-certs\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:13.999618 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.999079    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fc5b0603-f613-4f94-9c67-e5602c0ca649-iptables-alerter-script\") pod \"iptables-alerter-lmdw4\" (UID: \"fc5b0603-f613-4f94-9c67-e5602c0ca649\") " pod="openshift-network-operator/iptables-alerter-lmdw4"
Jul 24 15:36:13.999618 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.999436    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-sysctl-d\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:13.999618 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.999491    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7ecbc05-a1ef-4546-8915-3ca20c060b37-var-lib-kubelet\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:14.000121 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.999678    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/94e9b736-6944-4781-9b40-500ec82d10e2-dbus\") pod \"global-pull-secret-syncer-j8fqw\" (UID: \"94e9b736-6944-4781-9b40-500ec82d10e2\") " pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:14.000121 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:13.999916    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f144fc9d-a69b-4cb5-b99c-2092a7905b32-konnectivity-ca\") pod \"konnectivity-agent-lvgwz\" (UID: \"f144fc9d-a69b-4cb5-b99c-2092a7905b32\") " pod="kube-system/konnectivity-agent-lvgwz"
Jul 24 15:36:14.000484 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.000465    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f144fc9d-a69b-4cb5-b99c-2092a7905b32-agent-certs\") pod \"konnectivity-agent-lvgwz\" (UID: \"f144fc9d-a69b-4cb5-b99c-2092a7905b32\") " pod="kube-system/konnectivity-agent-lvgwz"
Jul 24 15:36:14.002195 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.002176    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c7ecbc05-a1ef-4546-8915-3ca20c060b37-etc-tuned\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:14.002305 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.002245    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7ecbc05-a1ef-4546-8915-3ca20c060b37-tmp\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:14.004479 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:14.004462    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:36:14.004479 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:14.004480    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:36:14.004603 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:14.004489    2561 projected.go:196] Error preparing data for projected volume kube-api-access-6jngr for pod openshift-network-diagnostics/network-check-target-qrsfq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:36:14.004603 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:14.004532    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr podName:ac6f7393-c091-44d0-9e64-4b58651a24b8 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:14.504520832 +0000 UTC m=+2.915231426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6jngr" (UniqueName: "kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr") pod "network-check-target-qrsfq" (UID: "ac6f7393-c091-44d0-9e64-4b58651a24b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:36:14.006780 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.006756    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7b5v\" (UniqueName: \"kubernetes.io/projected/aa3224e2-ecb7-4164-aa12-48213a6a5210-kube-api-access-v7b5v\") pod \"node-resolver-s6wn2\" (UID: \"aa3224e2-ecb7-4164-aa12-48213a6a5210\") " pod="openshift-dns/node-resolver-s6wn2"
Jul 24 15:36:14.008127 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.008102    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrbk9\" (UniqueName: \"kubernetes.io/projected/076c0fb1-5f86-4a0b-8725-100ea9966c41-kube-api-access-qrbk9\") pod \"node-ca-lsj8p\" (UID: \"076c0fb1-5f86-4a0b-8725-100ea9966c41\") " pod="openshift-image-registry/node-ca-lsj8p"
Jul 24 15:36:14.008223 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.008127    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b86vx\" (UniqueName: \"kubernetes.io/projected/fc5b0603-f613-4f94-9c67-e5602c0ca649-kube-api-access-b86vx\") pod \"iptables-alerter-lmdw4\" (UID: \"fc5b0603-f613-4f94-9c67-e5602c0ca649\") " pod="openshift-network-operator/iptables-alerter-lmdw4"
Jul 24 15:36:14.008423 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.008405    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v55jn\" (UniqueName: \"kubernetes.io/projected/c7ecbc05-a1ef-4546-8915-3ca20c060b37-kube-api-access-v55jn\") pod \"tuned-fqlp8\" (UID: \"c7ecbc05-a1ef-4546-8915-3ca20c060b37\") " pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:14.099963 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.099922    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-kubelet\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.099963 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.099966    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:14.100199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.099985    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-var-lib-cni-bin\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.100199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.099999    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-systemd-units\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.100199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100013    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-run-openvswitch\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.100199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100008    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-kubelet\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.100199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100039    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-sys-fs\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:14.100199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100065    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-var-lib-cni-multus\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.100199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100077    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-systemd-units\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.100199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100078    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-var-lib-cni-bin\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.100199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100090    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-multus-cni-dir\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.100199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100136    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-slash\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.100199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100154    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-multus-cni-dir\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.100199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100136    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-sys-fs\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:14.100199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100156    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-run-ovn-kubernetes\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.100199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100188    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:14.100199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100192    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-run-openvswitch\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.100199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100201    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-var-lib-cni-multus\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.100199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100210    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-run-ovn-kubernetes\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.100808 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100249    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-run-netns\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.100808 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100281    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-var-lib-openvswitch\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.100808 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100308    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-node-log\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.100808 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100338    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:14.100808 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100342    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-multus-socket-dir-parent\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.100808 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100368    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-run-netns\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.100808 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100365    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-etc-kubernetes\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.100808 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100395    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-slash\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.100808 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100398    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-etc-kubernetes\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.100808 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100456    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-multus-socket-dir-parent\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.100808 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100456    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-var-lib-openvswitch\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.100808 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100493    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-node-log\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.100808 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100497    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gv59\" (UniqueName: \"kubernetes.io/projected/933739ec-2f20-4ea6-80ea-158f9cd48902-kube-api-access-2gv59\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.100808 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100529    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/933739ec-2f20-4ea6-80ea-158f9cd48902-env-overrides\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.100808 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100556    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-os-release\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:14.100808 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100579    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-run-k8s-cni-cncf-io\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.100808 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100603    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-run-multus-certs\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.101599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100649    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-etc-openvswitch\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.101599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100682    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:14.101599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100711    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-etc-openvswitch\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.101599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100720    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-run-k8s-cni-cncf-io\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.101599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100734    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-run-multus-certs\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.101599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100740    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-os-release\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:14.101599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100762    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-socket-dir\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:14.101599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100840    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-socket-dir\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:14.101599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100848    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:14.101599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100881    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-cni-bin\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.101599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100905    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/933739ec-2f20-4ea6-80ea-158f9cd48902-ovnkube-script-lib\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.101599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100931    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-multus-daemon-config\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.101599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100960    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxjp8\" (UniqueName: \"kubernetes.io/projected/f1958e1f-5868-4a15-847c-28d61994d060-kube-api-access-qxjp8\") pod \"network-metrics-daemon-vmnkg\" (UID: \"f1958e1f-5868-4a15-847c-28d61994d060\") " pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:14.101599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100986    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-log-socket\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.101599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.100990    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/933739ec-2f20-4ea6-80ea-158f9cd48902-env-overrides\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.101599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101010    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:14.101599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101034    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-cnibin\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:14.102470 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101062    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97c7v\" (UniqueName: \"kubernetes.io/projected/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-kube-api-access-97c7v\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:14.102470 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101082    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-cni-bin\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.102470 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101086    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-os-release\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.102470 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101125    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-run-systemd\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.102470 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101143    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-os-release\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.102470 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101151    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-etc-selinux\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:14.102470 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101177    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-system-cni-dir\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:14.102470 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101194    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-cnibin\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:14.102470 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101201    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs\") pod \"network-metrics-daemon-vmnkg\" (UID: \"f1958e1f-5868-4a15-847c-28d61994d060\") " pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:14.102470 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101223    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-cni-binary-copy\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.102470 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101248    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/933739ec-2f20-4ea6-80ea-158f9cd48902-ovnkube-config\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.102470 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101274    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-registration-dir\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:14.102470 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101298    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-system-cni-dir\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.102470 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101323    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-run-netns\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.102470 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101337    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:14.102470 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101348    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/933739ec-2f20-4ea6-80ea-158f9cd48902-ovn-node-metrics-cert\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.102470 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101362    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:14.103200 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101376    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-cni-netd\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.103200 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101414    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.103200 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101444    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnk5s\" (UniqueName: \"kubernetes.io/projected/47cf7e7d-f0af-4ace-b76d-240b87f40041-kube-api-access-rnk5s\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:14.103200 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101475    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-device-dir\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:14.103200 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101500    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-cni-binary-copy\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:14.103200 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:14.101510    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:36:14.103200 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101524    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-cnibin\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.103200 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101548    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-var-lib-kubelet\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.103200 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:14.101567    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs podName:f1958e1f-5868-4a15-847c-28d61994d060 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:14.601550511 +0000 UTC m=+3.012261114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs") pod "network-metrics-daemon-vmnkg" (UID: "f1958e1f-5868-4a15-847c-28d61994d060") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:36:14.103200 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101593    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-var-lib-kubelet\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.103200 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101598    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-hostroot\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.103200 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101628    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6hkz\" (UniqueName: \"kubernetes.io/projected/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-kube-api-access-x6hkz\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.103200 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101633    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-hostroot\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.103200 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101654    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-run-ovn\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.103200 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101700    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-multus-conf-dir\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.103200 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101742    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-multus-daemon-config\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.103200 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101781    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-multus-conf-dir\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.104061 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101833    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-device-dir\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:14.104061 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101864    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/933739ec-2f20-4ea6-80ea-158f9cd48902-ovnkube-script-lib\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.104061 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101416    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-cni-netd\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.104061 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101913    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-log-socket\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.104061 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101942    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-run-systemd\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.104061 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101947    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.104061 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.102013    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-etc-selinux\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:14.104061 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.101445    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-system-cni-dir\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:14.104061 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.102054    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-system-cni-dir\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.104061 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.102093    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/47cf7e7d-f0af-4ace-b76d-240b87f40041-registration-dir\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:14.104061 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.102120    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/933739ec-2f20-4ea6-80ea-158f9cd48902-run-ovn\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.104061 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.102156    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-cnibin\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.104061 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.102278    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-cni-binary-copy\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:14.104061 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.102349    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-host-run-netns\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.104061 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.102596    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-cni-binary-copy\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.104061 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.102718    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/933739ec-2f20-4ea6-80ea-158f9cd48902-ovnkube-config\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.104843 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.104796    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal" event={"ID":"c43182eb3bfc6cb96f40919d9e144573","Type":"ContainerStarted","Data":"7ce16baba7e820bba1f5bafb387a23c9fdc7a83aaf4df960e666aa877cd2de4d"}
Jul 24 15:36:14.105244 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.105225    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/933739ec-2f20-4ea6-80ea-158f9cd48902-ovn-node-metrics-cert\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.106041 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.106020    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-164.ec2.internal" event={"ID":"b4237762195c7e99a7a069493539ac56","Type":"ContainerStarted","Data":"3867f6ae1741bab9123b2d7bbd7cf1bb4b74c05d3a3acead0f1df1f6519b3a22"}
Jul 24 15:36:14.109056 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.109001    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxjp8\" (UniqueName: \"kubernetes.io/projected/f1958e1f-5868-4a15-847c-28d61994d060-kube-api-access-qxjp8\") pod \"network-metrics-daemon-vmnkg\" (UID: \"f1958e1f-5868-4a15-847c-28d61994d060\") " pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:14.109452 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.109427    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gv59\" (UniqueName: \"kubernetes.io/projected/933739ec-2f20-4ea6-80ea-158f9cd48902-kube-api-access-2gv59\") pod \"ovnkube-node-th7v2\" (UID: \"933739ec-2f20-4ea6-80ea-158f9cd48902\") " pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.110311 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.110283    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnk5s\" (UniqueName: \"kubernetes.io/projected/47cf7e7d-f0af-4ace-b76d-240b87f40041-kube-api-access-rnk5s\") pod \"aws-ebs-csi-driver-node-lvgzb\" (UID: \"47cf7e7d-f0af-4ace-b76d-240b87f40041\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:14.110546 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.110528    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97c7v\" (UniqueName: \"kubernetes.io/projected/c9cb3936-a642-4db6-a658-e4c3f3b9c80a-kube-api-access-97c7v\") pod \"multus-additional-cni-plugins-l7xpv\" (UID: \"c9cb3936-a642-4db6-a658-e4c3f3b9c80a\") " pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:14.110622 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.110574    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6hkz\" (UniqueName: \"kubernetes.io/projected/882b2dc3-a4dd-46d0-89bb-ee03f7de5559-kube-api-access-x6hkz\") pod \"multus-5g4s6\" (UID: \"882b2dc3-a4dd-46d0-89bb-ee03f7de5559\") " pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.152521 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.152493    2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:36:14.183711 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.183672    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lsj8p"
Jul 24 15:36:14.198147 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.198122    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lmdw4"
Jul 24 15:36:14.210023 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.210002    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lvgwz"
Jul 24 15:36:14.216815 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.216790    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fqlp8"
Jul 24 15:36:14.226600 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.226582    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-s6wn2"
Jul 24 15:36:14.233209 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.233192    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5g4s6"
Jul 24 15:36:14.241812 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.241795    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-l7xpv"
Jul 24 15:36:14.249422 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.249402    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:14.262019 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.261997    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb"
Jul 24 15:36:14.506092 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.506002    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jngr\" (UniqueName: \"kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr\") pod \"network-check-target-qrsfq\" (UID: \"ac6f7393-c091-44d0-9e64-4b58651a24b8\") " pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:14.506092 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.506081    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret\") pod \"global-pull-secret-syncer-j8fqw\" (UID: \"94e9b736-6944-4781-9b40-500ec82d10e2\") " pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:14.506298 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:14.506203    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:36:14.506298 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:14.506213    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:36:14.506298 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:14.506234    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:36:14.506298 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:14.506245    2561 projected.go:196] Error preparing data for projected volume kube-api-access-6jngr for pod openshift-network-diagnostics/network-check-target-qrsfq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:36:14.506298 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:14.506265    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret podName:94e9b736-6944-4781-9b40-500ec82d10e2 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:15.506251271 +0000 UTC m=+3.916961867 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret") pod "global-pull-secret-syncer-j8fqw" (UID: "94e9b736-6944-4781-9b40-500ec82d10e2") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:36:14.506298 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:14.506295    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr podName:ac6f7393-c091-44d0-9e64-4b58651a24b8 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:15.506276815 +0000 UTC m=+3.916987423 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6jngr" (UniqueName: "kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr") pod "network-check-target-qrsfq" (UID: "ac6f7393-c091-44d0-9e64-4b58651a24b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:36:14.606603 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.606566    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs\") pod \"network-metrics-daemon-vmnkg\" (UID: \"f1958e1f-5868-4a15-847c-28d61994d060\") " pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:14.606780 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:14.606720    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:36:14.606839 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:14.606788    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs podName:f1958e1f-5868-4a15-847c-28d61994d060 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:15.606768841 +0000 UTC m=+4.017479443 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs") pod "network-metrics-daemon-vmnkg" (UID: "f1958e1f-5868-4a15-847c-28d61994d060") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:36:14.927564 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.927479    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 15:31:13 +0000 UTC" deadline="2028-02-20 03:36:12.770581304 +0000 UTC"
Jul 24 15:36:14.927564 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:14.927511    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13811h59m57.843073807s"
Jul 24 15:36:15.017373 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:15.017345    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod076c0fb1_5f86_4a0b_8725_100ea9966c41.slice/crio-e6ef0db3f394595e3da7923935acf004ffe47094a427148eb8893c765da10d2b WatchSource:0}: Error finding container e6ef0db3f394595e3da7923935acf004ffe47094a427148eb8893c765da10d2b: Status 404 returned error can't find the container with id e6ef0db3f394595e3da7923935acf004ffe47094a427148eb8893c765da10d2b
Jul 24 15:36:15.018727 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:15.018704    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod882b2dc3_a4dd_46d0_89bb_ee03f7de5559.slice/crio-10646165c392f289cf2c8cd37b31704cc66516d7587a8c183a8e89a04c65bb3f WatchSource:0}: Error finding container 10646165c392f289cf2c8cd37b31704cc66516d7587a8c183a8e89a04c65bb3f: Status 404 returned error can't find the container with id 10646165c392f289cf2c8cd37b31704cc66516d7587a8c183a8e89a04c65bb3f
Jul 24 15:36:15.020441 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:15.020423    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod933739ec_2f20_4ea6_80ea_158f9cd48902.slice/crio-b49576b413132eefc67fc62b6a4c4d6fc1435a3aff8391613b1a23a7a2ec73a9 WatchSource:0}: Error finding container b49576b413132eefc67fc62b6a4c4d6fc1435a3aff8391613b1a23a7a2ec73a9: Status 404 returned error can't find the container with id b49576b413132eefc67fc62b6a4c4d6fc1435a3aff8391613b1a23a7a2ec73a9
Jul 24 15:36:15.022636 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:15.022517    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7ecbc05_a1ef_4546_8915_3ca20c060b37.slice/crio-629576765fc78ca46cc758ec6bac84dda29301a9870780a1ca1d03a03abd9f54 WatchSource:0}: Error finding container 629576765fc78ca46cc758ec6bac84dda29301a9870780a1ca1d03a03abd9f54: Status 404 returned error can't find the container with id 629576765fc78ca46cc758ec6bac84dda29301a9870780a1ca1d03a03abd9f54
Jul 24 15:36:15.024253 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:15.024231    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa3224e2_ecb7_4164_aa12_48213a6a5210.slice/crio-bdaf9c76578c3743cafaec5518b92273987a986a38420c6fc5d9e11bc15d5db4 WatchSource:0}: Error finding container bdaf9c76578c3743cafaec5518b92273987a986a38420c6fc5d9e11bc15d5db4: Status 404 returned error can't find the container with id bdaf9c76578c3743cafaec5518b92273987a986a38420c6fc5d9e11bc15d5db4
Jul 24 15:36:15.024864 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:15.024847    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf144fc9d_a69b_4cb5_b99c_2092a7905b32.slice/crio-82c92459c28d35c0c94209f1e591fdf74359ffe32fcb8a9db5f7ed1e1b425fce WatchSource:0}: Error finding container 82c92459c28d35c0c94209f1e591fdf74359ffe32fcb8a9db5f7ed1e1b425fce: Status 404 returned error can't find the container with id 82c92459c28d35c0c94209f1e591fdf74359ffe32fcb8a9db5f7ed1e1b425fce
Jul 24 15:36:15.046505 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:15.046479    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc5b0603_f613_4f94_9c67_e5602c0ca649.slice/crio-0ffa8a05de5a3154793efa26cdee9124d1363de4d81d1b9161e2c75c53fad9ac WatchSource:0}: Error finding container 0ffa8a05de5a3154793efa26cdee9124d1363de4d81d1b9161e2c75c53fad9ac: Status 404 returned error can't find the container with id 0ffa8a05de5a3154793efa26cdee9124d1363de4d81d1b9161e2c75c53fad9ac
Jul 24 15:36:15.047261 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:15.047209    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9cb3936_a642_4db6_a658_e4c3f3b9c80a.slice/crio-c413c74166d89985f91237c6c386c17760dd77a60f0ad926467a8e0e31d028ed WatchSource:0}: Error finding container c413c74166d89985f91237c6c386c17760dd77a60f0ad926467a8e0e31d028ed: Status 404 returned error can't find the container with id c413c74166d89985f91237c6c386c17760dd77a60f0ad926467a8e0e31d028ed
Jul 24 15:36:15.048514 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:15.048485    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47cf7e7d_f0af_4ace_b76d_240b87f40041.slice/crio-a1a77f2270c5ed282127797bcec90d9bc5769912f722eda196abe722b8df4521 WatchSource:0}: Error finding container a1a77f2270c5ed282127797bcec90d9bc5769912f722eda196abe722b8df4521: Status 404 returned error can't find the container with id a1a77f2270c5ed282127797bcec90d9bc5769912f722eda196abe722b8df4521
Jul 24 15:36:15.100631 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:15.100606    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:15.100775 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:15.100753    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:15.108792 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:15.108763    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb" event={"ID":"47cf7e7d-f0af-4ace-b76d-240b87f40041","Type":"ContainerStarted","Data":"a1a77f2270c5ed282127797bcec90d9bc5769912f722eda196abe722b8df4521"}
Jul 24 15:36:15.109787 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:15.109765    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lmdw4" event={"ID":"fc5b0603-f613-4f94-9c67-e5602c0ca649","Type":"ContainerStarted","Data":"0ffa8a05de5a3154793efa26cdee9124d1363de4d81d1b9161e2c75c53fad9ac"}
Jul 24 15:36:15.111026 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:15.111006    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lvgwz" event={"ID":"f144fc9d-a69b-4cb5-b99c-2092a7905b32","Type":"ContainerStarted","Data":"82c92459c28d35c0c94209f1e591fdf74359ffe32fcb8a9db5f7ed1e1b425fce"}
Jul 24 15:36:15.112056 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:15.112031    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s6wn2" event={"ID":"aa3224e2-ecb7-4164-aa12-48213a6a5210","Type":"ContainerStarted","Data":"bdaf9c76578c3743cafaec5518b92273987a986a38420c6fc5d9e11bc15d5db4"}
Jul 24 15:36:15.113092 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:15.113073    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2" event={"ID":"933739ec-2f20-4ea6-80ea-158f9cd48902","Type":"ContainerStarted","Data":"b49576b413132eefc67fc62b6a4c4d6fc1435a3aff8391613b1a23a7a2ec73a9"}
Jul 24 15:36:15.114675 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:15.114637    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5g4s6" event={"ID":"882b2dc3-a4dd-46d0-89bb-ee03f7de5559","Type":"ContainerStarted","Data":"10646165c392f289cf2c8cd37b31704cc66516d7587a8c183a8e89a04c65bb3f"}
Jul 24 15:36:15.116056 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:15.115562    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lsj8p" event={"ID":"076c0fb1-5f86-4a0b-8725-100ea9966c41","Type":"ContainerStarted","Data":"e6ef0db3f394595e3da7923935acf004ffe47094a427148eb8893c765da10d2b"}
Jul 24 15:36:15.116962 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:15.116941    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l7xpv" event={"ID":"c9cb3936-a642-4db6-a658-e4c3f3b9c80a","Type":"ContainerStarted","Data":"c413c74166d89985f91237c6c386c17760dd77a60f0ad926467a8e0e31d028ed"}
Jul 24 15:36:15.118705 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:15.118682    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fqlp8" event={"ID":"c7ecbc05-a1ef-4546-8915-3ca20c060b37","Type":"ContainerStarted","Data":"629576765fc78ca46cc758ec6bac84dda29301a9870780a1ca1d03a03abd9f54"}
Jul 24 15:36:15.514965 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:15.514881    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret\") pod \"global-pull-secret-syncer-j8fqw\" (UID: \"94e9b736-6944-4781-9b40-500ec82d10e2\") " pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:15.515127 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:15.514969    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jngr\" (UniqueName: \"kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr\") pod \"network-check-target-qrsfq\" (UID: \"ac6f7393-c091-44d0-9e64-4b58651a24b8\") " pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:15.515187 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:15.515169    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:36:15.515247 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:15.515189    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:36:15.515247 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:15.515203    2561 projected.go:196] Error preparing data for projected volume kube-api-access-6jngr for pod openshift-network-diagnostics/network-check-target-qrsfq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:36:15.515395 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:15.515277    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr podName:ac6f7393-c091-44d0-9e64-4b58651a24b8 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:17.515255402 +0000 UTC m=+5.925966007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6jngr" (UniqueName: "kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr") pod "network-check-target-qrsfq" (UID: "ac6f7393-c091-44d0-9e64-4b58651a24b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:36:15.515395 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:15.515377    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:36:15.515493 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:15.515418    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret podName:94e9b736-6944-4781-9b40-500ec82d10e2 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:17.515407419 +0000 UTC m=+5.926118021 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret") pod "global-pull-secret-syncer-j8fqw" (UID: "94e9b736-6944-4781-9b40-500ec82d10e2") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:36:15.563515 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:15.563316    2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:36:15.616579 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:15.615973    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs\") pod \"network-metrics-daemon-vmnkg\" (UID: \"f1958e1f-5868-4a15-847c-28d61994d060\") " pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:15.616579 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:15.616146    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:36:15.616579 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:15.616206    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs podName:f1958e1f-5868-4a15-847c-28d61994d060 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:17.616188303 +0000 UTC m=+6.026898913 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs") pod "network-metrics-daemon-vmnkg" (UID: "f1958e1f-5868-4a15-847c-28d61994d060") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:36:15.928125 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:15.927988    2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 15:31:13 +0000 UTC" deadline="2028-05-04 10:26:15.364169102 +0000 UTC"
Jul 24 15:36:15.928125 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:15.928048    2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15594h49m59.436124345s"
Jul 24 15:36:16.102812 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:16.102780    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:16.103006 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:16.102919    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmnkg" podUID="f1958e1f-5868-4a15-847c-28d61994d060"
Jul 24 15:36:16.103342 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:16.103294    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:16.103438 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:16.103382    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8fqw" podUID="94e9b736-6944-4781-9b40-500ec82d10e2"
Jul 24 15:36:16.143418 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:16.143229    2561 generic.go:356] "Generic (PLEG): container finished" podID="c43182eb3bfc6cb96f40919d9e144573" containerID="bba88f59086b159ce388381786dd9cb5a8f4f0be34fd88161a875efbfa526b21" exitCode=0
Jul 24 15:36:16.143418 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:16.143340    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal" event={"ID":"c43182eb3bfc6cb96f40919d9e144573","Type":"ContainerDied","Data":"bba88f59086b159ce388381786dd9cb5a8f4f0be34fd88161a875efbfa526b21"}
Jul 24 15:36:16.151459 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:16.151403    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-164.ec2.internal" event={"ID":"b4237762195c7e99a7a069493539ac56","Type":"ContainerStarted","Data":"c01cd0a0efe9d60f76b8659872a65fa8e7b07084a8ea99fd241168b5682b464e"}
Jul 24 15:36:17.101434 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:17.100945    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:17.101434 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:17.101081    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:17.170064 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:17.169955    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal" event={"ID":"c43182eb3bfc6cb96f40919d9e144573","Type":"ContainerStarted","Data":"0621205969b4dc450a909169d41fd27c31df15dac0f20eb1a613fa89c56929ec"}
Jul 24 15:36:17.185312 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:17.184683    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-164.ec2.internal" podStartSLOduration=5.184645209 podStartE2EDuration="5.184645209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:16.173807786 +0000 UTC m=+4.584518404" watchObservedRunningTime="2026-07-24 15:36:17.184645209 +0000 UTC m=+5.595355827"
Jul 24 15:36:17.185312 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:17.184905    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-164.ec2.internal" podStartSLOduration=5.184898147 podStartE2EDuration="5.184898147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:17.183648852 +0000 UTC m=+5.594359539" watchObservedRunningTime="2026-07-24 15:36:17.184898147 +0000 UTC m=+5.595608766"
Jul 24 15:36:17.531584 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:17.531493    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jngr\" (UniqueName: \"kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr\") pod \"network-check-target-qrsfq\" (UID: \"ac6f7393-c091-44d0-9e64-4b58651a24b8\") " pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:17.531584 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:17.531581    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret\") pod \"global-pull-secret-syncer-j8fqw\" (UID: \"94e9b736-6944-4781-9b40-500ec82d10e2\") " pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:17.531830 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:17.531737    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:36:17.531830 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:17.531805    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret podName:94e9b736-6944-4781-9b40-500ec82d10e2 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:21.531785241 +0000 UTC m=+9.942495859 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret") pod "global-pull-secret-syncer-j8fqw" (UID: "94e9b736-6944-4781-9b40-500ec82d10e2") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:36:17.532241 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:17.532223    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:36:17.532327 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:17.532246    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:36:17.532327 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:17.532260    2561 projected.go:196] Error preparing data for projected volume kube-api-access-6jngr for pod openshift-network-diagnostics/network-check-target-qrsfq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:36:17.532327 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:17.532306    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr podName:ac6f7393-c091-44d0-9e64-4b58651a24b8 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:21.532291533 +0000 UTC m=+9.943002139 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6jngr" (UniqueName: "kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr") pod "network-check-target-qrsfq" (UID: "ac6f7393-c091-44d0-9e64-4b58651a24b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:36:17.632284 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:17.632243    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs\") pod \"network-metrics-daemon-vmnkg\" (UID: \"f1958e1f-5868-4a15-847c-28d61994d060\") " pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:17.632490 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:17.632457    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:36:17.632590 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:17.632528    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs podName:f1958e1f-5868-4a15-847c-28d61994d060 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:21.632508205 +0000 UTC m=+10.043218814 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs") pod "network-metrics-daemon-vmnkg" (UID: "f1958e1f-5868-4a15-847c-28d61994d060") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:36:18.103698 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:18.103179    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:18.103698 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:18.103317    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmnkg" podUID="f1958e1f-5868-4a15-847c-28d61994d060"
Jul 24 15:36:18.104515 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:18.104312    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:18.104515 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:18.104423    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8fqw" podUID="94e9b736-6944-4781-9b40-500ec82d10e2"
Jul 24 15:36:19.101201 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:19.100648    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:19.101201 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:19.100800    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:20.101680 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:20.100804    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:20.101680 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:20.100962    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmnkg" podUID="f1958e1f-5868-4a15-847c-28d61994d060"
Jul 24 15:36:20.101680 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:20.101461    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:20.101680 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:20.101570    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8fqw" podUID="94e9b736-6944-4781-9b40-500ec82d10e2"
Jul 24 15:36:21.100478 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:21.100449    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:21.100692 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:21.100584    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:21.566890 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:21.566164    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret\") pod \"global-pull-secret-syncer-j8fqw\" (UID: \"94e9b736-6944-4781-9b40-500ec82d10e2\") " pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:21.566890 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:21.566206    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jngr\" (UniqueName: \"kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr\") pod \"network-check-target-qrsfq\" (UID: \"ac6f7393-c091-44d0-9e64-4b58651a24b8\") " pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:21.566890 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:21.566329    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:36:21.566890 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:21.566349    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:36:21.566890 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:21.566361    2561 projected.go:196] Error preparing data for projected volume kube-api-access-6jngr for pod openshift-network-diagnostics/network-check-target-qrsfq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:36:21.566890 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:21.566412    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr podName:ac6f7393-c091-44d0-9e64-4b58651a24b8 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:29.566397881 +0000 UTC m=+17.977108479 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6jngr" (UniqueName: "kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr") pod "network-check-target-qrsfq" (UID: "ac6f7393-c091-44d0-9e64-4b58651a24b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:36:21.566890 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:21.566808    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:36:21.566890 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:21.566859    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret podName:94e9b736-6944-4781-9b40-500ec82d10e2 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:29.566844247 +0000 UTC m=+17.977554846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret") pod "global-pull-secret-syncer-j8fqw" (UID: "94e9b736-6944-4781-9b40-500ec82d10e2") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:36:21.666907 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:21.666867    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs\") pod \"network-metrics-daemon-vmnkg\" (UID: \"f1958e1f-5868-4a15-847c-28d61994d060\") " pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:21.667079 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:21.667061    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:36:21.667138 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:21.667128    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs podName:f1958e1f-5868-4a15-847c-28d61994d060 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:29.667109907 +0000 UTC m=+18.077820517 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs") pod "network-metrics-daemon-vmnkg" (UID: "f1958e1f-5868-4a15-847c-28d61994d060") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:36:22.101965 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:22.101928    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:22.102129 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:22.102067    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmnkg" podUID="f1958e1f-5868-4a15-847c-28d61994d060"
Jul 24 15:36:22.102204 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:22.102119    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:22.102288 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:22.102258    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8fqw" podUID="94e9b736-6944-4781-9b40-500ec82d10e2"
Jul 24 15:36:23.101504 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:23.101029    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:23.101504 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:23.101145    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:24.102570 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:24.102537    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:24.103010 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:24.102642    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmnkg" podUID="f1958e1f-5868-4a15-847c-28d61994d060"
Jul 24 15:36:24.103010 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:24.102967    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:24.103109 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:24.103059    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8fqw" podUID="94e9b736-6944-4781-9b40-500ec82d10e2"
Jul 24 15:36:25.100696 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:25.100651    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:25.100860 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:25.100772    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:26.100473 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:26.100383    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:26.100473 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:26.100418    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:26.100986 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:26.100535    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmnkg" podUID="f1958e1f-5868-4a15-847c-28d61994d060"
Jul 24 15:36:26.100986 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:26.100674    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8fqw" podUID="94e9b736-6944-4781-9b40-500ec82d10e2"
Jul 24 15:36:27.100737 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:27.100694    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:27.101171 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:27.100830    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:28.100849 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:28.100813    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:28.101294 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:28.100957    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmnkg" podUID="f1958e1f-5868-4a15-847c-28d61994d060"
Jul 24 15:36:28.101294 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:28.100971    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:28.101294 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:28.101054    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8fqw" podUID="94e9b736-6944-4781-9b40-500ec82d10e2"
Jul 24 15:36:29.100754 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:29.100721    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:29.100966 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:29.100841    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:29.622438 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:29.622406    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret\") pod \"global-pull-secret-syncer-j8fqw\" (UID: \"94e9b736-6944-4781-9b40-500ec82d10e2\") " pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:29.622604 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:29.622451    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jngr\" (UniqueName: \"kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr\") pod \"network-check-target-qrsfq\" (UID: \"ac6f7393-c091-44d0-9e64-4b58651a24b8\") " pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:29.622604 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:29.622563    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:36:29.622681 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:29.622624    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret podName:94e9b736-6944-4781-9b40-500ec82d10e2 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:45.62260989 +0000 UTC m=+34.033320485 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret") pod "global-pull-secret-syncer-j8fqw" (UID: "94e9b736-6944-4781-9b40-500ec82d10e2") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:36:29.622681 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:29.622566    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:36:29.622793 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:29.622687    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:36:29.622793 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:29.622698    2561 projected.go:196] Error preparing data for projected volume kube-api-access-6jngr for pod openshift-network-diagnostics/network-check-target-qrsfq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:36:29.622793 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:29.622743    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr podName:ac6f7393-c091-44d0-9e64-4b58651a24b8 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:45.6227298 +0000 UTC m=+34.033440402 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6jngr" (UniqueName: "kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr") pod "network-check-target-qrsfq" (UID: "ac6f7393-c091-44d0-9e64-4b58651a24b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:36:29.723462 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:29.723430    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs\") pod \"network-metrics-daemon-vmnkg\" (UID: \"f1958e1f-5868-4a15-847c-28d61994d060\") " pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:29.723620 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:29.723553    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:36:29.723620 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:29.723606    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs podName:f1958e1f-5868-4a15-847c-28d61994d060 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:45.723593057 +0000 UTC m=+34.134303652 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs") pod "network-metrics-daemon-vmnkg" (UID: "f1958e1f-5868-4a15-847c-28d61994d060") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:36:30.101068 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:30.101034    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:30.101068 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:30.101067    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:30.101609 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:30.101167    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8fqw" podUID="94e9b736-6944-4781-9b40-500ec82d10e2"
Jul 24 15:36:30.101609 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:30.101325    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmnkg" podUID="f1958e1f-5868-4a15-847c-28d61994d060"
Jul 24 15:36:31.101086 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:31.101048    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:31.101472 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:31.101169    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:32.101563 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:32.101529    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:32.102042 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:32.101638    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmnkg" podUID="f1958e1f-5868-4a15-847c-28d61994d060"
Jul 24 15:36:32.102042 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:32.101691    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:32.102042 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:32.101784    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8fqw" podUID="94e9b736-6944-4781-9b40-500ec82d10e2"
Jul 24 15:36:33.100086 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:33.100050    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:33.100240 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:33.100174    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:34.101123 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:34.100870    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:34.101586 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:34.101260    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmnkg" podUID="f1958e1f-5868-4a15-847c-28d61994d060"
Jul 24 15:36:34.101586 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:34.100899    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:34.101586 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:34.101381    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8fqw" podUID="94e9b736-6944-4781-9b40-500ec82d10e2"
Jul 24 15:36:34.204342 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:34.204053    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fqlp8" event={"ID":"c7ecbc05-a1ef-4546-8915-3ca20c060b37","Type":"ContainerStarted","Data":"dd6a586f5ca2c16d58795687508ccc2cb09c306acdc4b8f9bc0ca8156a5f2133"}
Jul 24 15:36:34.205553 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:34.205524    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb" event={"ID":"47cf7e7d-f0af-4ace-b76d-240b87f40041","Type":"ContainerStarted","Data":"3fc978a88b86fd94e88565a42fc35e74aab662ddd887c0f647ab003c957e82b0"}
Jul 24 15:36:34.207031 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:34.207010    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lvgwz" event={"ID":"f144fc9d-a69b-4cb5-b99c-2092a7905b32","Type":"ContainerStarted","Data":"eef506c5678e8269f9da1cb4493858b1dc165546efd3fab655efc2be14cf7c02"}
Jul 24 15:36:34.208826 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:34.208796    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-s6wn2" event={"ID":"aa3224e2-ecb7-4164-aa12-48213a6a5210","Type":"ContainerStarted","Data":"79464fd8532945e3875a80f7541ef89668e51b9270f0c4bcd8d66bb828e45f58"}
Jul 24 15:36:34.210746 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:34.210723    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2" event={"ID":"933739ec-2f20-4ea6-80ea-158f9cd48902","Type":"ContainerStarted","Data":"800147990da98dbe4f4404e06fd217b21590d5e98794e1c4a76d92b0c5564688"}
Jul 24 15:36:34.210898 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:34.210786    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2" event={"ID":"933739ec-2f20-4ea6-80ea-158f9cd48902","Type":"ContainerStarted","Data":"08fe7f0bd46c1467078abb4cc0372171dfa0299173a2a8b88a4f3e6ab49c4544"}
Jul 24 15:36:34.212427 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:34.212403    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5g4s6" event={"ID":"882b2dc3-a4dd-46d0-89bb-ee03f7de5559","Type":"ContainerStarted","Data":"ff64317c503e6676d3c558c2c587e0d1239d1f4e3b3d73453fac545240a83537"}
Jul 24 15:36:34.213881 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:34.213863    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lsj8p" event={"ID":"076c0fb1-5f86-4a0b-8725-100ea9966c41","Type":"ContainerStarted","Data":"272bac3464be7af3c9839181c4f1e0e6e7762508faed980d07ddeeb017973bcd"}
Jul 24 15:36:34.215216 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:34.215192    2561 generic.go:356] "Generic (PLEG): container finished" podID="c9cb3936-a642-4db6-a658-e4c3f3b9c80a" containerID="0efced8a9c4fe28d476bfae72de6548fe4b32b4f617e5ff2220f6e9e38c31eee" exitCode=0
Jul 24 15:36:34.215319 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:34.215232    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l7xpv" event={"ID":"c9cb3936-a642-4db6-a658-e4c3f3b9c80a","Type":"ContainerDied","Data":"0efced8a9c4fe28d476bfae72de6548fe4b32b4f617e5ff2220f6e9e38c31eee"}
Jul 24 15:36:34.228129 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:34.228090    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fqlp8" podStartSLOduration=3.400679463 podStartE2EDuration="22.228064467s" totalImagesPullingTime="18.827385004s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:34.227409475 +0000 UTC m=+22.638120092" watchObservedRunningTime="2026-07-24 15:36:34.228064467 +0000 UTC m=+22.638775117"
Jul 24 15:36:34.289999 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:34.287042    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5g4s6" podStartSLOduration=3.417279818 podStartE2EDuration="22.286872007s" totalImagesPullingTime="18.869592189s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:34.261849629 +0000 UTC m=+22.672560246" watchObservedRunningTime="2026-07-24 15:36:34.286872007 +0000 UTC m=+22.697582624"
Jul 24 15:36:34.326499 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:34.326458    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lsj8p" podStartSLOduration=3.494252849 podStartE2EDuration="22.326442373s" totalImagesPullingTime="18.832189524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:34.287714212 +0000 UTC m=+22.698424830" watchObservedRunningTime="2026-07-24 15:36:34.326442373 +0000 UTC m=+22.737152991"
Jul 24 15:36:34.373284 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:34.373240    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-s6wn2" podStartSLOduration=3.562211728 podStartE2EDuration="22.373224335s" totalImagesPullingTime="18.811012607s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:34.326780789 +0000 UTC m=+22.737491405" watchObservedRunningTime="2026-07-24 15:36:34.373224335 +0000 UTC m=+22.783934982"
Jul 24 15:36:34.414286 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:34.414246    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lvgwz" podStartSLOduration=3.604783635 podStartE2EDuration="22.41422911s" totalImagesPullingTime="18.809445475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:34.413563232 +0000 UTC m=+22.824273849" watchObservedRunningTime="2026-07-24 15:36:34.41422911 +0000 UTC m=+22.824939727"
Jul 24 15:36:35.100736 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:35.100703    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:35.101225 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:35.100841    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:35.218380 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:35.218342    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lmdw4" event={"ID":"fc5b0603-f613-4f94-9c67-e5602c0ca649","Type":"ContainerStarted","Data":"37782df6f7e018210bf43ebbe9a20fde922eb4a31766a6be116b479c40ae1ab5"}
Jul 24 15:36:35.221283 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:35.221256    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2" event={"ID":"933739ec-2f20-4ea6-80ea-158f9cd48902","Type":"ContainerStarted","Data":"b1f473050e5b16c5748d76d2313afd879a6e54d08a765a42cf9bbd27bb96e86b"}
Jul 24 15:36:35.221430 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:35.221320    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2" event={"ID":"933739ec-2f20-4ea6-80ea-158f9cd48902","Type":"ContainerStarted","Data":"7e154287ea4ce721e60508d2aa0fb75d0375a74e127c7b8c3d46874654be79ac"}
Jul 24 15:36:35.221430 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:35.221353    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2" event={"ID":"933739ec-2f20-4ea6-80ea-158f9cd48902","Type":"ContainerStarted","Data":"f16f8877eddd794a2533f159922a07e0a63ddb1f5c42d294ba18a43d2158a458"}
Jul 24 15:36:35.221430 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:35.221381    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2" event={"ID":"933739ec-2f20-4ea6-80ea-158f9cd48902","Type":"ContainerStarted","Data":"dfa3bdab3ccd44fe869364b5ca80ac84957bb87306efcfb036ced64940058da3"}
Jul 24 15:36:35.233845 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:35.233802    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lmdw4" podStartSLOduration=4.423425053 podStartE2EDuration="23.233788641s" totalImagesPullingTime="18.810363588s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:35.233480865 +0000 UTC m=+23.644191482" watchObservedRunningTime="2026-07-24 15:36:35.233788641 +0000 UTC m=+23.644499239"
Jul 24 15:36:35.476617 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:35.476585    2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 15:36:35.962562 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:35.962459    2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T15:36:35.4766151Z","UUID":"7fb6e412-c829-4c85-b578-33f138379064","Handler":null,"Name":"","Endpoint":""}
Jul 24 15:36:35.964045 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:35.964025    2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 15:36:35.964178 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:35.964060    2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 15:36:36.100391 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:36.100354    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:36.100550 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:36.100354    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:36.100550 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:36.100482    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8fqw" podUID="94e9b736-6944-4781-9b40-500ec82d10e2"
Jul 24 15:36:36.100550 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:36.100537    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmnkg" podUID="f1958e1f-5868-4a15-847c-28d61994d060"
Jul 24 15:36:36.225884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:36.225798    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb" event={"ID":"47cf7e7d-f0af-4ace-b76d-240b87f40041","Type":"ContainerStarted","Data":"e56834db031b6f7b664760abf242b3e998b3206dd3afa248abbc4f14efd8f24c"}
Jul 24 15:36:37.100908 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:37.100881    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:37.101088 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:37.100981    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:37.230830 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:37.230798    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb" event={"ID":"47cf7e7d-f0af-4ace-b76d-240b87f40041","Type":"ContainerStarted","Data":"74fbcfb82ca9d3b35c74e96ccfef7d92e925fde34bfadf251075bdefd7c938e1"}
Jul 24 15:36:37.234276 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:37.234249    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2" event={"ID":"933739ec-2f20-4ea6-80ea-158f9cd48902","Type":"ContainerStarted","Data":"0c264b05a4dd3597b3c1283c4f3efe8d2108d8c63b3458a8d1692311c1967dbb"}
Jul 24 15:36:37.247804 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:37.247763    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lvgzb" podStartSLOduration=4.095930372 podStartE2EDuration="25.24774678s" totalImagesPullingTime="21.151816408s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:37.24698812 +0000 UTC m=+25.657698736" watchObservedRunningTime="2026-07-24 15:36:37.24774678 +0000 UTC m=+25.658457398"
Jul 24 15:36:38.100257 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:38.100229    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:38.100451 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:38.100230    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:38.100451 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:38.100339    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8fqw" podUID="94e9b736-6944-4781-9b40-500ec82d10e2"
Jul 24 15:36:38.100557 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:38.100447    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmnkg" podUID="f1958e1f-5868-4a15-847c-28d61994d060"
Jul 24 15:36:38.275093 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:38.275057    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lvgwz"
Jul 24 15:36:38.275703 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:38.275674    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lvgwz"
Jul 24 15:36:38.342066 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:38.342034    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lvgwz_f144fc9d-a69b-4cb5-b99c-2092a7905b32/konnectivity-agent/0.log" limit=1024
Jul 24 15:36:38.473826 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:38.473778    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lvgwz_f144fc9d-a69b-4cb5-b99c-2092a7905b32/konnectivity-agent/0.log" limit=1024
Jul 24 15:36:39.100689 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:39.100432    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:39.100830 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:39.100773    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:39.239421 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:39.239392    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lvgwz"
Jul 24 15:36:39.239905 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:39.239886    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lvgwz"
Jul 24 15:36:39.308438 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:39.308411    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lvgwz_f144fc9d-a69b-4cb5-b99c-2092a7905b32/konnectivity-agent/0.log" limit=1024
Jul 24 15:36:40.100799 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:40.100721    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:40.100799 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:40.100762    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:40.100995 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:40.100846    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmnkg" podUID="f1958e1f-5868-4a15-847c-28d61994d060"
Jul 24 15:36:40.100995 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:40.100964    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8fqw" podUID="94e9b736-6944-4781-9b40-500ec82d10e2"
Jul 24 15:36:40.246978 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:40.246941    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2" event={"ID":"933739ec-2f20-4ea6-80ea-158f9cd48902","Type":"ContainerStarted","Data":"1e14cae37d823d862811c5ba651337790bb4e50ce8f26ef92251041dc605f5eb"}
Jul 24 15:36:40.247301 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:40.247278    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:40.247442 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:40.247350    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:40.248852 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:40.248816    2561 generic.go:356] "Generic (PLEG): container finished" podID="c9cb3936-a642-4db6-a658-e4c3f3b9c80a" containerID="85c38b33cc628959045d92f2ac61802bc2ecb7a998933fa7231acfde4e773d02" exitCode=0
Jul 24 15:36:40.248956 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:40.248879    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l7xpv" event={"ID":"c9cb3936-a642-4db6-a658-e4c3f3b9c80a","Type":"ContainerDied","Data":"85c38b33cc628959045d92f2ac61802bc2ecb7a998933fa7231acfde4e773d02"}
Jul 24 15:36:40.263768 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:40.263751    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:40.274986 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:40.274935    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2" podStartSLOduration=9.34186242 podStartE2EDuration="28.274919379s" totalImagesPullingTime="18.933056959s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:40.274732697 +0000 UTC m=+28.685443315" watchObservedRunningTime="2026-07-24 15:36:40.274919379 +0000 UTC m=+28.685629997"
Jul 24 15:36:40.424373 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:40.424158    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lvgwz_f144fc9d-a69b-4cb5-b99c-2092a7905b32/konnectivity-agent/0.log" limit=1024
Jul 24 15:36:40.517811 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:40.517770    2561 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lvgwz_f144fc9d-a69b-4cb5-b99c-2092a7905b32/konnectivity-agent/0.log" limit=1024
Jul 24 15:36:41.101090 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:41.101058    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:41.101288 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:41.101192    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:41.251727 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:41.251698    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:41.267424 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:41.267401    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:36:41.852652 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:41.852619    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vmnkg"]
Jul 24 15:36:41.853159 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:41.852777    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:41.853159 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:41.852895    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmnkg" podUID="f1958e1f-5868-4a15-847c-28d61994d060"
Jul 24 15:36:41.853387 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:41.853368    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j8fqw"]
Jul 24 15:36:41.853468 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:41.853455    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:41.853607 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:41.853548    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8fqw" podUID="94e9b736-6944-4781-9b40-500ec82d10e2"
Jul 24 15:36:41.854041 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:41.854019    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qrsfq"]
Jul 24 15:36:41.854106 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:41.854086    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:41.854184 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:41.854164    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:42.258417 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:42.256023    2561 generic.go:356] "Generic (PLEG): container finished" podID="c9cb3936-a642-4db6-a658-e4c3f3b9c80a" containerID="3a8df9cd356071d7fa79d20e0dffc06be51abcb3b9ec7f2a44545f46b1ab7938" exitCode=0
Jul 24 15:36:42.258417 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:42.257002    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l7xpv" event={"ID":"c9cb3936-a642-4db6-a658-e4c3f3b9c80a","Type":"ContainerDied","Data":"3a8df9cd356071d7fa79d20e0dffc06be51abcb3b9ec7f2a44545f46b1ab7938"}
Jul 24 15:36:43.100882 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:43.100811    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:43.101263 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:43.100919    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmnkg" podUID="f1958e1f-5868-4a15-847c-28d61994d060"
Jul 24 15:36:44.103241 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:44.103071    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:44.103622 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:44.103073    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:44.103622 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:44.103312    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8fqw" podUID="94e9b736-6944-4781-9b40-500ec82d10e2"
Jul 24 15:36:44.103622 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:44.103370    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:44.263874 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:44.263835    2561 generic.go:356] "Generic (PLEG): container finished" podID="c9cb3936-a642-4db6-a658-e4c3f3b9c80a" containerID="9eaebb83b95d84030082cf10716b7cb5479dd6cf244d9b06ac2f3d381ed49c21" exitCode=0
Jul 24 15:36:44.264006 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:44.263886    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l7xpv" event={"ID":"c9cb3936-a642-4db6-a658-e4c3f3b9c80a","Type":"ContainerDied","Data":"9eaebb83b95d84030082cf10716b7cb5479dd6cf244d9b06ac2f3d381ed49c21"}
Jul 24 15:36:45.100599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:45.100567    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:45.100765 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:45.100740    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vmnkg" podUID="f1958e1f-5868-4a15-847c-28d61994d060"
Jul 24 15:36:45.645854 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:45.645807    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jngr\" (UniqueName: \"kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr\") pod \"network-check-target-qrsfq\" (UID: \"ac6f7393-c091-44d0-9e64-4b58651a24b8\") " pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:45.646272 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:45.645900    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret\") pod \"global-pull-secret-syncer-j8fqw\" (UID: \"94e9b736-6944-4781-9b40-500ec82d10e2\") " pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:45.646272 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:45.645990    2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:36:45.646272 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:45.645993    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:36:45.646272 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:45.646021    2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:36:45.646272 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:45.646034    2561 projected.go:196] Error preparing data for projected volume kube-api-access-6jngr for pod openshift-network-diagnostics/network-check-target-qrsfq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:36:45.646272 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:45.646040    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret podName:94e9b736-6944-4781-9b40-500ec82d10e2 nodeName:}" failed. No retries permitted until 2026-07-24 15:37:17.646026693 +0000 UTC m=+66.056737288 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret") pod "global-pull-secret-syncer-j8fqw" (UID: "94e9b736-6944-4781-9b40-500ec82d10e2") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:36:45.646272 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:45.646084    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr podName:ac6f7393-c091-44d0-9e64-4b58651a24b8 nodeName:}" failed. No retries permitted until 2026-07-24 15:37:17.646067737 +0000 UTC m=+66.056778334 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6jngr" (UniqueName: "kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr") pod "network-check-target-qrsfq" (UID: "ac6f7393-c091-44d0-9e64-4b58651a24b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:36:45.746597 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:45.746563    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs\") pod \"network-metrics-daemon-vmnkg\" (UID: \"f1958e1f-5868-4a15-847c-28d61994d060\") " pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:45.746792 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:45.746699    2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:36:45.746792 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:45.746761    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs podName:f1958e1f-5868-4a15-847c-28d61994d060 nodeName:}" failed. No retries permitted until 2026-07-24 15:37:17.74674312 +0000 UTC m=+66.157453714 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs") pod "network-metrics-daemon-vmnkg" (UID: "f1958e1f-5868-4a15-847c-28d61994d060") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:36:46.104382 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.104350    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:46.104560 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:46.104466    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-j8fqw" podUID="94e9b736-6944-4781-9b40-500ec82d10e2"
Jul 24 15:36:46.104560 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.104510    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:46.104646 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:46.104624    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qrsfq" podUID="ac6f7393-c091-44d0-9e64-4b58651a24b8"
Jul 24 15:36:46.436729 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.436704    2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-164.ec2.internal" event="NodeReady"
Jul 24 15:36:46.436891 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.436881    2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 15:36:46.485012 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.484978    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mf5pv"]
Jul 24 15:36:46.489016 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.488992    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kfwqt"]
Jul 24 15:36:46.489162 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.489117    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:36:46.491280 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.491261    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r6s28\""
Jul 24 15:36:46.491397 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.491293    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 15:36:46.491397 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.491293    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 15:36:46.492789 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.492775    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"]
Jul 24 15:36:46.492903 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.492890    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kfwqt"
Jul 24 15:36:46.494838 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.494822    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lrd6q\""
Jul 24 15:36:46.494838 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.494833    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 15:36:46.495086 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.495049    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 15:36:46.495204 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.495153    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 15:36:46.496624 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.496605    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.498573 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.498550    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mf5pv"]
Jul 24 15:36:46.499444 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.498904    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ljdvh\""
Jul 24 15:36:46.499444 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.499140    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 24 15:36:46.499444 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.499127    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 24 15:36:46.499444 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.499351    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 24 15:36:46.501266 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.501244    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kfwqt"]
Jul 24 15:36:46.504907 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.504887    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 24 15:36:46.519891 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.519865    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"]
Jul 24 15:36:46.653893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.653799    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-bound-sa-token\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.653893 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.653847    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-tmp-dir\") pod \"dns-default-mf5pv\" (UID: \"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9\") " pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:36:46.654311 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.653990    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d88d473b-18bf-4f9d-98ef-518843c19345-ca-trust-extracted\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.654311 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.654011    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d88d473b-18bf-4f9d-98ef-518843c19345-registry-certificates\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.654311 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.654028    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d88d473b-18bf-4f9d-98ef-518843c19345-installation-pull-secrets\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.654311 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.654071    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9bds\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-kube-api-access-l9bds\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.654311 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.654186    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls\") pod \"dns-default-mf5pv\" (UID: \"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9\") " pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:36:46.654311 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.654254    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d88d473b-18bf-4f9d-98ef-518843c19345-image-registry-private-configuration\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.654311 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.654276    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-config-volume\") pod \"dns-default-mf5pv\" (UID: \"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9\") " pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:36:46.654311 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.654290    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9699ef5a-c5cc-4b62-9e8d-17c6f686571b-cert\") pod \"ingress-canary-kfwqt\" (UID: \"9699ef5a-c5cc-4b62-9e8d-17c6f686571b\") " pod="openshift-ingress-canary/ingress-canary-kfwqt"
Jul 24 15:36:46.654311 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.654307    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hw79\" (UniqueName: \"kubernetes.io/projected/9699ef5a-c5cc-4b62-9e8d-17c6f686571b-kube-api-access-6hw79\") pod \"ingress-canary-kfwqt\" (UID: \"9699ef5a-c5cc-4b62-9e8d-17c6f686571b\") " pod="openshift-ingress-canary/ingress-canary-kfwqt"
Jul 24 15:36:46.654695 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.654330    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x2xq\" (UniqueName: \"kubernetes.io/projected/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-kube-api-access-4x2xq\") pod \"dns-default-mf5pv\" (UID: \"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9\") " pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:36:46.654695 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.654426    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.654695 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.654459    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d88d473b-18bf-4f9d-98ef-518843c19345-trusted-ca\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.755675 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.755633    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-bound-sa-token\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.755859 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.755695    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-tmp-dir\") pod \"dns-default-mf5pv\" (UID: \"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9\") " pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:36:46.755859 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.755744    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d88d473b-18bf-4f9d-98ef-518843c19345-ca-trust-extracted\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.755859 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.755768    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d88d473b-18bf-4f9d-98ef-518843c19345-registry-certificates\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.755859 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.755794    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d88d473b-18bf-4f9d-98ef-518843c19345-installation-pull-secrets\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.755859 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.755817    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9bds\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-kube-api-access-l9bds\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.755859 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.755843    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls\") pod \"dns-default-mf5pv\" (UID: \"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9\") " pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:36:46.756142 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.755890    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d88d473b-18bf-4f9d-98ef-518843c19345-image-registry-private-configuration\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.756142 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.755917    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-config-volume\") pod \"dns-default-mf5pv\" (UID: \"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9\") " pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:36:46.756142 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.755940    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9699ef5a-c5cc-4b62-9e8d-17c6f686571b-cert\") pod \"ingress-canary-kfwqt\" (UID: \"9699ef5a-c5cc-4b62-9e8d-17c6f686571b\") " pod="openshift-ingress-canary/ingress-canary-kfwqt"
Jul 24 15:36:46.756142 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.755969    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hw79\" (UniqueName: \"kubernetes.io/projected/9699ef5a-c5cc-4b62-9e8d-17c6f686571b-kube-api-access-6hw79\") pod \"ingress-canary-kfwqt\" (UID: \"9699ef5a-c5cc-4b62-9e8d-17c6f686571b\") " pod="openshift-ingress-canary/ingress-canary-kfwqt"
Jul 24 15:36:46.756142 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:46.755975    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:36:46.756142 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.755997    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x2xq\" (UniqueName: \"kubernetes.io/projected/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-kube-api-access-4x2xq\") pod \"dns-default-mf5pv\" (UID: \"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9\") " pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:36:46.756142 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.756042    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.756142 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.756070    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d88d473b-18bf-4f9d-98ef-518843c19345-trusted-ca\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.756142 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.756102    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-tmp-dir\") pod \"dns-default-mf5pv\" (UID: \"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9\") " pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:36:46.756446 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.756183    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d88d473b-18bf-4f9d-98ef-518843c19345-ca-trust-extracted\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.756446 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:46.756197    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 15:36:46.756446 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:46.756211    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f5d4b4c6-jnhnd: secret "image-registry-tls" not found
Jul 24 15:36:46.756446 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:46.756235    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls podName:68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:47.256213307 +0000 UTC m=+35.666923916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls") pod "dns-default-mf5pv" (UID: "68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9") : secret "dns-default-metrics-tls" not found
Jul 24 15:36:46.756446 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:46.756267    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls podName:d88d473b-18bf-4f9d-98ef-518843c19345 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:47.256251429 +0000 UTC m=+35.666962038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls") pod "image-registry-65f5d4b4c6-jnhnd" (UID: "d88d473b-18bf-4f9d-98ef-518843c19345") : secret "image-registry-tls" not found
Jul 24 15:36:46.756446 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.756438    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-config-volume\") pod \"dns-default-mf5pv\" (UID: \"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9\") " pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:36:46.756905 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.756881    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d88d473b-18bf-4f9d-98ef-518843c19345-registry-certificates\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.757337 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.757295    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d88d473b-18bf-4f9d-98ef-518843c19345-trusted-ca\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.760946 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.760922    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9699ef5a-c5cc-4b62-9e8d-17c6f686571b-cert\") pod \"ingress-canary-kfwqt\" (UID: \"9699ef5a-c5cc-4b62-9e8d-17c6f686571b\") " pod="openshift-ingress-canary/ingress-canary-kfwqt"
Jul 24 15:36:46.760946 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.760940    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d88d473b-18bf-4f9d-98ef-518843c19345-image-registry-private-configuration\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.761072 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.760980    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d88d473b-18bf-4f9d-98ef-518843c19345-installation-pull-secrets\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.763989 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.763968    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hw79\" (UniqueName: \"kubernetes.io/projected/9699ef5a-c5cc-4b62-9e8d-17c6f686571b-kube-api-access-6hw79\") pod \"ingress-canary-kfwqt\" (UID: \"9699ef5a-c5cc-4b62-9e8d-17c6f686571b\") " pod="openshift-ingress-canary/ingress-canary-kfwqt"
Jul 24 15:36:46.764094 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.764057    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9bds\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-kube-api-access-l9bds\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.765053 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.765030    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-bound-sa-token\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:46.765236 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.765217    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x2xq\" (UniqueName: \"kubernetes.io/projected/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-kube-api-access-4x2xq\") pod \"dns-default-mf5pv\" (UID: \"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9\") " pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:36:46.809095 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.809059    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kfwqt"
Jul 24 15:36:46.958706 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:46.958647    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kfwqt"]
Jul 24 15:36:46.964525 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:46.964493    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9699ef5a_c5cc_4b62_9e8d_17c6f686571b.slice/crio-36653324a5e82f7fc3eca53142a5fe8af2fa1499302a0a952808ca26d7d738ab WatchSource:0}: Error finding container 36653324a5e82f7fc3eca53142a5fe8af2fa1499302a0a952808ca26d7d738ab: Status 404 returned error can't find the container with id 36653324a5e82f7fc3eca53142a5fe8af2fa1499302a0a952808ca26d7d738ab
Jul 24 15:36:47.100892 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:47.100860    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:36:47.103444 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:47.103417    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 15:36:47.103444 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:47.103436    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w77kg\""
Jul 24 15:36:47.259322 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:47.259291    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:47.259522 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:47.259363    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls\") pod \"dns-default-mf5pv\" (UID: \"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9\") " pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:36:47.259522 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:47.259447    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 15:36:47.259522 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:47.259464    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f5d4b4c6-jnhnd: secret "image-registry-tls" not found
Jul 24 15:36:47.259645 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:47.259525    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:36:47.259645 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:47.259542    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls podName:d88d473b-18bf-4f9d-98ef-518843c19345 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:48.259523009 +0000 UTC m=+36.670233606 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls") pod "image-registry-65f5d4b4c6-jnhnd" (UID: "d88d473b-18bf-4f9d-98ef-518843c19345") : secret "image-registry-tls" not found
Jul 24 15:36:47.259645 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:47.259584    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls podName:68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:48.259570219 +0000 UTC m=+36.670280829 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls") pod "dns-default-mf5pv" (UID: "68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9") : secret "dns-default-metrics-tls" not found
Jul 24 15:36:47.272960 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:47.272927    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kfwqt" event={"ID":"9699ef5a-c5cc-4b62-9e8d-17c6f686571b","Type":"ContainerStarted","Data":"36653324a5e82f7fc3eca53142a5fe8af2fa1499302a0a952808ca26d7d738ab"}
Jul 24 15:36:47.922632 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:47.922392    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-s4pwm"]
Jul 24 15:36:47.927546 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:47.927520    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s4pwm"
Jul 24 15:36:47.930213 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:47.930191    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 24 15:36:47.930213 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:47.930203    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 24 15:36:47.931012 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:47.930995    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5xqks\""
Jul 24 15:36:47.938232 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:47.938210    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-s4pwm"]
Jul 24 15:36:48.066449 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.066412    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vmgv\" (UniqueName: \"kubernetes.io/projected/4cc2d387-c3f0-4b9d-a571-8d0e133ae85f-kube-api-access-5vmgv\") pod \"migrator-6865c9967-s4pwm\" (UID: \"4cc2d387-c3f0-4b9d-a571-8d0e133ae85f\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s4pwm"
Jul 24 15:36:48.105127 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.105100    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:36:48.105287 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.105256    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:36:48.107533 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.107510    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t6g6g\""
Jul 24 15:36:48.107533 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.107522    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 15:36:48.107533 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.107510    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 15:36:48.107790 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.107518    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 15:36:48.167327 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.167284    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vmgv\" (UniqueName: \"kubernetes.io/projected/4cc2d387-c3f0-4b9d-a571-8d0e133ae85f-kube-api-access-5vmgv\") pod \"migrator-6865c9967-s4pwm\" (UID: \"4cc2d387-c3f0-4b9d-a571-8d0e133ae85f\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s4pwm"
Jul 24 15:36:48.175707 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.175629    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vmgv\" (UniqueName: \"kubernetes.io/projected/4cc2d387-c3f0-4b9d-a571-8d0e133ae85f-kube-api-access-5vmgv\") pod \"migrator-6865c9967-s4pwm\" (UID: \"4cc2d387-c3f0-4b9d-a571-8d0e133ae85f\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s4pwm"
Jul 24 15:36:48.238695 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.238638    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s4pwm"
Jul 24 15:36:48.268048 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.268013    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:48.268188 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.268116    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls\") pod \"dns-default-mf5pv\" (UID: \"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9\") " pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:36:48.268244 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:48.268198    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 15:36:48.268244 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:48.268222    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f5d4b4c6-jnhnd: secret "image-registry-tls" not found
Jul 24 15:36:48.268342 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:48.268283    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:36:48.268342 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:48.268293    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls podName:d88d473b-18bf-4f9d-98ef-518843c19345 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:50.268272974 +0000 UTC m=+38.678983584 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls") pod "image-registry-65f5d4b4c6-jnhnd" (UID: "d88d473b-18bf-4f9d-98ef-518843c19345") : secret "image-registry-tls" not found
Jul 24 15:36:48.268428 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:48.268360    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls podName:68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:50.268333414 +0000 UTC m=+38.679044012 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls") pod "dns-default-mf5pv" (UID: "68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9") : secret "dns-default-metrics-tls" not found
Jul 24 15:36:48.512487 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.512452    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-88dmg"]
Jul 24 15:36:48.515965 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.515940    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:48.518160 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.518137    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 15:36:48.518364 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.518338    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 15:36:48.518458 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.518419    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 15:36:48.518458 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.518452    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 15:36:48.518743 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.518729    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9pmnt\""
Jul 24 15:36:48.529638 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.529615    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-88dmg"]
Jul 24 15:36:48.671674 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.671638    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bad5c0b7-e451-4460-886d-d43b45404633-crio-socket\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:48.671865 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.671770    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:48.671865 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.671814    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bad5c0b7-e451-4460-886d-d43b45404633-data-volume\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:48.671972 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.671864    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:48.671972 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.671886    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w647r\" (UniqueName: \"kubernetes.io/projected/bad5c0b7-e451-4460-886d-d43b45404633-kube-api-access-w647r\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:48.772424 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.772333    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:48.772424 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.772382    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w647r\" (UniqueName: \"kubernetes.io/projected/bad5c0b7-e451-4460-886d-d43b45404633-kube-api-access-w647r\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:48.772650 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:48.772501    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:36:48.772650 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.772566    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bad5c0b7-e451-4460-886d-d43b45404633-crio-socket\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:48.772650 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:48.772582    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm podName:bad5c0b7-e451-4460-886d-d43b45404633 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:49.27256307 +0000 UTC m=+37.683273697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm") pod "insights-runtime-extractor-88dmg" (UID: "bad5c0b7-e451-4460-886d-d43b45404633") : configmap "kube-rbac-proxy" not found
Jul 24 15:36:48.772650 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.772644    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:48.772886 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.772699    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bad5c0b7-e451-4460-886d-d43b45404633-data-volume\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:48.772886 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:48.772854    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:36:48.772988 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.772889    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bad5c0b7-e451-4460-886d-d43b45404633-crio-socket\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:48.772988 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:48.772903    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls podName:bad5c0b7-e451-4460-886d-d43b45404633 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:49.272890365 +0000 UTC m=+37.683600963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88dmg" (UID: "bad5c0b7-e451-4460-886d-d43b45404633") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:36:48.773096 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.773074    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bad5c0b7-e451-4460-886d-d43b45404633-data-volume\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:48.781424 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:48.781398    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w647r\" (UniqueName: \"kubernetes.io/projected/bad5c0b7-e451-4460-886d-d43b45404633-kube-api-access-w647r\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:49.026073 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:49.026043    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-s4pwm"]
Jul 24 15:36:49.030350 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:49.030322    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cc2d387_c3f0_4b9d_a571_8d0e133ae85f.slice/crio-c11ee88eb26d9ee21b9c88fddc76ba59cc9f77823528d43059c0a20e4f725ba9 WatchSource:0}: Error finding container c11ee88eb26d9ee21b9c88fddc76ba59cc9f77823528d43059c0a20e4f725ba9: Status 404 returned error can't find the container with id c11ee88eb26d9ee21b9c88fddc76ba59cc9f77823528d43059c0a20e4f725ba9
Jul 24 15:36:49.277136 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:49.277051    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:49.277136 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:49.277121    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:49.277366 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:49.277166    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:36:49.277366 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:49.277229    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls podName:bad5c0b7-e451-4460-886d-d43b45404633 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:50.277210283 +0000 UTC m=+38.687920881 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88dmg" (UID: "bad5c0b7-e451-4460-886d-d43b45404633") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:36:49.277366 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:49.277252    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:36:49.277366 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:49.277296    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm podName:bad5c0b7-e451-4460-886d-d43b45404633 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:50.277283373 +0000 UTC m=+38.687993968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm") pod "insights-runtime-extractor-88dmg" (UID: "bad5c0b7-e451-4460-886d-d43b45404633") : configmap "kube-rbac-proxy" not found
Jul 24 15:36:49.279484 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:49.279452    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s4pwm" event={"ID":"4cc2d387-c3f0-4b9d-a571-8d0e133ae85f","Type":"ContainerStarted","Data":"c11ee88eb26d9ee21b9c88fddc76ba59cc9f77823528d43059c0a20e4f725ba9"}
Jul 24 15:36:49.282704 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:49.282681    2561 generic.go:356] "Generic (PLEG): container finished" podID="c9cb3936-a642-4db6-a658-e4c3f3b9c80a" containerID="15c4a33cc2dc3b7e0e0410cab100c12be3055302b05d055511c456ec1855785b" exitCode=0
Jul 24 15:36:49.282813 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:49.282721    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l7xpv" event={"ID":"c9cb3936-a642-4db6-a658-e4c3f3b9c80a","Type":"ContainerDied","Data":"15c4a33cc2dc3b7e0e0410cab100c12be3055302b05d055511c456ec1855785b"}
Jul 24 15:36:50.285545 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.285513    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:50.286018 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.285568    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls\") pod \"dns-default-mf5pv\" (UID: \"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9\") " pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:36:50.286018 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.285638    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:50.286018 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:50.285655    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:36:50.286018 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.285688    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:50.286018 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:50.285750    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm podName:bad5c0b7-e451-4460-886d-d43b45404633 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:52.285726833 +0000 UTC m=+40.696437460 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm") pod "insights-runtime-extractor-88dmg" (UID: "bad5c0b7-e451-4460-886d-d43b45404633") : configmap "kube-rbac-proxy" not found
Jul 24 15:36:50.286018 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:50.285788    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:36:50.286018 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:50.285839    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls podName:bad5c0b7-e451-4460-886d-d43b45404633 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:52.28581578 +0000 UTC m=+40.696526387 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88dmg" (UID: "bad5c0b7-e451-4460-886d-d43b45404633") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:36:50.286018 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:50.285886    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:36:50.286018 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:50.285916    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 15:36:50.286018 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:50.285933    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls podName:68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:54.285918913 +0000 UTC m=+42.696629521 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls") pod "dns-default-mf5pv" (UID: "68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9") : secret "dns-default-metrics-tls" not found
Jul 24 15:36:50.286018 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:50.285934    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f5d4b4c6-jnhnd: secret "image-registry-tls" not found
Jul 24 15:36:50.286018 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:50.285985    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls podName:d88d473b-18bf-4f9d-98ef-518843c19345 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:54.285967579 +0000 UTC m=+42.696678175 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls") pod "image-registry-65f5d4b4c6-jnhnd" (UID: "d88d473b-18bf-4f9d-98ef-518843c19345") : secret "image-registry-tls" not found
Jul 24 15:36:50.289504 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.289482    2561 generic.go:356] "Generic (PLEG): container finished" podID="c9cb3936-a642-4db6-a658-e4c3f3b9c80a" containerID="ecf489702878f022601800428e61a5e52d3fb974e1584795fe3274bc7bb74e66" exitCode=0
Jul 24 15:36:50.289615 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.289548    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l7xpv" event={"ID":"c9cb3936-a642-4db6-a658-e4c3f3b9c80a","Type":"ContainerDied","Data":"ecf489702878f022601800428e61a5e52d3fb974e1584795fe3274bc7bb74e66"}
Jul 24 15:36:50.688244 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.688157    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-vm5ff"]
Jul 24 15:36:50.706358 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.706327    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-vm5ff"]
Jul 24 15:36:50.706545 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.706459    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-vm5ff"
Jul 24 15:36:50.712416 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.712395    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 24 15:36:50.712416 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.712406    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 24 15:36:50.712580 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.712542    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 24 15:36:50.713070 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.713048    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 24 15:36:50.713487 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.713298    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-7dnjd\""
Jul 24 15:36:50.789150 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.789111    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5819533a-d92c-458c-bcc0-97baf0beee7f-signing-key\") pod \"service-ca-658d7b7c78-vm5ff\" (UID: \"5819533a-d92c-458c-bcc0-97baf0beee7f\") " pod="openshift-service-ca/service-ca-658d7b7c78-vm5ff"
Jul 24 15:36:50.789150 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.789152    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n56cp\" (UniqueName: \"kubernetes.io/projected/5819533a-d92c-458c-bcc0-97baf0beee7f-kube-api-access-n56cp\") pod \"service-ca-658d7b7c78-vm5ff\" (UID: \"5819533a-d92c-458c-bcc0-97baf0beee7f\") " pod="openshift-service-ca/service-ca-658d7b7c78-vm5ff"
Jul 24 15:36:50.789368 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.789241    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5819533a-d92c-458c-bcc0-97baf0beee7f-signing-cabundle\") pod \"service-ca-658d7b7c78-vm5ff\" (UID: \"5819533a-d92c-458c-bcc0-97baf0beee7f\") " pod="openshift-service-ca/service-ca-658d7b7c78-vm5ff"
Jul 24 15:36:50.890103 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.890067    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5819533a-d92c-458c-bcc0-97baf0beee7f-signing-cabundle\") pod \"service-ca-658d7b7c78-vm5ff\" (UID: \"5819533a-d92c-458c-bcc0-97baf0beee7f\") " pod="openshift-service-ca/service-ca-658d7b7c78-vm5ff"
Jul 24 15:36:50.890297 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.890215    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5819533a-d92c-458c-bcc0-97baf0beee7f-signing-key\") pod \"service-ca-658d7b7c78-vm5ff\" (UID: \"5819533a-d92c-458c-bcc0-97baf0beee7f\") " pod="openshift-service-ca/service-ca-658d7b7c78-vm5ff"
Jul 24 15:36:50.890297 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.890243    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n56cp\" (UniqueName: \"kubernetes.io/projected/5819533a-d92c-458c-bcc0-97baf0beee7f-kube-api-access-n56cp\") pod \"service-ca-658d7b7c78-vm5ff\" (UID: \"5819533a-d92c-458c-bcc0-97baf0beee7f\") " pod="openshift-service-ca/service-ca-658d7b7c78-vm5ff"
Jul 24 15:36:50.890868 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.890841    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5819533a-d92c-458c-bcc0-97baf0beee7f-signing-cabundle\") pod \"service-ca-658d7b7c78-vm5ff\" (UID: \"5819533a-d92c-458c-bcc0-97baf0beee7f\") " pod="openshift-service-ca/service-ca-658d7b7c78-vm5ff"
Jul 24 15:36:50.895724 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.895531    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5819533a-d92c-458c-bcc0-97baf0beee7f-signing-key\") pod \"service-ca-658d7b7c78-vm5ff\" (UID: \"5819533a-d92c-458c-bcc0-97baf0beee7f\") " pod="openshift-service-ca/service-ca-658d7b7c78-vm5ff"
Jul 24 15:36:50.899932 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:50.899033    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n56cp\" (UniqueName: \"kubernetes.io/projected/5819533a-d92c-458c-bcc0-97baf0beee7f-kube-api-access-n56cp\") pod \"service-ca-658d7b7c78-vm5ff\" (UID: \"5819533a-d92c-458c-bcc0-97baf0beee7f\") " pod="openshift-service-ca/service-ca-658d7b7c78-vm5ff"
Jul 24 15:36:51.017063 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:51.017027    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-vm5ff"
Jul 24 15:36:51.088593 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:51.088567    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s6wn2_aa3224e2-ecb7-4164-aa12-48213a6a5210/dns-node-resolver/0.log"
Jul 24 15:36:51.835821 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:51.835723    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-vm5ff"]
Jul 24 15:36:51.840562 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:36:51.840532    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5819533a_d92c_458c_bcc0_97baf0beee7f.slice/crio-de32f0f4d35636ca8eb5151c28c57d899c7897a7ebaa0cc036150b9f259e3dcb WatchSource:0}: Error finding container de32f0f4d35636ca8eb5151c28c57d899c7897a7ebaa0cc036150b9f259e3dcb: Status 404 returned error can't find the container with id de32f0f4d35636ca8eb5151c28c57d899c7897a7ebaa0cc036150b9f259e3dcb
Jul 24 15:36:52.089427 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:52.089404    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lsj8p_076c0fb1-5f86-4a0b-8725-100ea9966c41/node-ca/0.log"
Jul 24 15:36:52.295156 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:52.295128    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s4pwm" event={"ID":"4cc2d387-c3f0-4b9d-a571-8d0e133ae85f","Type":"ContainerStarted","Data":"62410ab1b4cfc67b745710b8afb7b0e526a56fcabfe8e4f5d81dfd4e66bebc83"}
Jul 24 15:36:52.295331 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:52.295192    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s4pwm" event={"ID":"4cc2d387-c3f0-4b9d-a571-8d0e133ae85f","Type":"ContainerStarted","Data":"2ab9ae9b68f88841d190c0805c85c96f071c2018a744589ff38f9152d1b9d41f"}
Jul 24 15:36:52.296313 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:52.296293    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-vm5ff" event={"ID":"5819533a-d92c-458c-bcc0-97baf0beee7f","Type":"ContainerStarted","Data":"de32f0f4d35636ca8eb5151c28c57d899c7897a7ebaa0cc036150b9f259e3dcb"}
Jul 24 15:36:52.299574 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:52.299548    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-l7xpv" event={"ID":"c9cb3936-a642-4db6-a658-e4c3f3b9c80a","Type":"ContainerStarted","Data":"57506c15e56dbc57a6c2a156695e09367d6101b24728c10887476235a9c4679d"}
Jul 24 15:36:52.300849 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:52.300828    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kfwqt" event={"ID":"9699ef5a-c5cc-4b62-9e8d-17c6f686571b","Type":"ContainerStarted","Data":"6748f209955608c7e4684c9b734df80d27b1f341b46bada3c131a1c8c0c6da4a"}
Jul 24 15:36:52.302224 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:52.302207    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:52.302281 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:52.302251    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:52.302326 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:52.302318    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:36:52.302366 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:52.302328    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:36:52.302401 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:52.302372    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls podName:bad5c0b7-e451-4460-886d-d43b45404633 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:56.302358857 +0000 UTC m=+44.713069452 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88dmg" (UID: "bad5c0b7-e451-4460-886d-d43b45404633") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:36:52.302437 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:52.302415    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm podName:bad5c0b7-e451-4460-886d-d43b45404633 nodeName:}" failed. No retries permitted until 2026-07-24 15:36:56.302408697 +0000 UTC m=+44.713119292 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm") pod "insights-runtime-extractor-88dmg" (UID: "bad5c0b7-e451-4460-886d-d43b45404633") : configmap "kube-rbac-proxy" not found
Jul 24 15:36:52.313797 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:52.313754    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-s4pwm" podStartSLOduration=2.660845974 podStartE2EDuration="5.313741477s" totalImagesPullingTime="2.652895503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:36:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:52.312561627 +0000 UTC m=+40.723272239" watchObservedRunningTime="2026-07-24 15:36:52.313741477 +0000 UTC m=+40.724452093"
Jul 24 15:36:52.334837 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:52.334791    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-l7xpv" podStartSLOduration=9.104914145 podStartE2EDuration="40.334777045s" totalImagesPullingTime="31.2298629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:52.334178561 +0000 UTC m=+40.744889178" watchObservedRunningTime="2026-07-24 15:36:52.334777045 +0000 UTC m=+40.745487640"
Jul 24 15:36:52.350325 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:52.350278    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kfwqt" podStartSLOduration=1.634290306 podStartE2EDuration="6.350265898s" totalImagesPullingTime="4.715975592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:36:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:52.349637267 +0000 UTC m=+40.760347919" watchObservedRunningTime="2026-07-24 15:36:52.350265898 +0000 UTC m=+40.760976515"
Jul 24 15:36:52.689865 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:52.689783    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kfwqt_9699ef5a-c5cc-4b62-9e8d-17c6f686571b/serve-healthcheck-canary/0.log"
Jul 24 15:36:52.889500 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:52.889468    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-s4pwm_4cc2d387-c3f0-4b9d-a571-8d0e133ae85f/migrator/0.log"
Jul 24 15:36:53.088962 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:53.088936    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-s4pwm_4cc2d387-c3f0-4b9d-a571-8d0e133ae85f/graceful-termination/0.log"
Jul 24 15:36:54.307714 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:54.307678    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-vm5ff" event={"ID":"5819533a-d92c-458c-bcc0-97baf0beee7f","Type":"ContainerStarted","Data":"53df91490b345da27fcd6407efd1a55ff83b539621e4ca043e077e452d268e95"}
Jul 24 15:36:54.319839 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:54.319812    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls\") pod \"dns-default-mf5pv\" (UID: \"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9\") " pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:36:54.319932 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:54.319869    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:36:54.319974 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:54.319953    2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:36:54.320004 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:54.319994    2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 24 15:36:54.320036 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:54.320006    2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65f5d4b4c6-jnhnd: secret "image-registry-tls" not found
Jul 24 15:36:54.320036 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:54.320009    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls podName:68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9 nodeName:}" failed. No retries permitted until 2026-07-24 15:37:02.31999368 +0000 UTC m=+50.730704280 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls") pod "dns-default-mf5pv" (UID: "68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9") : secret "dns-default-metrics-tls" not found
Jul 24 15:36:54.320102 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:54.320040    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls podName:d88d473b-18bf-4f9d-98ef-518843c19345 nodeName:}" failed. No retries permitted until 2026-07-24 15:37:02.320029934 +0000 UTC m=+50.730740528 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls") pod "image-registry-65f5d4b4c6-jnhnd" (UID: "d88d473b-18bf-4f9d-98ef-518843c19345") : secret "image-registry-tls" not found
Jul 24 15:36:54.324397 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:54.324355    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-vm5ff" podStartSLOduration=2.032026077 podStartE2EDuration="4.324341258s" totalImagesPullingTime="2.292315181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:36:54.323527835 +0000 UTC m=+42.734238453" watchObservedRunningTime="2026-07-24 15:36:54.324341258 +0000 UTC m=+42.735051903"
Jul 24 15:36:56.337093 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:56.337061    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:56.337544 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:36:56.337110    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:36:56.337544 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:56.337191    2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:36:56.337544 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:56.337241    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm podName:bad5c0b7-e451-4460-886d-d43b45404633 nodeName:}" failed. No retries permitted until 2026-07-24 15:37:04.337228131 +0000 UTC m=+52.747938726 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm") pod "insights-runtime-extractor-88dmg" (UID: "bad5c0b7-e451-4460-886d-d43b45404633") : configmap "kube-rbac-proxy" not found
Jul 24 15:36:56.337544 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:56.337254    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:36:56.337544 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:36:56.337326    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls podName:bad5c0b7-e451-4460-886d-d43b45404633 nodeName:}" failed. No retries permitted until 2026-07-24 15:37:04.337305665 +0000 UTC m=+52.748016277 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88dmg" (UID: "bad5c0b7-e451-4460-886d-d43b45404633") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:37:02.384911 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:02.384867    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls\") pod \"dns-default-mf5pv\" (UID: \"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9\") " pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:37:02.385465 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:02.384934    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:37:02.387574 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:02.387552    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9-metrics-tls\") pod \"dns-default-mf5pv\" (UID: \"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9\") " pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:37:02.387688 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:02.387645    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls\") pod \"image-registry-65f5d4b4c6-jnhnd\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") " pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:37:02.401568 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:02.401537    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:37:02.415700 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:02.415670    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:37:02.559425 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:02.559393    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mf5pv"]
Jul 24 15:37:02.562156 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:37:02.562127    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68ecf7f0_483c_4c8d_bf84_e61e1bb89fb9.slice/crio-ea5d2b5b02f37c6e42c42b7c4aaba5927a6c4877fef8ccdccf58f1460aff786a WatchSource:0}: Error finding container ea5d2b5b02f37c6e42c42b7c4aaba5927a6c4877fef8ccdccf58f1460aff786a: Status 404 returned error can't find the container with id ea5d2b5b02f37c6e42c42b7c4aaba5927a6c4877fef8ccdccf58f1460aff786a
Jul 24 15:37:02.569586 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:02.569562    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"]
Jul 24 15:37:02.590643 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:37:02.590612    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd88d473b_18bf_4f9d_98ef_518843c19345.slice/crio-1098ecde048abccbabbbf892d9723608a5e8b7cc8daf7d66fdb864510328e76a WatchSource:0}: Error finding container 1098ecde048abccbabbbf892d9723608a5e8b7cc8daf7d66fdb864510328e76a: Status 404 returned error can't find the container with id 1098ecde048abccbabbbf892d9723608a5e8b7cc8daf7d66fdb864510328e76a
Jul 24 15:37:03.331038 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:03.331007    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mf5pv" event={"ID":"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9","Type":"ContainerStarted","Data":"ea5d2b5b02f37c6e42c42b7c4aaba5927a6c4877fef8ccdccf58f1460aff786a"}
Jul 24 15:37:03.332584 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:03.332558    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd" event={"ID":"d88d473b-18bf-4f9d-98ef-518843c19345","Type":"ContainerStarted","Data":"6a688cd30f0de898100ab9839fee95da0665d5f020011fa645ceb78b873d9e5c"}
Jul 24 15:37:03.332733 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:03.332598    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd" event={"ID":"d88d473b-18bf-4f9d-98ef-518843c19345","Type":"ContainerStarted","Data":"1098ecde048abccbabbbf892d9723608a5e8b7cc8daf7d66fdb864510328e76a"}
Jul 24 15:37:03.332733 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:03.332702    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:37:03.355171 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:03.355122    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd" podStartSLOduration=53.355106911 podStartE2EDuration="53.355106911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:36:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:03.35376891 +0000 UTC m=+51.764479552" watchObservedRunningTime="2026-07-24 15:37:03.355106911 +0000 UTC m=+51.765817527"
Jul 24 15:37:04.398124 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:04.398086    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:37:04.398634 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:04.398191    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:37:04.398634 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:37:04.398494    2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:37:04.398634 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:37:04.398566    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls podName:bad5c0b7-e451-4460-886d-d43b45404633 nodeName:}" failed. No retries permitted until 2026-07-24 15:37:20.398547354 +0000 UTC m=+68.809257964 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls") pod "insights-runtime-extractor-88dmg" (UID: "bad5c0b7-e451-4460-886d-d43b45404633") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:37:04.398779 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:04.398738    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bad5c0b7-e451-4460-886d-d43b45404633-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:37:05.342789 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:05.342722    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mf5pv" event={"ID":"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9","Type":"ContainerStarted","Data":"336afcd32588b51db433733f04a66fdff70547d55bd246e4dde3fc76a0024d13"}
Jul 24 15:37:05.342922 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:05.342809    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mf5pv" event={"ID":"68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9","Type":"ContainerStarted","Data":"f9b3873783c53178b195393e1eea7f83e5f825469eddd3d657b0f555c0edbdbd"}
Jul 24 15:37:05.342922 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:05.342857    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:37:05.361911 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:05.361850    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mf5pv" podStartSLOduration=16.848618003 podStartE2EDuration="19.361834751s" totalImagesPullingTime="2.513216748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:36:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:05.360831207 +0000 UTC m=+53.771541824" watchObservedRunningTime="2026-07-24 15:37:05.361834751 +0000 UTC m=+53.772545367"
Jul 24 15:37:13.270150 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:13.270122    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-th7v2"
Jul 24 15:37:14.066978 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.066952    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw"]
Jul 24 15:37:14.069780 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.069758    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw"
Jul 24 15:37:14.072137 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.072117    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 24 15:37:14.072250 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.072158    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 24 15:37:14.072250 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.072164    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 24 15:37:14.072392 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.072378    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 24 15:37:14.079019 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.079002    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"]
Jul 24 15:37:14.080842 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.080824    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"]
Jul 24 15:37:14.081011 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.080994    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.084173 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.084154    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 24 15:37:14.084367 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.084179    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 24 15:37:14.084367 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.084186    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 24 15:37:14.084367 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.084280    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 24 15:37:14.089791 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.089770    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw"]
Jul 24 15:37:14.096366 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.096345    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"]
Jul 24 15:37:14.161582 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.161539    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-697f546964-d25bc"]
Jul 24 15:37:14.161785 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.161729    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-ca\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.161785 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.161758    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-727c9\" (UniqueName: \"kubernetes.io/projected/d41b0032-da60-48c7-8a07-6be2b85965fc-kube-api-access-727c9\") pod \"klusterlet-addon-workmgr-6b4fc98fc-4zfhw\" (UID: \"d41b0032-da60-48c7-8a07-6be2b85965fc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw"
Jul 24 15:37:14.161914 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.161796    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mljl\" (UniqueName: \"kubernetes.io/projected/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-kube-api-access-8mljl\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.161914 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.161895    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d41b0032-da60-48c7-8a07-6be2b85965fc-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b4fc98fc-4zfhw\" (UID: \"d41b0032-da60-48c7-8a07-6be2b85965fc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw"
Jul 24 15:37:14.162023 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.161930    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.162023 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.161964    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d41b0032-da60-48c7-8a07-6be2b85965fc-tmp\") pod \"klusterlet-addon-workmgr-6b4fc98fc-4zfhw\" (UID: \"d41b0032-da60-48c7-8a07-6be2b85965fc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw"
Jul 24 15:37:14.162023 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.161995    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-hub\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.162202 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.162031    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.162202 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.162072    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.163594 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.163579    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.179826 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.179804    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-697f546964-d25bc"]
Jul 24 15:37:14.263175 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263143    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/97b984fd-3eae-4d5c-8ea2-73573fa71c96-installation-pull-secrets\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.263381 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263194    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/97b984fd-3eae-4d5c-8ea2-73573fa71c96-ca-trust-extracted\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.263381 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263221    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d41b0032-da60-48c7-8a07-6be2b85965fc-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b4fc98fc-4zfhw\" (UID: \"d41b0032-da60-48c7-8a07-6be2b85965fc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw"
Jul 24 15:37:14.263381 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263269    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.263381 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263311    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d41b0032-da60-48c7-8a07-6be2b85965fc-tmp\") pod \"klusterlet-addon-workmgr-6b4fc98fc-4zfhw\" (UID: \"d41b0032-da60-48c7-8a07-6be2b85965fc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw"
Jul 24 15:37:14.263381 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263339    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-hub\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.263381 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263365    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/97b984fd-3eae-4d5c-8ea2-73573fa71c96-image-registry-private-configuration\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.263705 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263410    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.263705 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263463    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.263705 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263500    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/97b984fd-3eae-4d5c-8ea2-73573fa71c96-registry-certificates\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.263705 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263628    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97b984fd-3eae-4d5c-8ea2-73573fa71c96-registry-tls\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.263705 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263690    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-ca\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.264013 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263718    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97b984fd-3eae-4d5c-8ea2-73573fa71c96-bound-sa-token\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.264013 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263751    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-727c9\" (UniqueName: \"kubernetes.io/projected/d41b0032-da60-48c7-8a07-6be2b85965fc-kube-api-access-727c9\") pod \"klusterlet-addon-workmgr-6b4fc98fc-4zfhw\" (UID: \"d41b0032-da60-48c7-8a07-6be2b85965fc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw"
Jul 24 15:37:14.264013 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263790    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/97b984fd-3eae-4d5c-8ea2-73573fa71c96-trusted-ca\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.264013 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263848    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mljl\" (UniqueName: \"kubernetes.io/projected/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-kube-api-access-8mljl\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.264013 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263877    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n85jx\" (UniqueName: \"kubernetes.io/projected/97b984fd-3eae-4d5c-8ea2-73573fa71c96-kube-api-access-n85jx\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.264013 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.263750    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d41b0032-da60-48c7-8a07-6be2b85965fc-tmp\") pod \"klusterlet-addon-workmgr-6b4fc98fc-4zfhw\" (UID: \"d41b0032-da60-48c7-8a07-6be2b85965fc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw"
Jul 24 15:37:14.264300 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.264077    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.266093 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.266067    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-ca\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.266222 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.266198    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-hub\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.266306 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.266291    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.266524 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.266505    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d41b0032-da60-48c7-8a07-6be2b85965fc-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b4fc98fc-4zfhw\" (UID: \"d41b0032-da60-48c7-8a07-6be2b85965fc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw"
Jul 24 15:37:14.266831 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.266814    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.275720 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.275696    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mljl\" (UniqueName: \"kubernetes.io/projected/ff05150e-efd3-4c2c-83e6-f39d4a1ceff7-kube-api-access-8mljl\") pod \"cluster-proxy-proxy-agent-848ddb7f68-rjjmh\" (UID: \"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.276137 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.276116    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-727c9\" (UniqueName: \"kubernetes.io/projected/d41b0032-da60-48c7-8a07-6be2b85965fc-kube-api-access-727c9\") pod \"klusterlet-addon-workmgr-6b4fc98fc-4zfhw\" (UID: \"d41b0032-da60-48c7-8a07-6be2b85965fc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw"
Jul 24 15:37:14.364843 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.364762    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/97b984fd-3eae-4d5c-8ea2-73573fa71c96-registry-certificates\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.364843 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.364810    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97b984fd-3eae-4d5c-8ea2-73573fa71c96-registry-tls\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.364843 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.364832    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97b984fd-3eae-4d5c-8ea2-73573fa71c96-bound-sa-token\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.365096 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.364895    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/97b984fd-3eae-4d5c-8ea2-73573fa71c96-trusted-ca\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.365096 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.364974    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n85jx\" (UniqueName: \"kubernetes.io/projected/97b984fd-3eae-4d5c-8ea2-73573fa71c96-kube-api-access-n85jx\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.365096 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.365009    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/97b984fd-3eae-4d5c-8ea2-73573fa71c96-installation-pull-secrets\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.365096 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.365059    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/97b984fd-3eae-4d5c-8ea2-73573fa71c96-ca-trust-extracted\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.365282 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.365197    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/97b984fd-3eae-4d5c-8ea2-73573fa71c96-image-registry-private-configuration\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.365494 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.365463    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/97b984fd-3eae-4d5c-8ea2-73573fa71c96-ca-trust-extracted\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.365848 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.365800    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/97b984fd-3eae-4d5c-8ea2-73573fa71c96-registry-certificates\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.366129 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.366007    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/97b984fd-3eae-4d5c-8ea2-73573fa71c96-trusted-ca\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.367902 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.367883    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/97b984fd-3eae-4d5c-8ea2-73573fa71c96-installation-pull-secrets\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.367902 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.367876    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/97b984fd-3eae-4d5c-8ea2-73573fa71c96-registry-tls\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.368075 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.368051    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/97b984fd-3eae-4d5c-8ea2-73573fa71c96-image-registry-private-configuration\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.376433 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.376413    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97b984fd-3eae-4d5c-8ea2-73573fa71c96-bound-sa-token\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.376536 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.376468    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n85jx\" (UniqueName: \"kubernetes.io/projected/97b984fd-3eae-4d5c-8ea2-73573fa71c96-kube-api-access-n85jx\") pod \"image-registry-697f546964-d25bc\" (UID: \"97b984fd-3eae-4d5c-8ea2-73573fa71c96\") " pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.379331 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.379311    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw"
Jul 24 15:37:14.397232 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.397209    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"
Jul 24 15:37:14.472048 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.471987    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:14.524027 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.522116    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw"]
Jul 24 15:37:14.525691 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:37:14.525629    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd41b0032_da60_48c7_8a07_6be2b85965fc.slice/crio-eee63090c54a9ae31ba3ac235aea5a05456771d61eac037369140661604fcdf1 WatchSource:0}: Error finding container eee63090c54a9ae31ba3ac235aea5a05456771d61eac037369140661604fcdf1: Status 404 returned error can't find the container with id eee63090c54a9ae31ba3ac235aea5a05456771d61eac037369140661604fcdf1
Jul 24 15:37:14.552681 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.552632    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh"]
Jul 24 15:37:14.555528 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:37:14.555506    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff05150e_efd3_4c2c_83e6_f39d4a1ceff7.slice/crio-92e49cc930578caa1f3c7072a70435333c831af39c8a2a5755795ea04c37fef7 WatchSource:0}: Error finding container 92e49cc930578caa1f3c7072a70435333c831af39c8a2a5755795ea04c37fef7: Status 404 returned error can't find the container with id 92e49cc930578caa1f3c7072a70435333c831af39c8a2a5755795ea04c37fef7
Jul 24 15:37:14.613469 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:14.613436    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-697f546964-d25bc"]
Jul 24 15:37:14.616497 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:37:14.616445    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97b984fd_3eae_4d5c_8ea2_73573fa71c96.slice/crio-159cc3193f43fd2616ed22f18fc863061f46b5a18e1da94e3ad3121d9e2948fe WatchSource:0}: Error finding container 159cc3193f43fd2616ed22f18fc863061f46b5a18e1da94e3ad3121d9e2948fe: Status 404 returned error can't find the container with id 159cc3193f43fd2616ed22f18fc863061f46b5a18e1da94e3ad3121d9e2948fe
Jul 24 15:37:15.349556 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:15.349515    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mf5pv"
Jul 24 15:37:15.384308 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:15.384265    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-697f546964-d25bc" event={"ID":"97b984fd-3eae-4d5c-8ea2-73573fa71c96","Type":"ContainerStarted","Data":"c23b7615662c83f6f634ce66241de3bd0fb508b323b356db489900ac7f8506a8"}
Jul 24 15:37:15.384308 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:15.384332    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-697f546964-d25bc" event={"ID":"97b984fd-3eae-4d5c-8ea2-73573fa71c96","Type":"ContainerStarted","Data":"159cc3193f43fd2616ed22f18fc863061f46b5a18e1da94e3ad3121d9e2948fe"}
Jul 24 15:37:15.384571 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:15.384416    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:15.386197 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:15.386124    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh" event={"ID":"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7","Type":"ContainerStarted","Data":"92e49cc930578caa1f3c7072a70435333c831af39c8a2a5755795ea04c37fef7"}
Jul 24 15:37:15.387691 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:15.387646    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw" event={"ID":"d41b0032-da60-48c7-8a07-6be2b85965fc","Type":"ContainerStarted","Data":"eee63090c54a9ae31ba3ac235aea5a05456771d61eac037369140661604fcdf1"}
Jul 24 15:37:15.418224 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:15.418167    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-697f546964-d25bc" podStartSLOduration=1.418148272 podStartE2EDuration="1.418148272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:37:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:15.417158467 +0000 UTC m=+63.827869084" watchObservedRunningTime="2026-07-24 15:37:15.418148272 +0000 UTC m=+63.828858889"
Jul 24 15:37:17.692196 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:17.692153    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret\") pod \"global-pull-secret-syncer-j8fqw\" (UID: \"94e9b736-6944-4781-9b40-500ec82d10e2\") " pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:37:17.692711 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:17.692219    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jngr\" (UniqueName: \"kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr\") pod \"network-check-target-qrsfq\" (UID: \"ac6f7393-c091-44d0-9e64-4b58651a24b8\") " pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:37:17.694883 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:17.694854    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 15:37:17.694999 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:17.694910    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 15:37:17.705275 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:17.705099    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 15:37:17.705759 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:17.705731    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/94e9b736-6944-4781-9b40-500ec82d10e2-original-pull-secret\") pod \"global-pull-secret-syncer-j8fqw\" (UID: \"94e9b736-6944-4781-9b40-500ec82d10e2\") " pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:37:17.717367 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:17.717341    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jngr\" (UniqueName: \"kubernetes.io/projected/ac6f7393-c091-44d0-9e64-4b58651a24b8-kube-api-access-6jngr\") pod \"network-check-target-qrsfq\" (UID: \"ac6f7393-c091-44d0-9e64-4b58651a24b8\") " pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:37:17.793515 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:17.793483    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs\") pod \"network-metrics-daemon-vmnkg\" (UID: \"f1958e1f-5868-4a15-847c-28d61994d060\") " pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:37:17.796054 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:17.796032    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 15:37:17.806449 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:17.806430    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1958e1f-5868-4a15-847c-28d61994d060-metrics-certs\") pod \"network-metrics-daemon-vmnkg\" (UID: \"f1958e1f-5868-4a15-847c-28d61994d060\") " pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:37:17.816493 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:17.816472    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-j8fqw"
Jul 24 15:37:17.823778 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:17.823757    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t6g6g\""
Jul 24 15:37:17.831726 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:17.831709    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:37:18.013208 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:18.013172    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w77kg\""
Jul 24 15:37:18.021017 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:18.020990    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vmnkg"
Jul 24 15:37:20.047087 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.046967    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-j8fqw"]
Jul 24 15:37:20.052015 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:37:20.051952    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94e9b736_6944_4781_9b40_500ec82d10e2.slice/crio-bf5130b77d1fce766ffd3ca4deaddc06934c44dbcefb35271997693670195b95 WatchSource:0}: Error finding container bf5130b77d1fce766ffd3ca4deaddc06934c44dbcefb35271997693670195b95: Status 404 returned error can't find the container with id bf5130b77d1fce766ffd3ca4deaddc06934c44dbcefb35271997693670195b95
Jul 24 15:37:20.071907 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.071880    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vmnkg"]
Jul 24 15:37:20.074105 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.074084    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qrsfq"]
Jul 24 15:37:20.075434 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:37:20.075396    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1958e1f_5868_4a15_847c_28d61994d060.slice/crio-4a5fa6d41321d54fd562ca1a5f01590cfd1027c1185c2dec77cb316f8072133a WatchSource:0}: Error finding container 4a5fa6d41321d54fd562ca1a5f01590cfd1027c1185c2dec77cb316f8072133a: Status 404 returned error can't find the container with id 4a5fa6d41321d54fd562ca1a5f01590cfd1027c1185c2dec77cb316f8072133a
Jul 24 15:37:20.077487 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:37:20.077352    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac6f7393_c091_44d0_9e64_4b58651a24b8.slice/crio-456c4ec2b3d8b936cfebd479171f86b2b5c458e8575bf6d79c153a2089f39c34 WatchSource:0}: Error finding container 456c4ec2b3d8b936cfebd479171f86b2b5c458e8575bf6d79c153a2089f39c34: Status 404 returned error can't find the container with id 456c4ec2b3d8b936cfebd479171f86b2b5c458e8575bf6d79c153a2089f39c34
Jul 24 15:37:20.409315 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.409280    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qrsfq" event={"ID":"ac6f7393-c091-44d0-9e64-4b58651a24b8","Type":"ContainerStarted","Data":"456c4ec2b3d8b936cfebd479171f86b2b5c458e8575bf6d79c153a2089f39c34"}
Jul 24 15:37:20.410618 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.410591    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vmnkg" event={"ID":"f1958e1f-5868-4a15-847c-28d61994d060","Type":"ContainerStarted","Data":"4a5fa6d41321d54fd562ca1a5f01590cfd1027c1185c2dec77cb316f8072133a"}
Jul 24 15:37:20.412640 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.412617    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh" event={"ID":"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7","Type":"ContainerStarted","Data":"6c6854905ff4b10b0bf131230821f512e46fb9c4e6274602d19558a72eca0b4f"}
Jul 24 15:37:20.412756 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.412685    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh" event={"ID":"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7","Type":"ContainerStarted","Data":"17a813c2cffc5248635ef062eb2810a100197bb3ba3cf6a550447c07b0c155d4"}
Jul 24 15:37:20.412756 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.412704    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh" event={"ID":"ff05150e-efd3-4c2c-83e6-f39d4a1ceff7","Type":"ContainerStarted","Data":"157c93e0becd401d0f3a287e75b0d43df801d60637dfda60afd489e9c9c4506d"}
Jul 24 15:37:20.414140 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.414108    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw" event={"ID":"d41b0032-da60-48c7-8a07-6be2b85965fc","Type":"ContainerStarted","Data":"009bde3a31184a7715c56cc8a9747be368da5c16258be01ce7ea7a134f488641"}
Jul 24 15:37:20.414305 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.414290    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw"
Jul 24 15:37:20.415386 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.415353    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j8fqw" event={"ID":"94e9b736-6944-4781-9b40-500ec82d10e2","Type":"ContainerStarted","Data":"bf5130b77d1fce766ffd3ca4deaddc06934c44dbcefb35271997693670195b95"}
Jul 24 15:37:20.416307 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.416292    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw"
Jul 24 15:37:20.423069 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.423050    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:37:20.425303 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.425286    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bad5c0b7-e451-4460-886d-d43b45404633-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-88dmg\" (UID: \"bad5c0b7-e451-4460-886d-d43b45404633\") " pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:37:20.432422 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.432379    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-848ddb7f68-rjjmh" podStartSLOduration=1.074317745 podStartE2EDuration="6.432365381s" totalImagesPullingTime="5.358047636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:37:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:20.431772582 +0000 UTC m=+68.842483199" watchObservedRunningTime="2026-07-24 15:37:20.432365381 +0000 UTC m=+68.843075998"
Jul 24 15:37:20.447066 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.447019    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b4fc98fc-4zfhw" podStartSLOduration=1.075081318 podStartE2EDuration="6.44700435s" totalImagesPullingTime="5.371923032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:37:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:20.446521919 +0000 UTC m=+68.857232539" watchObservedRunningTime="2026-07-24 15:37:20.44700435 +0000 UTC m=+68.857714968"
Jul 24 15:37:20.629643 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.629599    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9pmnt\""
Jul 24 15:37:20.637773 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.637505    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-88dmg"
Jul 24 15:37:20.800426 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:20.799112    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-88dmg"]
Jul 24 15:37:20.804256 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:37:20.804228    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbad5c0b7_e451_4460_886d_d43b45404633.slice/crio-491de63c021458ddcb79bbbf7df8ca927f06f7681e5682113cc99170ae649a3d WatchSource:0}: Error finding container 491de63c021458ddcb79bbbf7df8ca927f06f7681e5682113cc99170ae649a3d: Status 404 returned error can't find the container with id 491de63c021458ddcb79bbbf7df8ca927f06f7681e5682113cc99170ae649a3d
Jul 24 15:37:21.426514 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:21.426405    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-88dmg" event={"ID":"bad5c0b7-e451-4460-886d-d43b45404633","Type":"ContainerStarted","Data":"491de63c021458ddcb79bbbf7df8ca927f06f7681e5682113cc99170ae649a3d"}
Jul 24 15:37:22.435201 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:22.435088    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vmnkg" event={"ID":"f1958e1f-5868-4a15-847c-28d61994d060","Type":"ContainerStarted","Data":"191ce2ad0eefeeaaa857c4fb9cfba347ed99bcade4aa2f503a685804da2bb6e5"}
Jul 24 15:37:22.435201 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:22.435144    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vmnkg" event={"ID":"f1958e1f-5868-4a15-847c-28d61994d060","Type":"ContainerStarted","Data":"ca4e1b8985cd196e8b3fa0c953e5f98d29d0b9adedac8498f1739ad1849b4534"}
Jul 24 15:37:22.449922 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:22.449861    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vmnkg" podStartSLOduration=69.179236279 podStartE2EDuration="1m10.449845156s" totalImagesPullingTime="1.270608877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:22.449354 +0000 UTC m=+70.860064616" watchObservedRunningTime="2026-07-24 15:37:22.449845156 +0000 UTC m=+70.860555786"
Jul 24 15:37:24.087532 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:24.087506    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:37:27.372166 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.372130    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-t2hxv"]
Jul 24 15:37:27.406045 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.405275    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.408270 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.407565    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9cbmn\""
Jul 24 15:37:27.408270 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.407876    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 15:37:27.408270 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.407981    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 15:37:27.408270 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.408103    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 15:37:27.408609 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.408517    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 15:37:27.409009 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.408756    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 15:37:27.409009 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.408804    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 15:37:27.476506 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.476471    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/68bec78f-2086-4308-bb27-d18cb6b8c3fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.476681 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.476543    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68bec78f-2086-4308-bb27-d18cb6b8c3fd-metrics-client-ca\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.476681 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.476575    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68bec78f-2086-4308-bb27-d18cb6b8c3fd-sys\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.476681 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.476601    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68bec78f-2086-4308-bb27-d18cb6b8c3fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.476854 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.476739    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/68bec78f-2086-4308-bb27-d18cb6b8c3fd-node-exporter-textfile\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.476854 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.476772    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snkrj\" (UniqueName: \"kubernetes.io/projected/68bec78f-2086-4308-bb27-d18cb6b8c3fd-kube-api-access-snkrj\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.476854 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.476804    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68bec78f-2086-4308-bb27-d18cb6b8c3fd-node-exporter-wtmp\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.476976 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.476891    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68bec78f-2086-4308-bb27-d18cb6b8c3fd-root\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.476976 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.476926    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68bec78f-2086-4308-bb27-d18cb6b8c3fd-node-exporter-tls\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.578256 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.578226    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/68bec78f-2086-4308-bb27-d18cb6b8c3fd-node-exporter-textfile\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.578376 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.578269    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snkrj\" (UniqueName: \"kubernetes.io/projected/68bec78f-2086-4308-bb27-d18cb6b8c3fd-kube-api-access-snkrj\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.578376 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.578298    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68bec78f-2086-4308-bb27-d18cb6b8c3fd-node-exporter-wtmp\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.578376 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.578335    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68bec78f-2086-4308-bb27-d18cb6b8c3fd-root\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.578376 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.578364    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68bec78f-2086-4308-bb27-d18cb6b8c3fd-node-exporter-tls\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.578614 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.578392    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/68bec78f-2086-4308-bb27-d18cb6b8c3fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.578614 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.578427    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68bec78f-2086-4308-bb27-d18cb6b8c3fd-metrics-client-ca\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.578614 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.578450    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68bec78f-2086-4308-bb27-d18cb6b8c3fd-sys\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.578614 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.578473    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68bec78f-2086-4308-bb27-d18cb6b8c3fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.578614 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.578605    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/68bec78f-2086-4308-bb27-d18cb6b8c3fd-node-exporter-textfile\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.578882 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.578806    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68bec78f-2086-4308-bb27-d18cb6b8c3fd-node-exporter-wtmp\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.578882 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.578858    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68bec78f-2086-4308-bb27-d18cb6b8c3fd-root\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.579034 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.578995    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68bec78f-2086-4308-bb27-d18cb6b8c3fd-sys\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.579265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.579244    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68bec78f-2086-4308-bb27-d18cb6b8c3fd-metrics-client-ca\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.579363 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.579340    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/68bec78f-2086-4308-bb27-d18cb6b8c3fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.581461 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.581433    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68bec78f-2086-4308-bb27-d18cb6b8c3fd-node-exporter-tls\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.582165 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.582147    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68bec78f-2086-4308-bb27-d18cb6b8c3fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.586191 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.586167    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snkrj\" (UniqueName: \"kubernetes.io/projected/68bec78f-2086-4308-bb27-d18cb6b8c3fd-kube-api-access-snkrj\") pod \"node-exporter-t2hxv\" (UID: \"68bec78f-2086-4308-bb27-d18cb6b8c3fd\") " pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:27.718228 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:27.718188    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t2hxv"
Jul 24 15:37:28.463468 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:28.463430    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-j8fqw" event={"ID":"94e9b736-6944-4781-9b40-500ec82d10e2","Type":"ContainerStarted","Data":"3577235644aeb0a1c698e296e24b3262b3d410cfab42508064a2f06ff5b69f0b"}
Jul 24 15:37:28.464887 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:28.464853    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t2hxv" event={"ID":"68bec78f-2086-4308-bb27-d18cb6b8c3fd","Type":"ContainerStarted","Data":"9ef9b32e29697009ca60cbaba90f567f11adbaccd14fad0651c878415a0cba6b"}
Jul 24 15:37:28.466392 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:28.466367    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qrsfq" event={"ID":"ac6f7393-c091-44d0-9e64-4b58651a24b8","Type":"ContainerStarted","Data":"14a46fc337663ed601b35c5b97f3e9bd39b4cc1be066949b8e5908b2fd71a5ba"}
Jul 24 15:37:28.466519 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:28.466503    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:37:28.468020 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:28.467989    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-88dmg" event={"ID":"bad5c0b7-e451-4460-886d-d43b45404633","Type":"ContainerStarted","Data":"99c9a4b3893e6cda2352e74984dff0cd88553c19a60408ae5d857c683903ddd4"}
Jul 24 15:37:28.479513 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:28.479472    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-j8fqw" podStartSLOduration=68.271825221 podStartE2EDuration="1m16.479459821s" totalImagesPullingTime="8.2076346s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:28.478542737 +0000 UTC m=+76.889253355" watchObservedRunningTime="2026-07-24 15:37:28.479459821 +0000 UTC m=+76.890170477"
Jul 24 15:37:28.494347 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:28.494306    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qrsfq" podStartSLOduration=68.312019302 podStartE2EDuration="1m16.49429362s" totalImagesPullingTime="8.182274318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:28.493016971 +0000 UTC m=+76.903727611" watchObservedRunningTime="2026-07-24 15:37:28.49429362 +0000 UTC m=+76.905004236"
Jul 24 15:37:29.474100 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:29.474066    2561 generic.go:356] "Generic (PLEG): container finished" podID="68bec78f-2086-4308-bb27-d18cb6b8c3fd" containerID="c729cf92d20ed46a9e2269966edf6f2245c8d22cd86006f71564401e16982d4f" exitCode=0
Jul 24 15:37:29.475617 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:29.475033    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t2hxv" event={"ID":"68bec78f-2086-4308-bb27-d18cb6b8c3fd","Type":"ContainerDied","Data":"c729cf92d20ed46a9e2269966edf6f2245c8d22cd86006f71564401e16982d4f"}
Jul 24 15:37:30.480328 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:30.480288    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t2hxv" event={"ID":"68bec78f-2086-4308-bb27-d18cb6b8c3fd","Type":"ContainerStarted","Data":"3977b620b77afdd8045b060ec1d74e47600ad13614151a078a9f26c40eb7853d"}
Jul 24 15:37:30.480853 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:30.480358    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t2hxv" event={"ID":"68bec78f-2086-4308-bb27-d18cb6b8c3fd","Type":"ContainerStarted","Data":"2e67b95895345579ae627c1949666f72cc628138530e2793200d1abede78ae3f"}
Jul 24 15:37:30.508865 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:30.508807    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-t2hxv" podStartSLOduration=2.3983976289999998 podStartE2EDuration="3.508790691s" totalImagesPullingTime="1.031756465s" totalInitContainerRuntime="78.636597ms" isStatefulPod=true podCreationTimestamp="2026-07-24 15:37:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:30.501455148 +0000 UTC m=+78.912165796" watchObservedRunningTime="2026-07-24 15:37:30.508790691 +0000 UTC m=+78.919501308"
Jul 24 15:37:32.492761 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:32.492723    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-88dmg" event={"ID":"bad5c0b7-e451-4460-886d-d43b45404633","Type":"ContainerStarted","Data":"946dd6d15af42ec5ea45a8390e6e0697d21b58cc55c1843e6e77a4101a1c3da1"}
Jul 24 15:37:35.506489 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:35.506453    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-88dmg" event={"ID":"bad5c0b7-e451-4460-886d-d43b45404633","Type":"ContainerStarted","Data":"106b4b0314359405eef059fda040a1ddd4ded4b1695b3e5277643b55bb7b74f3"}
Jul 24 15:37:35.525911 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:35.525859    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-88dmg" podStartSLOduration=33.161987343 podStartE2EDuration="47.5258457s" totalImagesPullingTime="14.363858357s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:36:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:37:35.524036252 +0000 UTC m=+83.934746869" watchObservedRunningTime="2026-07-24 15:37:35.5258457 +0000 UTC m=+83.936556317"
Jul 24 15:37:36.396109 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:36.396082    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-697f546964-d25bc"
Jul 24 15:37:39.101356 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.101280    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd" podUID="d88d473b-18bf-4f9d-98ef-518843c19345" containerName="registry" containerID="cri-o://6a688cd30f0de898100ab9839fee95da0665d5f020011fa645ceb78b873d9e5c" gracePeriod=30
Jul 24 15:37:39.349173 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.349151    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:37:39.471813 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.471790    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d88d473b-18bf-4f9d-98ef-518843c19345-registry-certificates\") pod \"d88d473b-18bf-4f9d-98ef-518843c19345\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") "
Jul 24 15:37:39.471922 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.471826    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d88d473b-18bf-4f9d-98ef-518843c19345-installation-pull-secrets\") pod \"d88d473b-18bf-4f9d-98ef-518843c19345\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") "
Jul 24 15:37:39.471922 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.471858    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-bound-sa-token\") pod \"d88d473b-18bf-4f9d-98ef-518843c19345\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") "
Jul 24 15:37:39.471922 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.471890    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d88d473b-18bf-4f9d-98ef-518843c19345-trusted-ca\") pod \"d88d473b-18bf-4f9d-98ef-518843c19345\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") "
Jul 24 15:37:39.471922 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.471917    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9bds\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-kube-api-access-l9bds\") pod \"d88d473b-18bf-4f9d-98ef-518843c19345\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") "
Jul 24 15:37:39.472095 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.471951    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d88d473b-18bf-4f9d-98ef-518843c19345-image-registry-private-configuration\") pod \"d88d473b-18bf-4f9d-98ef-518843c19345\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") "
Jul 24 15:37:39.472095 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.471990    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d88d473b-18bf-4f9d-98ef-518843c19345-ca-trust-extracted\") pod \"d88d473b-18bf-4f9d-98ef-518843c19345\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") "
Jul 24 15:37:39.472095 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.472014    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls\") pod \"d88d473b-18bf-4f9d-98ef-518843c19345\" (UID: \"d88d473b-18bf-4f9d-98ef-518843c19345\") "
Jul 24 15:37:39.472244 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.472221    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d88d473b-18bf-4f9d-98ef-518843c19345-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d88d473b-18bf-4f9d-98ef-518843c19345" (UID: "d88d473b-18bf-4f9d-98ef-518843c19345"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:37:39.472411 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.472389    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d88d473b-18bf-4f9d-98ef-518843c19345-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d88d473b-18bf-4f9d-98ef-518843c19345" (UID: "d88d473b-18bf-4f9d-98ef-518843c19345"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:37:39.474468 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.474444    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d88d473b-18bf-4f9d-98ef-518843c19345" (UID: "d88d473b-18bf-4f9d-98ef-518843c19345"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:37:39.474575 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.474547    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d88d473b-18bf-4f9d-98ef-518843c19345-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d88d473b-18bf-4f9d-98ef-518843c19345" (UID: "d88d473b-18bf-4f9d-98ef-518843c19345"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:37:39.474697 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.474655    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d88d473b-18bf-4f9d-98ef-518843c19345-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d88d473b-18bf-4f9d-98ef-518843c19345" (UID: "d88d473b-18bf-4f9d-98ef-518843c19345"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:37:39.474872 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.474851    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d88d473b-18bf-4f9d-98ef-518843c19345" (UID: "d88d473b-18bf-4f9d-98ef-518843c19345"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:37:39.474993 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.474972    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-kube-api-access-l9bds" (OuterVolumeSpecName: "kube-api-access-l9bds") pod "d88d473b-18bf-4f9d-98ef-518843c19345" (UID: "d88d473b-18bf-4f9d-98ef-518843c19345"). InnerVolumeSpecName "kube-api-access-l9bds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:37:39.481579 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.481554    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d88d473b-18bf-4f9d-98ef-518843c19345-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d88d473b-18bf-4f9d-98ef-518843c19345" (UID: "d88d473b-18bf-4f9d-98ef-518843c19345"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:37:39.521994 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.521970    2561 generic.go:356] "Generic (PLEG): container finished" podID="d88d473b-18bf-4f9d-98ef-518843c19345" containerID="6a688cd30f0de898100ab9839fee95da0665d5f020011fa645ceb78b873d9e5c" exitCode=0
Jul 24 15:37:39.522091 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.522016    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"
Jul 24 15:37:39.522091 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.522041    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd" event={"ID":"d88d473b-18bf-4f9d-98ef-518843c19345","Type":"ContainerDied","Data":"6a688cd30f0de898100ab9839fee95da0665d5f020011fa645ceb78b873d9e5c"}
Jul 24 15:37:39.522091 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.522079    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65f5d4b4c6-jnhnd" event={"ID":"d88d473b-18bf-4f9d-98ef-518843c19345","Type":"ContainerDied","Data":"1098ecde048abccbabbbf892d9723608a5e8b7cc8daf7d66fdb864510328e76a"}
Jul 24 15:37:39.522207 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.522099    2561 scope.go:117] "RemoveContainer" containerID="6a688cd30f0de898100ab9839fee95da0665d5f020011fa645ceb78b873d9e5c"
Jul 24 15:37:39.530926 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.530907    2561 scope.go:117] "RemoveContainer" containerID="6a688cd30f0de898100ab9839fee95da0665d5f020011fa645ceb78b873d9e5c"
Jul 24 15:37:39.531178 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:37:39.531157    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a688cd30f0de898100ab9839fee95da0665d5f020011fa645ceb78b873d9e5c\": container with ID starting with 6a688cd30f0de898100ab9839fee95da0665d5f020011fa645ceb78b873d9e5c not found: ID does not exist" containerID="6a688cd30f0de898100ab9839fee95da0665d5f020011fa645ceb78b873d9e5c"
Jul 24 15:37:39.531239 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.531184    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a688cd30f0de898100ab9839fee95da0665d5f020011fa645ceb78b873d9e5c"} err="failed to get container status \"6a688cd30f0de898100ab9839fee95da0665d5f020011fa645ceb78b873d9e5c\": rpc error: code = NotFound desc = could not find container \"6a688cd30f0de898100ab9839fee95da0665d5f020011fa645ceb78b873d9e5c\": container with ID starting with 6a688cd30f0de898100ab9839fee95da0665d5f020011fa645ceb78b873d9e5c not found: ID does not exist"
Jul 24 15:37:39.542848 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.542829    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"]
Jul 24 15:37:39.546493 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.546472    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-65f5d4b4c6-jnhnd"]
Jul 24 15:37:39.573287 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.573266    2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d88d473b-18bf-4f9d-98ef-518843c19345-image-registry-private-configuration\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:37:39.573391 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.573292    2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d88d473b-18bf-4f9d-98ef-518843c19345-ca-trust-extracted\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:37:39.573391 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.573308    2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-registry-tls\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:37:39.573391 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.573321    2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d88d473b-18bf-4f9d-98ef-518843c19345-registry-certificates\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:37:39.573391 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.573334    2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d88d473b-18bf-4f9d-98ef-518843c19345-installation-pull-secrets\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:37:39.573391 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.573348    2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-bound-sa-token\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:37:39.573391 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.573361    2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d88d473b-18bf-4f9d-98ef-518843c19345-trusted-ca\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:37:39.573391 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:39.573373    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9bds\" (UniqueName: \"kubernetes.io/projected/d88d473b-18bf-4f9d-98ef-518843c19345-kube-api-access-l9bds\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:37:40.105187 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:40.105151    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d88d473b-18bf-4f9d-98ef-518843c19345" path="/var/lib/kubelet/pods/d88d473b-18bf-4f9d-98ef-518843c19345/volumes"
Jul 24 15:37:59.478086 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:37:59.478058    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qrsfq"
Jul 24 15:40:51.643246 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:51.643163    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zgzsz"]
Jul 24 15:40:51.643640 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:51.643394    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d88d473b-18bf-4f9d-98ef-518843c19345" containerName="registry"
Jul 24 15:40:51.643640 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:51.643405    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d88d473b-18bf-4f9d-98ef-518843c19345" containerName="registry"
Jul 24 15:40:51.643640 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:51.643452    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d88d473b-18bf-4f9d-98ef-518843c19345" containerName="registry"
Jul 24 15:40:51.646169 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:51.646154    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zgzsz"
Jul 24 15:40:51.648633 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:51.648613    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 15:40:51.649522 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:51.649500    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 15:40:51.649623 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:51.649528    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-b6zbw\""
Jul 24 15:40:51.656725 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:51.656700    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zgzsz"]
Jul 24 15:40:51.721827 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:51.721774    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg2fh\" (UniqueName: \"kubernetes.io/projected/5cb16e01-2dfd-46f6-b147-e97092ed46d4-kube-api-access-gg2fh\") pod \"openshift-lws-operator-fd8ccff4c-zgzsz\" (UID: \"5cb16e01-2dfd-46f6-b147-e97092ed46d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zgzsz"
Jul 24 15:40:51.722009 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:51.721833    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5cb16e01-2dfd-46f6-b147-e97092ed46d4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zgzsz\" (UID: \"5cb16e01-2dfd-46f6-b147-e97092ed46d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zgzsz"
Jul 24 15:40:51.822713 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:51.822643    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg2fh\" (UniqueName: \"kubernetes.io/projected/5cb16e01-2dfd-46f6-b147-e97092ed46d4-kube-api-access-gg2fh\") pod \"openshift-lws-operator-fd8ccff4c-zgzsz\" (UID: \"5cb16e01-2dfd-46f6-b147-e97092ed46d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zgzsz"
Jul 24 15:40:51.822892 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:51.822728    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5cb16e01-2dfd-46f6-b147-e97092ed46d4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zgzsz\" (UID: \"5cb16e01-2dfd-46f6-b147-e97092ed46d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zgzsz"
Jul 24 15:40:51.823130 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:51.823111    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5cb16e01-2dfd-46f6-b147-e97092ed46d4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zgzsz\" (UID: \"5cb16e01-2dfd-46f6-b147-e97092ed46d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zgzsz"
Jul 24 15:40:51.832569 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:51.832528    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg2fh\" (UniqueName: \"kubernetes.io/projected/5cb16e01-2dfd-46f6-b147-e97092ed46d4-kube-api-access-gg2fh\") pod \"openshift-lws-operator-fd8ccff4c-zgzsz\" (UID: \"5cb16e01-2dfd-46f6-b147-e97092ed46d4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zgzsz"
Jul 24 15:40:51.954844 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:51.954812    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zgzsz"
Jul 24 15:40:52.093819 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:52.093622    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zgzsz"]
Jul 24 15:40:52.096553 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:40:52.096525    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5cb16e01_2dfd_46f6_b147_e97092ed46d4.slice/crio-a9d1d2a2e0eca49b4fc4b0590b851d078fbccd206d5b42b14561eefc055583d4 WatchSource:0}: Error finding container a9d1d2a2e0eca49b4fc4b0590b851d078fbccd206d5b42b14561eefc055583d4: Status 404 returned error can't find the container with id a9d1d2a2e0eca49b4fc4b0590b851d078fbccd206d5b42b14561eefc055583d4
Jul 24 15:40:52.185675 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:52.185633    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zgzsz" event={"ID":"5cb16e01-2dfd-46f6-b147-e97092ed46d4","Type":"ContainerStarted","Data":"a9d1d2a2e0eca49b4fc4b0590b851d078fbccd206d5b42b14561eefc055583d4"}
Jul 24 15:40:55.200590 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:55.200490    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zgzsz" event={"ID":"5cb16e01-2dfd-46f6-b147-e97092ed46d4","Type":"ContainerStarted","Data":"fa78142269cd9acfa08130804ea172befe1ca1ae0e674f8e9a7c07296ca3373f"}
Jul 24 15:40:55.217594 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:40:55.217531    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zgzsz" podStartSLOduration=1.373134361 podStartE2EDuration="4.2175129s" totalImagesPullingTime="2.844378539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:40:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:40:55.216044111 +0000 UTC m=+283.626754732" watchObservedRunningTime="2026-07-24 15:40:55.2175129 +0000 UTC m=+283.628223516"
Jul 24 15:41:10.825678 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:10.825635    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"]
Jul 24 15:41:10.829028 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:10.829005    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:10.831523 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:10.831500    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 15:41:10.831690 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:10.831523    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 15:41:10.831690 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:10.831554    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 15:41:10.832086 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:10.832071    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bk4ws\""
Jul 24 15:41:10.832311 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:10.832296    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 15:41:10.850873 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:10.850812    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"]
Jul 24 15:41:10.964729 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:10.964687    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/472a5da2-28a3-4250-9eb9-98f1ae8efa46-tmp\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-r9vps\" (UID: \"472a5da2-28a3-4250-9eb9-98f1ae8efa46\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:10.964729 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:10.964737    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/472a5da2-28a3-4250-9eb9-98f1ae8efa46-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-r9vps\" (UID: \"472a5da2-28a3-4250-9eb9-98f1ae8efa46\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:10.964960 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:10.964808    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/472a5da2-28a3-4250-9eb9-98f1ae8efa46-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-r9vps\" (UID: \"472a5da2-28a3-4250-9eb9-98f1ae8efa46\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:10.964960 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:10.964832    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/472a5da2-28a3-4250-9eb9-98f1ae8efa46-manifests\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-r9vps\" (UID: \"472a5da2-28a3-4250-9eb9-98f1ae8efa46\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:10.964960 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:10.964847    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7qwc\" (UniqueName: \"kubernetes.io/projected/472a5da2-28a3-4250-9eb9-98f1ae8efa46-kube-api-access-k7qwc\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-r9vps\" (UID: \"472a5da2-28a3-4250-9eb9-98f1ae8efa46\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:11.065857 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:11.065813    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/472a5da2-28a3-4250-9eb9-98f1ae8efa46-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-r9vps\" (UID: \"472a5da2-28a3-4250-9eb9-98f1ae8efa46\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:11.065857 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:11.065856    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/472a5da2-28a3-4250-9eb9-98f1ae8efa46-manifests\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-r9vps\" (UID: \"472a5da2-28a3-4250-9eb9-98f1ae8efa46\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:11.066134 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:11.065874    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7qwc\" (UniqueName: \"kubernetes.io/projected/472a5da2-28a3-4250-9eb9-98f1ae8efa46-kube-api-access-k7qwc\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-r9vps\" (UID: \"472a5da2-28a3-4250-9eb9-98f1ae8efa46\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:11.066134 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:11.066006    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/472a5da2-28a3-4250-9eb9-98f1ae8efa46-tmp\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-r9vps\" (UID: \"472a5da2-28a3-4250-9eb9-98f1ae8efa46\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:11.066134 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:11.066054    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/472a5da2-28a3-4250-9eb9-98f1ae8efa46-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-r9vps\" (UID: \"472a5da2-28a3-4250-9eb9-98f1ae8efa46\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:11.066314 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:11.066290    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/472a5da2-28a3-4250-9eb9-98f1ae8efa46-manifests\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-r9vps\" (UID: \"472a5da2-28a3-4250-9eb9-98f1ae8efa46\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:11.068374 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:11.068341    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/472a5da2-28a3-4250-9eb9-98f1ae8efa46-tmp\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-r9vps\" (UID: \"472a5da2-28a3-4250-9eb9-98f1ae8efa46\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:11.068578 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:11.068559    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/472a5da2-28a3-4250-9eb9-98f1ae8efa46-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-r9vps\" (UID: \"472a5da2-28a3-4250-9eb9-98f1ae8efa46\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:11.068688 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:11.068642    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/472a5da2-28a3-4250-9eb9-98f1ae8efa46-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-r9vps\" (UID: \"472a5da2-28a3-4250-9eb9-98f1ae8efa46\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:11.075758 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:11.075695    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7qwc\" (UniqueName: \"kubernetes.io/projected/472a5da2-28a3-4250-9eb9-98f1ae8efa46-kube-api-access-k7qwc\") pod \"opendatahub-operator-controller-manager-7b7484f4dc-r9vps\" (UID: \"472a5da2-28a3-4250-9eb9-98f1ae8efa46\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:11.139506 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:11.139448    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:11.285437 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:11.285407    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"]
Jul 24 15:41:11.288058 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:41:11.288030    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod472a5da2_28a3_4250_9eb9_98f1ae8efa46.slice/crio-41bed9bdf7b021bf19b07e274fab99ada7dee65279b374751aae3e5a8758c633 WatchSource:0}: Error finding container 41bed9bdf7b021bf19b07e274fab99ada7dee65279b374751aae3e5a8758c633: Status 404 returned error can't find the container with id 41bed9bdf7b021bf19b07e274fab99ada7dee65279b374751aae3e5a8758c633
Jul 24 15:41:11.903809 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:11.903779    2561 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 15:41:12.263823 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:12.263785    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps" event={"ID":"472a5da2-28a3-4250-9eb9-98f1ae8efa46","Type":"ContainerStarted","Data":"41bed9bdf7b021bf19b07e274fab99ada7dee65279b374751aae3e5a8758c633"}
Jul 24 15:41:15.277672 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:15.277630    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps" event={"ID":"472a5da2-28a3-4250-9eb9-98f1ae8efa46","Type":"ContainerStarted","Data":"35cf69d07edca20344e290a0fb3c31d4e397d867ef1b7704ad526a43bd412182"}
Jul 24 15:41:16.283060 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:16.283025    2561 generic.go:356] "Generic (PLEG): container finished" podID="472a5da2-28a3-4250-9eb9-98f1ae8efa46" containerID="35cf69d07edca20344e290a0fb3c31d4e397d867ef1b7704ad526a43bd412182" exitCode=0
Jul 24 15:41:16.283467 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:16.283117    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps" event={"ID":"472a5da2-28a3-4250-9eb9-98f1ae8efa46","Type":"ContainerDied","Data":"35cf69d07edca20344e290a0fb3c31d4e397d867ef1b7704ad526a43bd412182"}
Jul 24 15:41:16.283551 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:16.283537    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:41:17.289001 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:17.288963    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps" event={"ID":"472a5da2-28a3-4250-9eb9-98f1ae8efa46","Type":"ContainerStarted","Data":"b76e4a31afc239e8c735de04a80f2e269d577c31e070df87247cdf56d270546f"}
Jul 24 15:41:17.289451 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:17.289109    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:17.309543 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:17.309489    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps" podStartSLOduration=2.911316703 podStartE2EDuration="7.309472569s" totalImagesPullingTime="3.399217515s" totalInitContainerRuntime="998.938351ms" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:17.308645195 +0000 UTC m=+305.719355815" watchObservedRunningTime="2026-07-24 15:41:17.309472569 +0000 UTC m=+305.720183248"
Jul 24 15:41:28.295575 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:28.295545    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b7484f4dc-r9vps"
Jul 24 15:41:30.892554 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:30.892523    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-59fb5479dc-kcpmq"]
Jul 24 15:41:30.901078 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:30.901056    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59fb5479dc-kcpmq"
Jul 24 15:41:30.904179 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:30.904154    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 15:41:30.904426 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:30.904404    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 15:41:30.905078 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:30.905057    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9vt4j\""
Jul 24 15:41:30.905201 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:30.905076    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 24 15:41:30.905201 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:30.905063    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 15:41:30.926109 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:30.926076    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59fb5479dc-kcpmq"]
Jul 24 15:41:31.021699 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.021644    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ef6a247-e63f-4351-906b-479a8ddb2294-tmp\") pod \"kube-auth-proxy-59fb5479dc-kcpmq\" (UID: \"5ef6a247-e63f-4351-906b-479a8ddb2294\") " pod="openshift-ingress/kube-auth-proxy-59fb5479dc-kcpmq"
Jul 24 15:41:31.021882 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.021748    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ef6a247-e63f-4351-906b-479a8ddb2294-tls-certs\") pod \"kube-auth-proxy-59fb5479dc-kcpmq\" (UID: \"5ef6a247-e63f-4351-906b-479a8ddb2294\") " pod="openshift-ingress/kube-auth-proxy-59fb5479dc-kcpmq"
Jul 24 15:41:31.021882 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.021828    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sj78\" (UniqueName: \"kubernetes.io/projected/5ef6a247-e63f-4351-906b-479a8ddb2294-kube-api-access-9sj78\") pod \"kube-auth-proxy-59fb5479dc-kcpmq\" (UID: \"5ef6a247-e63f-4351-906b-479a8ddb2294\") " pod="openshift-ingress/kube-auth-proxy-59fb5479dc-kcpmq"
Jul 24 15:41:31.122793 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.122753    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sj78\" (UniqueName: \"kubernetes.io/projected/5ef6a247-e63f-4351-906b-479a8ddb2294-kube-api-access-9sj78\") pod \"kube-auth-proxy-59fb5479dc-kcpmq\" (UID: \"5ef6a247-e63f-4351-906b-479a8ddb2294\") " pod="openshift-ingress/kube-auth-proxy-59fb5479dc-kcpmq"
Jul 24 15:41:31.122793 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.122802    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ef6a247-e63f-4351-906b-479a8ddb2294-tmp\") pod \"kube-auth-proxy-59fb5479dc-kcpmq\" (UID: \"5ef6a247-e63f-4351-906b-479a8ddb2294\") " pod="openshift-ingress/kube-auth-proxy-59fb5479dc-kcpmq"
Jul 24 15:41:31.123051 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.122921    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ef6a247-e63f-4351-906b-479a8ddb2294-tls-certs\") pod \"kube-auth-proxy-59fb5479dc-kcpmq\" (UID: \"5ef6a247-e63f-4351-906b-479a8ddb2294\") " pod="openshift-ingress/kube-auth-proxy-59fb5479dc-kcpmq"
Jul 24 15:41:31.125250 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.125216    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ef6a247-e63f-4351-906b-479a8ddb2294-tmp\") pod \"kube-auth-proxy-59fb5479dc-kcpmq\" (UID: \"5ef6a247-e63f-4351-906b-479a8ddb2294\") " pod="openshift-ingress/kube-auth-proxy-59fb5479dc-kcpmq"
Jul 24 15:41:31.125514 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.125493    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ef6a247-e63f-4351-906b-479a8ddb2294-tls-certs\") pod \"kube-auth-proxy-59fb5479dc-kcpmq\" (UID: \"5ef6a247-e63f-4351-906b-479a8ddb2294\") " pod="openshift-ingress/kube-auth-proxy-59fb5479dc-kcpmq"
Jul 24 15:41:31.132199 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.132174    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sj78\" (UniqueName: \"kubernetes.io/projected/5ef6a247-e63f-4351-906b-479a8ddb2294-kube-api-access-9sj78\") pod \"kube-auth-proxy-59fb5479dc-kcpmq\" (UID: \"5ef6a247-e63f-4351-906b-479a8ddb2294\") " pod="openshift-ingress/kube-auth-proxy-59fb5479dc-kcpmq"
Jul 24 15:41:31.211144 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.211102    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-59fb5479dc-kcpmq"
Jul 24 15:41:31.365079 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.365031    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-59fb5479dc-kcpmq"]
Jul 24 15:41:31.367577 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:41:31.367547    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef6a247_e63f_4351_906b_479a8ddb2294.slice/crio-0379d5a90b1b2034d0d41c42ab0b686999e18a87a44aec27f7d4b2bc6625d381 WatchSource:0}: Error finding container 0379d5a90b1b2034d0d41c42ab0b686999e18a87a44aec27f7d4b2bc6625d381: Status 404 returned error can't find the container with id 0379d5a90b1b2034d0d41c42ab0b686999e18a87a44aec27f7d4b2bc6625d381
Jul 24 15:41:31.583976 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.583897    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gbdjq"]
Jul 24 15:41:31.588672 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.588635    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gbdjq"
Jul 24 15:41:31.591093 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.591073    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 24 15:41:31.603383 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.603343    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gbdjq"]
Jul 24 15:41:31.727294 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.727246    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc394fc0-3581-48d0-ba1e-9c70876d9a73-tls-certs\") pod \"model-serving-api-54d89bd5fc-gbdjq\" (UID: \"dc394fc0-3581-48d0-ba1e-9c70876d9a73\") " pod="opendatahub/model-serving-api-54d89bd5fc-gbdjq"
Jul 24 15:41:31.727471 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.727312    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n46bf\" (UniqueName: \"kubernetes.io/projected/dc394fc0-3581-48d0-ba1e-9c70876d9a73-kube-api-access-n46bf\") pod \"model-serving-api-54d89bd5fc-gbdjq\" (UID: \"dc394fc0-3581-48d0-ba1e-9c70876d9a73\") " pod="opendatahub/model-serving-api-54d89bd5fc-gbdjq"
Jul 24 15:41:31.827780 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.827749    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc394fc0-3581-48d0-ba1e-9c70876d9a73-tls-certs\") pod \"model-serving-api-54d89bd5fc-gbdjq\" (UID: \"dc394fc0-3581-48d0-ba1e-9c70876d9a73\") " pod="opendatahub/model-serving-api-54d89bd5fc-gbdjq"
Jul 24 15:41:31.827958 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.827819    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n46bf\" (UniqueName: \"kubernetes.io/projected/dc394fc0-3581-48d0-ba1e-9c70876d9a73-kube-api-access-n46bf\") pod \"model-serving-api-54d89bd5fc-gbdjq\" (UID: \"dc394fc0-3581-48d0-ba1e-9c70876d9a73\") " pod="opendatahub/model-serving-api-54d89bd5fc-gbdjq"
Jul 24 15:41:31.830243 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.830215    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc394fc0-3581-48d0-ba1e-9c70876d9a73-tls-certs\") pod \"model-serving-api-54d89bd5fc-gbdjq\" (UID: \"dc394fc0-3581-48d0-ba1e-9c70876d9a73\") " pod="opendatahub/model-serving-api-54d89bd5fc-gbdjq"
Jul 24 15:41:31.864304 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.864233    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n46bf\" (UniqueName: \"kubernetes.io/projected/dc394fc0-3581-48d0-ba1e-9c70876d9a73-kube-api-access-n46bf\") pod \"model-serving-api-54d89bd5fc-gbdjq\" (UID: \"dc394fc0-3581-48d0-ba1e-9c70876d9a73\") " pod="opendatahub/model-serving-api-54d89bd5fc-gbdjq"
Jul 24 15:41:31.899709 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:31.899679    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gbdjq"
Jul 24 15:41:32.065479 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.065456    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gbdjq"]
Jul 24 15:41:32.067822 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:41:32.067785    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc394fc0_3581_48d0_ba1e_9c70876d9a73.slice/crio-c84dbe9fe04ebd85ceda58e8a8be1564f6f4dfc2da3d88eaefeba46c2a971307 WatchSource:0}: Error finding container c84dbe9fe04ebd85ceda58e8a8be1564f6f4dfc2da3d88eaefeba46c2a971307: Status 404 returned error can't find the container with id c84dbe9fe04ebd85ceda58e8a8be1564f6f4dfc2da3d88eaefeba46c2a971307
Jul 24 15:41:32.349251 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.349210    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gbdjq" event={"ID":"dc394fc0-3581-48d0-ba1e-9c70876d9a73","Type":"ContainerStarted","Data":"c84dbe9fe04ebd85ceda58e8a8be1564f6f4dfc2da3d88eaefeba46c2a971307"}
Jul 24 15:41:32.350555 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.350525    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59fb5479dc-kcpmq" event={"ID":"5ef6a247-e63f-4351-906b-479a8ddb2294","Type":"ContainerStarted","Data":"0379d5a90b1b2034d0d41c42ab0b686999e18a87a44aec27f7d4b2bc6625d381"}
Jul 24 15:41:32.513901 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.513864    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"]
Jul 24 15:41:32.516999 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.516980    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.521173 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.521147    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 24 15:41:32.521314 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.521155    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wqjj2\""
Jul 24 15:41:32.521314 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.521293    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 24 15:41:32.521489 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.521459    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 15:41:32.521779 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.521765    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 24 15:41:32.539385 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.539352    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"]
Jul 24 15:41:32.637192 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.637103    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.637192 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.637154    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.637451 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.637238    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.637451 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.637360    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd96q\" (UniqueName: \"kubernetes.io/projected/ff832ed5-2890-496d-a4a8-ad33ffa0d201-kube-api-access-gd96q\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.637451 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.637406    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff832ed5-2890-496d-a4a8-ad33ffa0d201-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.637451 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.637441    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.637685 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.637482    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.738014 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.737976    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.738196 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.738034    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.738196 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.738065    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.738196 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.738102    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.738196 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.738157    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd96q\" (UniqueName: \"kubernetes.io/projected/ff832ed5-2890-496d-a4a8-ad33ffa0d201-kube-api-access-gd96q\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.738196 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.738183    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff832ed5-2890-496d-a4a8-ad33ffa0d201-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.738469 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.738208    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.738780 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.738728    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.741035 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.741009    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.741153 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.741065    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.741253 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.741226    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff832ed5-2890-496d-a4a8-ad33ffa0d201-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.741635 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.741616    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.748412 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.748385    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.748627 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.748607    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd96q\" (UniqueName: \"kubernetes.io/projected/ff832ed5-2890-496d-a4a8-ad33ffa0d201-kube-api-access-gd96q\") pod \"istiod-openshift-gateway-7b7995d5cd-rxsh2\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.828798 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.828763    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:32.997531 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:32.997456    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"]
Jul 24 15:41:33.001931 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:41:33.001905    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff832ed5_2890_496d_a4a8_ad33ffa0d201.slice/crio-4bff881761061fa3aa71941ce11dd4bab7605c6418beaddb15c25278bc82ba96 WatchSource:0}: Error finding container 4bff881761061fa3aa71941ce11dd4bab7605c6418beaddb15c25278bc82ba96: Status 404 returned error can't find the container with id 4bff881761061fa3aa71941ce11dd4bab7605c6418beaddb15c25278bc82ba96
Jul 24 15:41:33.356801 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:33.356764    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2" event={"ID":"ff832ed5-2890-496d-a4a8-ad33ffa0d201","Type":"ContainerStarted","Data":"4bff881761061fa3aa71941ce11dd4bab7605c6418beaddb15c25278bc82ba96"}
Jul 24 15:41:34.363177 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:34.363132    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-59fb5479dc-kcpmq" event={"ID":"5ef6a247-e63f-4351-906b-479a8ddb2294","Type":"ContainerStarted","Data":"5f467de2bebbe71c2c7759218c313acdd75dd52ed1c0db410e9bec78c0a72e71"}
Jul 24 15:41:34.365324 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:34.365285    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gbdjq" event={"ID":"dc394fc0-3581-48d0-ba1e-9c70876d9a73","Type":"ContainerStarted","Data":"81a2d86117eaf611874074298fb03c6cf53210c14d1df862b6cc71b0c2b75d17"}
Jul 24 15:41:34.365476 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:34.365438    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-gbdjq"
Jul 24 15:41:34.381475 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:34.381424    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-59fb5479dc-kcpmq" podStartSLOduration=1.469440868 podStartE2EDuration="4.381406839s" totalImagesPullingTime="2.911965971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:34.380087436 +0000 UTC m=+322.790798052" watchObservedRunningTime="2026-07-24 15:41:34.381406839 +0000 UTC m=+322.792117456"
Jul 24 15:41:34.399967 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:34.399906    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-gbdjq" podStartSLOduration=1.19318048 podStartE2EDuration="3.399883396s" totalImagesPullingTime="2.206702916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:34.39723542 +0000 UTC m=+322.807946038" watchObservedRunningTime="2026-07-24 15:41:34.399883396 +0000 UTC m=+322.810594014"
Jul 24 15:41:36.139174 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:36.139133    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 15:41:36.139465 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:36.139197    2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 15:41:36.374961 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:36.374919    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2" event={"ID":"ff832ed5-2890-496d-a4a8-ad33ffa0d201","Type":"ContainerStarted","Data":"fa2fc28ae656501a2c96aa0303929fa6e0da704f2f8906b62cfa27d5a6bb2119"}
Jul 24 15:41:36.375134 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:36.375050    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:36.397016 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:36.396905    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2" podStartSLOduration=1.262377266 podStartE2EDuration="4.39688971s" totalImagesPullingTime="3.134512444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:36.395020725 +0000 UTC m=+324.805731342" watchObservedRunningTime="2026-07-24 15:41:36.39688971 +0000 UTC m=+324.807600328"
Jul 24 15:41:37.381405 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:37.381376    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:41.239646 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.239611    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-pqdnc"]
Jul 24 15:41:41.243406 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.243382    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pqdnc"
Jul 24 15:41:41.249519 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.249495    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 24 15:41:41.249747 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.249720    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-zw5gc\""
Jul 24 15:41:41.256809 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.256777    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-pqdnc"]
Jul 24 15:41:41.261432 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.261408    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68"]
Jul 24 15:41:41.267681 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.267641    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68"
Jul 24 15:41:41.270256 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.270237    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-d4tc9\""
Jul 24 15:41:41.270387 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.270238    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 24 15:41:41.274797 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.274735    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68"]
Jul 24 15:41:41.421223 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.421186    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd-cert\") pod \"kserve-controller-manager-d6f4b65b8-pqdnc\" (UID: \"9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pqdnc"
Jul 24 15:41:41.421428 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.421245    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2vn9\" (UniqueName: \"kubernetes.io/projected/9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd-kube-api-access-c2vn9\") pod \"kserve-controller-manager-d6f4b65b8-pqdnc\" (UID: \"9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pqdnc"
Jul 24 15:41:41.421428 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.421353    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eac2967-63a8-4d6e-89a0-3fd21f1cee11-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-8zx68\" (UID: \"9eac2967-63a8-4d6e-89a0-3fd21f1cee11\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68"
Jul 24 15:41:41.421428 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.421419    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r99n\" (UniqueName: \"kubernetes.io/projected/9eac2967-63a8-4d6e-89a0-3fd21f1cee11-kube-api-access-9r99n\") pod \"llmisvc-controller-manager-777c6bd7d5-8zx68\" (UID: \"9eac2967-63a8-4d6e-89a0-3fd21f1cee11\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68"
Jul 24 15:41:41.522238 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.522144    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd-cert\") pod \"kserve-controller-manager-d6f4b65b8-pqdnc\" (UID: \"9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pqdnc"
Jul 24 15:41:41.522238 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.522192    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2vn9\" (UniqueName: \"kubernetes.io/projected/9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd-kube-api-access-c2vn9\") pod \"kserve-controller-manager-d6f4b65b8-pqdnc\" (UID: \"9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pqdnc"
Jul 24 15:41:41.522238 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.522220    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eac2967-63a8-4d6e-89a0-3fd21f1cee11-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-8zx68\" (UID: \"9eac2967-63a8-4d6e-89a0-3fd21f1cee11\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68"
Jul 24 15:41:41.522524 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.522248    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r99n\" (UniqueName: \"kubernetes.io/projected/9eac2967-63a8-4d6e-89a0-3fd21f1cee11-kube-api-access-9r99n\") pod \"llmisvc-controller-manager-777c6bd7d5-8zx68\" (UID: \"9eac2967-63a8-4d6e-89a0-3fd21f1cee11\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68"
Jul 24 15:41:41.522524 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:41:41.522336    2561 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 24 15:41:41.522524 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:41:41.522405    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eac2967-63a8-4d6e-89a0-3fd21f1cee11-cert podName:9eac2967-63a8-4d6e-89a0-3fd21f1cee11 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:42.022383977 +0000 UTC m=+330.433094592 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9eac2967-63a8-4d6e-89a0-3fd21f1cee11-cert") pod "llmisvc-controller-manager-777c6bd7d5-8zx68" (UID: "9eac2967-63a8-4d6e-89a0-3fd21f1cee11") : secret "llmisvc-webhook-server-cert" not found
Jul 24 15:41:41.522524 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:41:41.522336    2561 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 24 15:41:41.522524 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:41:41.522480    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd-cert podName:9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd nodeName:}" failed. No retries permitted until 2026-07-24 15:41:42.022462585 +0000 UTC m=+330.433173195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd-cert") pod "kserve-controller-manager-d6f4b65b8-pqdnc" (UID: "9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd") : secret "kserve-webhook-server-cert" not found
Jul 24 15:41:41.540852 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.540818    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r99n\" (UniqueName: \"kubernetes.io/projected/9eac2967-63a8-4d6e-89a0-3fd21f1cee11-kube-api-access-9r99n\") pod \"llmisvc-controller-manager-777c6bd7d5-8zx68\" (UID: \"9eac2967-63a8-4d6e-89a0-3fd21f1cee11\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68"
Jul 24 15:41:41.541257 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:41.541235    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2vn9\" (UniqueName: \"kubernetes.io/projected/9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd-kube-api-access-c2vn9\") pod \"kserve-controller-manager-d6f4b65b8-pqdnc\" (UID: \"9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pqdnc"
Jul 24 15:41:42.025431 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:42.025397    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd-cert\") pod \"kserve-controller-manager-d6f4b65b8-pqdnc\" (UID: \"9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pqdnc"
Jul 24 15:41:42.025625 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:42.025463    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eac2967-63a8-4d6e-89a0-3fd21f1cee11-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-8zx68\" (UID: \"9eac2967-63a8-4d6e-89a0-3fd21f1cee11\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68"
Jul 24 15:41:42.025625 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:41:42.025572    2561 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 24 15:41:42.025752 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:41:42.025638    2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eac2967-63a8-4d6e-89a0-3fd21f1cee11-cert podName:9eac2967-63a8-4d6e-89a0-3fd21f1cee11 nodeName:}" failed. No retries permitted until 2026-07-24 15:41:43.025620276 +0000 UTC m=+331.436330881 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9eac2967-63a8-4d6e-89a0-3fd21f1cee11-cert") pod "llmisvc-controller-manager-777c6bd7d5-8zx68" (UID: "9eac2967-63a8-4d6e-89a0-3fd21f1cee11") : secret "llmisvc-webhook-server-cert" not found
Jul 24 15:41:42.028516 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:42.028489    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd-cert\") pod \"kserve-controller-manager-d6f4b65b8-pqdnc\" (UID: \"9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pqdnc"
Jul 24 15:41:42.156158 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:42.156117    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pqdnc"
Jul 24 15:41:42.303113 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:42.303032    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-pqdnc"]
Jul 24 15:41:42.306035 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:41:42.306008    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a4f23fb_7e92_45fb_bc69_2a506e5fb6bd.slice/crio-78854a39d4fbd769adfcdbe51c3da8b9588bae7678f7b3204a1c1af9c053b7f1 WatchSource:0}: Error finding container 78854a39d4fbd769adfcdbe51c3da8b9588bae7678f7b3204a1c1af9c053b7f1: Status 404 returned error can't find the container with id 78854a39d4fbd769adfcdbe51c3da8b9588bae7678f7b3204a1c1af9c053b7f1
Jul 24 15:41:42.401116 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:42.401082    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pqdnc" event={"ID":"9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd","Type":"ContainerStarted","Data":"78854a39d4fbd769adfcdbe51c3da8b9588bae7678f7b3204a1c1af9c053b7f1"}
Jul 24 15:41:43.035457 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:43.035400    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eac2967-63a8-4d6e-89a0-3fd21f1cee11-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-8zx68\" (UID: \"9eac2967-63a8-4d6e-89a0-3fd21f1cee11\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68"
Jul 24 15:41:43.038794 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:43.038766    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eac2967-63a8-4d6e-89a0-3fd21f1cee11-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-8zx68\" (UID: \"9eac2967-63a8-4d6e-89a0-3fd21f1cee11\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68"
Jul 24 15:41:43.081614 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:43.081575    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68"
Jul 24 15:41:43.236176 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:43.236148    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68"]
Jul 24 15:41:43.238145 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:41:43.238118    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9eac2967_63a8_4d6e_89a0_3fd21f1cee11.slice/crio-86ac524b38fbd544f0f83038208d63f6c3aae0dba125c6a0c507a2d4446f33dd WatchSource:0}: Error finding container 86ac524b38fbd544f0f83038208d63f6c3aae0dba125c6a0c507a2d4446f33dd: Status 404 returned error can't find the container with id 86ac524b38fbd544f0f83038208d63f6c3aae0dba125c6a0c507a2d4446f33dd
Jul 24 15:41:43.405873 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:43.405768    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68" event={"ID":"9eac2967-63a8-4d6e-89a0-3fd21f1cee11","Type":"ContainerStarted","Data":"86ac524b38fbd544f0f83038208d63f6c3aae0dba125c6a0c507a2d4446f33dd"}
Jul 24 15:41:44.848935 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:44.848890    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"]
Jul 24 15:41:44.849337 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:44.849179    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2" podUID="ff832ed5-2890-496d-a4a8-ad33ffa0d201" containerName="discovery" containerID="cri-o://fa2fc28ae656501a2c96aa0303929fa6e0da704f2f8906b62cfa27d5a6bb2119" gracePeriod=30
Jul 24 15:41:45.133574 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.133547    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:45.154982 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.154950    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gd96q\" (UniqueName: \"kubernetes.io/projected/ff832ed5-2890-496d-a4a8-ad33ffa0d201-kube-api-access-gd96q\") pod \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") "
Jul 24 15:41:45.155223 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.154992    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff832ed5-2890-496d-a4a8-ad33ffa0d201-local-certs\") pod \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") "
Jul 24 15:41:45.155223 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.155011    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-cacerts\") pod \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") "
Jul 24 15:41:45.155223 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.155045    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-csr-ca-configmap\") pod \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") "
Jul 24 15:41:45.155223 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.155151    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-csr-dns-cert\") pod \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") "
Jul 24 15:41:45.155464 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.155252    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-kubeconfig\") pod \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") "
Jul 24 15:41:45.155464 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.155299    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-token\") pod \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\" (UID: \"ff832ed5-2890-496d-a4a8-ad33ffa0d201\") "
Jul 24 15:41:45.155464 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.155412    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "ff832ed5-2890-496d-a4a8-ad33ffa0d201" (UID: "ff832ed5-2890-496d-a4a8-ad33ffa0d201"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:41:45.155623 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.155544    2561 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-csr-ca-configmap\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:41:45.158064 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.158017    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-cacerts" (OuterVolumeSpecName: "cacerts") pod "ff832ed5-2890-496d-a4a8-ad33ffa0d201" (UID: "ff832ed5-2890-496d-a4a8-ad33ffa0d201"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:41:45.158203 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.158095    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff832ed5-2890-496d-a4a8-ad33ffa0d201-kube-api-access-gd96q" (OuterVolumeSpecName: "kube-api-access-gd96q") pod "ff832ed5-2890-496d-a4a8-ad33ffa0d201" (UID: "ff832ed5-2890-496d-a4a8-ad33ffa0d201"). InnerVolumeSpecName "kube-api-access-gd96q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:41:45.158288 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.158265    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "ff832ed5-2890-496d-a4a8-ad33ffa0d201" (UID: "ff832ed5-2890-496d-a4a8-ad33ffa0d201"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:41:45.158341 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.158306    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-token" (OuterVolumeSpecName: "istio-token") pod "ff832ed5-2890-496d-a4a8-ad33ffa0d201" (UID: "ff832ed5-2890-496d-a4a8-ad33ffa0d201"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:41:45.158534 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.158505    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "ff832ed5-2890-496d-a4a8-ad33ffa0d201" (UID: "ff832ed5-2890-496d-a4a8-ad33ffa0d201"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:41:45.158638 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.158607    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff832ed5-2890-496d-a4a8-ad33ffa0d201-local-certs" (OuterVolumeSpecName: "local-certs") pod "ff832ed5-2890-496d-a4a8-ad33ffa0d201" (UID: "ff832ed5-2890-496d-a4a8-ad33ffa0d201"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:41:45.257018 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.256982    2561 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-kubeconfig\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:41:45.257018 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.257017    2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-token\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:41:45.257018 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.257027    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gd96q\" (UniqueName: \"kubernetes.io/projected/ff832ed5-2890-496d-a4a8-ad33ffa0d201-kube-api-access-gd96q\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:41:45.257277 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.257040    2561 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff832ed5-2890-496d-a4a8-ad33ffa0d201-local-certs\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:41:45.257277 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.257054    2561 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-cacerts\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:41:45.257277 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.257062    2561 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff832ed5-2890-496d-a4a8-ad33ffa0d201-istio-csr-dns-cert\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:41:45.375043 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.374951    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-gbdjq"
Jul 24 15:41:45.418830 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.418505    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pqdnc" event={"ID":"9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd","Type":"ContainerStarted","Data":"c9451ddf020f210fe9f81c1cc3c058145577a9871365b37a2a6ea16aa09ec1cf"}
Jul 24 15:41:45.418830 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.418581    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pqdnc"
Jul 24 15:41:45.420222 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.420190    2561 generic.go:356] "Generic (PLEG): container finished" podID="ff832ed5-2890-496d-a4a8-ad33ffa0d201" containerID="fa2fc28ae656501a2c96aa0303929fa6e0da704f2f8906b62cfa27d5a6bb2119" exitCode=0
Jul 24 15:41:45.420355 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.420240    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"
Jul 24 15:41:45.420355 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.420275    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2" event={"ID":"ff832ed5-2890-496d-a4a8-ad33ffa0d201","Type":"ContainerDied","Data":"fa2fc28ae656501a2c96aa0303929fa6e0da704f2f8906b62cfa27d5a6bb2119"}
Jul 24 15:41:45.420355 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.420323    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2" event={"ID":"ff832ed5-2890-496d-a4a8-ad33ffa0d201","Type":"ContainerDied","Data":"4bff881761061fa3aa71941ce11dd4bab7605c6418beaddb15c25278bc82ba96"}
Jul 24 15:41:45.420355 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.420342    2561 scope.go:117] "RemoveContainer" containerID="fa2fc28ae656501a2c96aa0303929fa6e0da704f2f8906b62cfa27d5a6bb2119"
Jul 24 15:41:45.436359 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.436296    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pqdnc" podStartSLOduration=1.7645274020000001 podStartE2EDuration="4.436272977s" totalImagesPullingTime="2.671745575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:45.434478539 +0000 UTC m=+333.845189156" watchObservedRunningTime="2026-07-24 15:41:45.436272977 +0000 UTC m=+333.846983597"
Jul 24 15:41:45.451275 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.451242    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"]
Jul 24 15:41:45.458532 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.458492    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rxsh2"]
Jul 24 15:41:45.536392 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.536369    2561 scope.go:117] "RemoveContainer" containerID="fa2fc28ae656501a2c96aa0303929fa6e0da704f2f8906b62cfa27d5a6bb2119"
Jul 24 15:41:45.536779 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:41:45.536756    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa2fc28ae656501a2c96aa0303929fa6e0da704f2f8906b62cfa27d5a6bb2119\": container with ID starting with fa2fc28ae656501a2c96aa0303929fa6e0da704f2f8906b62cfa27d5a6bb2119 not found: ID does not exist" containerID="fa2fc28ae656501a2c96aa0303929fa6e0da704f2f8906b62cfa27d5a6bb2119"
Jul 24 15:41:45.536860 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:45.536804    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa2fc28ae656501a2c96aa0303929fa6e0da704f2f8906b62cfa27d5a6bb2119"} err="failed to get container status \"fa2fc28ae656501a2c96aa0303929fa6e0da704f2f8906b62cfa27d5a6bb2119\": rpc error: code = NotFound desc = could not find container \"fa2fc28ae656501a2c96aa0303929fa6e0da704f2f8906b62cfa27d5a6bb2119\": container with ID starting with fa2fc28ae656501a2c96aa0303929fa6e0da704f2f8906b62cfa27d5a6bb2119 not found: ID does not exist"
Jul 24 15:41:46.106730 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:46.106697    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff832ed5-2890-496d-a4a8-ad33ffa0d201" path="/var/lib/kubelet/pods/ff832ed5-2890-496d-a4a8-ad33ffa0d201/volumes"
Jul 24 15:41:47.431485 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:47.431400    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68" event={"ID":"9eac2967-63a8-4d6e-89a0-3fd21f1cee11","Type":"ContainerStarted","Data":"b5ad141855352e579b96432068e38ed199680cba6878ee5e441f7730d17e8dfa"}
Jul 24 15:41:47.431886 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:47.431532    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68"
Jul 24 15:41:47.453350 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:41:47.449438    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68" podStartSLOduration=2.525434512 podStartE2EDuration="6.449412598s" totalImagesPullingTime="3.923978086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:41:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:41:47.447900456 +0000 UTC m=+335.858611085" watchObservedRunningTime="2026-07-24 15:41:47.449412598 +0000 UTC m=+335.860123216"
Jul 24 15:42:16.431531 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:16.431490    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pqdnc"
Jul 24 15:42:18.438129 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:18.438103    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-8zx68"
Jul 24 15:42:31.603786 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:31.603747    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-l2p6g"]
Jul 24 15:42:31.604177 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:31.604041    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff832ed5-2890-496d-a4a8-ad33ffa0d201" containerName="discovery"
Jul 24 15:42:31.604177 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:31.604053    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff832ed5-2890-496d-a4a8-ad33ffa0d201" containerName="discovery"
Jul 24 15:42:31.604177 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:31.604100    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff832ed5-2890-496d-a4a8-ad33ffa0d201" containerName="discovery"
Jul 24 15:42:31.613336 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:31.613307    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-l2p6g"
Jul 24 15:42:31.615763 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:31.615736    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 15:42:31.615925 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:31.615736    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 15:42:31.615925 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:31.615737    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-k66bw\""
Jul 24 15:42:31.624226 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:31.624197    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-l2p6g"]
Jul 24 15:42:31.734020 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:31.733974    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqflc\" (UniqueName: \"kubernetes.io/projected/16e654aa-ef11-4e42-9511-f1fa052a886f-kube-api-access-gqflc\") pod \"kuadrant-operator-catalog-l2p6g\" (UID: \"16e654aa-ef11-4e42-9511-f1fa052a886f\") " pod="kuadrant-system/kuadrant-operator-catalog-l2p6g"
Jul 24 15:42:31.834706 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:31.834648    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqflc\" (UniqueName: \"kubernetes.io/projected/16e654aa-ef11-4e42-9511-f1fa052a886f-kube-api-access-gqflc\") pod \"kuadrant-operator-catalog-l2p6g\" (UID: \"16e654aa-ef11-4e42-9511-f1fa052a886f\") " pod="kuadrant-system/kuadrant-operator-catalog-l2p6g"
Jul 24 15:42:31.843503 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:31.843467    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqflc\" (UniqueName: \"kubernetes.io/projected/16e654aa-ef11-4e42-9511-f1fa052a886f-kube-api-access-gqflc\") pod \"kuadrant-operator-catalog-l2p6g\" (UID: \"16e654aa-ef11-4e42-9511-f1fa052a886f\") " pod="kuadrant-system/kuadrant-operator-catalog-l2p6g"
Jul 24 15:42:31.923776 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:31.923685    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-l2p6g"
Jul 24 15:42:31.971150 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:31.971105    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-l2p6g"]
Jul 24 15:42:32.067698 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:32.067671    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-l2p6g"]
Jul 24 15:42:32.070077 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:42:32.070051    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16e654aa_ef11_4e42_9511_f1fa052a886f.slice/crio-3f26c35cef2d893f4006a3eca1ecd2f122e3fe0f88922d2c362c156be8c28f4b WatchSource:0}: Error finding container 3f26c35cef2d893f4006a3eca1ecd2f122e3fe0f88922d2c362c156be8c28f4b: Status 404 returned error can't find the container with id 3f26c35cef2d893f4006a3eca1ecd2f122e3fe0f88922d2c362c156be8c28f4b
Jul 24 15:42:32.183988 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:32.183903    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zjvt9"]
Jul 24 15:42:32.188195 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:32.188174    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zjvt9"
Jul 24 15:42:32.197220 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:32.197186    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zjvt9"]
Jul 24 15:42:32.338833 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:32.338791    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wwmn\" (UniqueName: \"kubernetes.io/projected/5802f3ed-2120-4987-82e4-b3e420cf3ba3-kube-api-access-2wwmn\") pod \"kuadrant-operator-catalog-zjvt9\" (UID: \"5802f3ed-2120-4987-82e4-b3e420cf3ba3\") " pod="kuadrant-system/kuadrant-operator-catalog-zjvt9"
Jul 24 15:42:32.440003 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:32.439892    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wwmn\" (UniqueName: \"kubernetes.io/projected/5802f3ed-2120-4987-82e4-b3e420cf3ba3-kube-api-access-2wwmn\") pod \"kuadrant-operator-catalog-zjvt9\" (UID: \"5802f3ed-2120-4987-82e4-b3e420cf3ba3\") " pod="kuadrant-system/kuadrant-operator-catalog-zjvt9"
Jul 24 15:42:32.450058 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:32.450030    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wwmn\" (UniqueName: \"kubernetes.io/projected/5802f3ed-2120-4987-82e4-b3e420cf3ba3-kube-api-access-2wwmn\") pod \"kuadrant-operator-catalog-zjvt9\" (UID: \"5802f3ed-2120-4987-82e4-b3e420cf3ba3\") " pod="kuadrant-system/kuadrant-operator-catalog-zjvt9"
Jul 24 15:42:32.498592 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:32.498561    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zjvt9"
Jul 24 15:42:32.614820 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:32.614773    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-l2p6g" event={"ID":"16e654aa-ef11-4e42-9511-f1fa052a886f","Type":"ContainerStarted","Data":"3f26c35cef2d893f4006a3eca1ecd2f122e3fe0f88922d2c362c156be8c28f4b"}
Jul 24 15:42:32.663312 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:32.663263    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zjvt9"]
Jul 24 15:42:32.701052 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:42:32.700974    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5802f3ed_2120_4987_82e4_b3e420cf3ba3.slice/crio-386e7d24c7573fb40d2a5cebe69cdcd1f453aee288d883c07089f5032ff5758b WatchSource:0}: Error finding container 386e7d24c7573fb40d2a5cebe69cdcd1f453aee288d883c07089f5032ff5758b: Status 404 returned error can't find the container with id 386e7d24c7573fb40d2a5cebe69cdcd1f453aee288d883c07089f5032ff5758b
Jul 24 15:42:33.620768 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:33.620736    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zjvt9" event={"ID":"5802f3ed-2120-4987-82e4-b3e420cf3ba3","Type":"ContainerStarted","Data":"386e7d24c7573fb40d2a5cebe69cdcd1f453aee288d883c07089f5032ff5758b"}
Jul 24 15:42:34.626241 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:34.626199    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-l2p6g" event={"ID":"16e654aa-ef11-4e42-9511-f1fa052a886f","Type":"ContainerStarted","Data":"7ec5ca83406e10b5bfc272efc1996304a84aff836656ee71291e41aa81315974"}
Jul 24 15:42:34.626785 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:34.626258    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-l2p6g" podUID="16e654aa-ef11-4e42-9511-f1fa052a886f" containerName="registry-server" containerID="cri-o://7ec5ca83406e10b5bfc272efc1996304a84aff836656ee71291e41aa81315974" gracePeriod=2
Jul 24 15:42:34.627711 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:34.627682    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zjvt9" event={"ID":"5802f3ed-2120-4987-82e4-b3e420cf3ba3","Type":"ContainerStarted","Data":"3fc7c378b7c299623bb9b50d515b019699f43f48958a84c6ff6036f61fdb5202"}
Jul 24 15:42:34.642350 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:34.642301    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-l2p6g" podStartSLOduration=1.627779169 podStartE2EDuration="3.642283395s" totalImagesPullingTime="2.014504226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:42:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:42:34.640784988 +0000 UTC m=+383.051495606" watchObservedRunningTime="2026-07-24 15:42:34.642283395 +0000 UTC m=+383.052994012"
Jul 24 15:42:34.656532 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:34.656479    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-zjvt9" podStartSLOduration=1.272334026 podStartE2EDuration="2.656461413s" totalImagesPullingTime="1.384127387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:42:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:42:34.655282571 +0000 UTC m=+383.065993198" watchObservedRunningTime="2026-07-24 15:42:34.656461413 +0000 UTC m=+383.067172030"
Jul 24 15:42:34.865586 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:34.865557    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-l2p6g"
Jul 24 15:42:34.961546 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:34.961517    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqflc\" (UniqueName: \"kubernetes.io/projected/16e654aa-ef11-4e42-9511-f1fa052a886f-kube-api-access-gqflc\") pod \"16e654aa-ef11-4e42-9511-f1fa052a886f\" (UID: \"16e654aa-ef11-4e42-9511-f1fa052a886f\") "
Jul 24 15:42:34.963965 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:34.963927    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16e654aa-ef11-4e42-9511-f1fa052a886f-kube-api-access-gqflc" (OuterVolumeSpecName: "kube-api-access-gqflc") pod "16e654aa-ef11-4e42-9511-f1fa052a886f" (UID: "16e654aa-ef11-4e42-9511-f1fa052a886f"). InnerVolumeSpecName "kube-api-access-gqflc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:42:35.062523 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:35.062483    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqflc\" (UniqueName: \"kubernetes.io/projected/16e654aa-ef11-4e42-9511-f1fa052a886f-kube-api-access-gqflc\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:42:35.633238 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:35.633203    2561 generic.go:356] "Generic (PLEG): container finished" podID="16e654aa-ef11-4e42-9511-f1fa052a886f" containerID="7ec5ca83406e10b5bfc272efc1996304a84aff836656ee71291e41aa81315974" exitCode=0
Jul 24 15:42:35.633630 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:35.633248    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-l2p6g"
Jul 24 15:42:35.633630 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:35.633295    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-l2p6g" event={"ID":"16e654aa-ef11-4e42-9511-f1fa052a886f","Type":"ContainerDied","Data":"7ec5ca83406e10b5bfc272efc1996304a84aff836656ee71291e41aa81315974"}
Jul 24 15:42:35.633630 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:35.633344    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-l2p6g" event={"ID":"16e654aa-ef11-4e42-9511-f1fa052a886f","Type":"ContainerDied","Data":"3f26c35cef2d893f4006a3eca1ecd2f122e3fe0f88922d2c362c156be8c28f4b"}
Jul 24 15:42:35.633630 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:35.633372    2561 scope.go:117] "RemoveContainer" containerID="7ec5ca83406e10b5bfc272efc1996304a84aff836656ee71291e41aa81315974"
Jul 24 15:42:35.642292 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:35.642275    2561 scope.go:117] "RemoveContainer" containerID="7ec5ca83406e10b5bfc272efc1996304a84aff836656ee71291e41aa81315974"
Jul 24 15:42:35.642601 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:42:35.642577    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ec5ca83406e10b5bfc272efc1996304a84aff836656ee71291e41aa81315974\": container with ID starting with 7ec5ca83406e10b5bfc272efc1996304a84aff836656ee71291e41aa81315974 not found: ID does not exist" containerID="7ec5ca83406e10b5bfc272efc1996304a84aff836656ee71291e41aa81315974"
Jul 24 15:42:35.642737 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:35.642607    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ec5ca83406e10b5bfc272efc1996304a84aff836656ee71291e41aa81315974"} err="failed to get container status \"7ec5ca83406e10b5bfc272efc1996304a84aff836656ee71291e41aa81315974\": rpc error: code = NotFound desc = could not find container \"7ec5ca83406e10b5bfc272efc1996304a84aff836656ee71291e41aa81315974\": container with ID starting with 7ec5ca83406e10b5bfc272efc1996304a84aff836656ee71291e41aa81315974 not found: ID does not exist"
Jul 24 15:42:35.654400 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:35.654371    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-l2p6g"]
Jul 24 15:42:35.659794 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:35.659766    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-l2p6g"]
Jul 24 15:42:36.105770 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:36.105741    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16e654aa-ef11-4e42-9511-f1fa052a886f" path="/var/lib/kubelet/pods/16e654aa-ef11-4e42-9511-f1fa052a886f/volumes"
Jul 24 15:42:42.499538 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:42.499487    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-zjvt9"
Jul 24 15:42:42.499538 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:42.499559    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-zjvt9"
Jul 24 15:42:42.521678 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:42.521632    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-zjvt9"
Jul 24 15:42:42.684747 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:42:42.684720    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-zjvt9"
Jul 24 15:43:05.809708 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:05.809674    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"]
Jul 24 15:43:05.810130 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:05.809961    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16e654aa-ef11-4e42-9511-f1fa052a886f" containerName="registry-server"
Jul 24 15:43:05.810130 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:05.809971    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="16e654aa-ef11-4e42-9511-f1fa052a886f" containerName="registry-server"
Jul 24 15:43:05.810130 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:05.810033    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="16e654aa-ef11-4e42-9511-f1fa052a886f" containerName="registry-server"
Jul 24 15:43:05.818030 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:05.817996    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:05.821137 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:05.821110    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-ssnjg\""
Jul 24 15:43:05.826406 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:05.826350    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"]
Jul 24 15:43:05.900231 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:05.900192    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69dd3778-70d9-4b18-8934-a338c3dc937b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bxq7p\" (UID: \"69dd3778-70d9-4b18-8934-a338c3dc937b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:05.900406 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:05.900242    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w88fc\" (UniqueName: \"kubernetes.io/projected/69dd3778-70d9-4b18-8934-a338c3dc937b-kube-api-access-w88fc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bxq7p\" (UID: \"69dd3778-70d9-4b18-8934-a338c3dc937b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:06.001498 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:06.001458    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69dd3778-70d9-4b18-8934-a338c3dc937b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bxq7p\" (UID: \"69dd3778-70d9-4b18-8934-a338c3dc937b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:06.001616 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:06.001513    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w88fc\" (UniqueName: \"kubernetes.io/projected/69dd3778-70d9-4b18-8934-a338c3dc937b-kube-api-access-w88fc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bxq7p\" (UID: \"69dd3778-70d9-4b18-8934-a338c3dc937b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:06.001903 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:06.001883    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69dd3778-70d9-4b18-8934-a338c3dc937b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bxq7p\" (UID: \"69dd3778-70d9-4b18-8934-a338c3dc937b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:06.010257 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:06.010229    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w88fc\" (UniqueName: \"kubernetes.io/projected/69dd3778-70d9-4b18-8934-a338c3dc937b-kube-api-access-w88fc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-bxq7p\" (UID: \"69dd3778-70d9-4b18-8934-a338c3dc937b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:06.128406 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:06.128310    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:06.274796 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:06.274757    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"]
Jul 24 15:43:06.278277 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:43:06.278239    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69dd3778_70d9_4b18_8934_a338c3dc937b.slice/crio-549803dd4bac4f5bfb5ea04d1b6e05aafe58496b40ff6f6956bd10d1e79b4df8 WatchSource:0}: Error finding container 549803dd4bac4f5bfb5ea04d1b6e05aafe58496b40ff6f6956bd10d1e79b4df8: Status 404 returned error can't find the container with id 549803dd4bac4f5bfb5ea04d1b6e05aafe58496b40ff6f6956bd10d1e79b4df8
Jul 24 15:43:06.759497 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:06.759465    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p" event={"ID":"69dd3778-70d9-4b18-8934-a338c3dc937b","Type":"ContainerStarted","Data":"549803dd4bac4f5bfb5ea04d1b6e05aafe58496b40ff6f6956bd10d1e79b4df8"}
Jul 24 15:43:07.810923 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:07.810890    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"]
Jul 24 15:43:07.814436 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:07.814415    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"
Jul 24 15:43:07.817058 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:07.817032    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-prr9f\""
Jul 24 15:43:07.826681 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:07.826627    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"]
Jul 24 15:43:07.916740 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:07.916695    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2w68\" (UniqueName: \"kubernetes.io/projected/ab6b960f-a7b2-4e76-8b3e-33f9a9124981-kube-api-access-h2w68\") pod \"limitador-operator-controller-manager-859d7b78bd-6qr6h\" (UID: \"ab6b960f-a7b2-4e76-8b3e-33f9a9124981\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"
Jul 24 15:43:08.017244 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:08.017203    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2w68\" (UniqueName: \"kubernetes.io/projected/ab6b960f-a7b2-4e76-8b3e-33f9a9124981-kube-api-access-h2w68\") pod \"limitador-operator-controller-manager-859d7b78bd-6qr6h\" (UID: \"ab6b960f-a7b2-4e76-8b3e-33f9a9124981\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"
Jul 24 15:43:08.027363 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:08.027327    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2w68\" (UniqueName: \"kubernetes.io/projected/ab6b960f-a7b2-4e76-8b3e-33f9a9124981-kube-api-access-h2w68\") pod \"limitador-operator-controller-manager-859d7b78bd-6qr6h\" (UID: \"ab6b960f-a7b2-4e76-8b3e-33f9a9124981\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"
Jul 24 15:43:08.124880 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:08.124784    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"
Jul 24 15:43:08.282185 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:08.282149    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"]
Jul 24 15:43:08.285612 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:43:08.285580    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab6b960f_a7b2_4e76_8b3e_33f9a9124981.slice/crio-a4785c40f6031b8e1598fb12cd865559199d77490bb4b573919c8d742262075c WatchSource:0}: Error finding container a4785c40f6031b8e1598fb12cd865559199d77490bb4b573919c8d742262075c: Status 404 returned error can't find the container with id a4785c40f6031b8e1598fb12cd865559199d77490bb4b573919c8d742262075c
Jul 24 15:43:08.769140 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:08.769103    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h" event={"ID":"ab6b960f-a7b2-4e76-8b3e-33f9a9124981","Type":"ContainerStarted","Data":"a4785c40f6031b8e1598fb12cd865559199d77490bb4b573919c8d742262075c"}
Jul 24 15:43:10.219077 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:10.219044    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hnnbz"]
Jul 24 15:43:10.222321 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:10.222300    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hnnbz"
Jul 24 15:43:10.224490 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:10.224475    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-hld6p\""
Jul 24 15:43:10.231634 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:10.231610    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hnnbz"]
Jul 24 15:43:10.336152 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:10.336115    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdvrb\" (UniqueName: \"kubernetes.io/projected/5e7a3852-e1c7-42f7-a96a-05b6c00f18c9-kube-api-access-rdvrb\") pod \"authorino-operator-75cc8d5cb7-hnnbz\" (UID: \"5e7a3852-e1c7-42f7-a96a-05b6c00f18c9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hnnbz"
Jul 24 15:43:10.437676 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:10.437617    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdvrb\" (UniqueName: \"kubernetes.io/projected/5e7a3852-e1c7-42f7-a96a-05b6c00f18c9-kube-api-access-rdvrb\") pod \"authorino-operator-75cc8d5cb7-hnnbz\" (UID: \"5e7a3852-e1c7-42f7-a96a-05b6c00f18c9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hnnbz"
Jul 24 15:43:10.446621 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:10.446588    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdvrb\" (UniqueName: \"kubernetes.io/projected/5e7a3852-e1c7-42f7-a96a-05b6c00f18c9-kube-api-access-rdvrb\") pod \"authorino-operator-75cc8d5cb7-hnnbz\" (UID: \"5e7a3852-e1c7-42f7-a96a-05b6c00f18c9\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-hnnbz"
Jul 24 15:43:10.532872 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:10.532778    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hnnbz"
Jul 24 15:43:10.780587 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:10.780549    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h" event={"ID":"ab6b960f-a7b2-4e76-8b3e-33f9a9124981","Type":"ContainerStarted","Data":"966ba17618acb6b3bb2bec7f8b69c48e768453b9f100dc4ebc615f7453219a6d"}
Jul 24 15:43:10.780817 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:10.780797    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"
Jul 24 15:43:10.797357 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:10.797241    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h" podStartSLOduration=2.161487543 podStartE2EDuration="3.797216331s" totalImagesPullingTime="1.635728788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:43:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:43:10.795263501 +0000 UTC m=+419.205974119" watchObservedRunningTime="2026-07-24 15:43:10.797216331 +0000 UTC m=+419.207926951"
Jul 24 15:43:11.697208 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:11.694769    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-hnnbz"]
Jul 24 15:43:11.701242 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:43:11.701212    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e7a3852_e1c7_42f7_a96a_05b6c00f18c9.slice/crio-02bde093e47973d5e612346426863a20f4e19b01f18e38a9490be44d39fd1b17 WatchSource:0}: Error finding container 02bde093e47973d5e612346426863a20f4e19b01f18e38a9490be44d39fd1b17: Status 404 returned error can't find the container with id 02bde093e47973d5e612346426863a20f4e19b01f18e38a9490be44d39fd1b17
Jul 24 15:43:11.786107 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:11.786062    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hnnbz" event={"ID":"5e7a3852-e1c7-42f7-a96a-05b6c00f18c9","Type":"ContainerStarted","Data":"02bde093e47973d5e612346426863a20f4e19b01f18e38a9490be44d39fd1b17"}
Jul 24 15:43:11.787810 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:11.787776    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p" event={"ID":"69dd3778-70d9-4b18-8934-a338c3dc937b","Type":"ContainerStarted","Data":"29fcdc0282d916d96e9d5c24b7cd237880dd8aa6f5e4d63475ac98e9ca755f90"}
Jul 24 15:43:11.787970 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:11.787918    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:11.809209 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:11.809095    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p" podStartSLOduration=1.479941166 podStartE2EDuration="6.809078836s" totalImagesPullingTime="5.32913767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:43:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:43:11.807190655 +0000 UTC m=+420.217901272" watchObservedRunningTime="2026-07-24 15:43:11.809078836 +0000 UTC m=+420.219789452"
Jul 24 15:43:14.807028 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:14.806987    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hnnbz" event={"ID":"5e7a3852-e1c7-42f7-a96a-05b6c00f18c9","Type":"ContainerStarted","Data":"b8de9999a91581b61542a37736e57412f84f9d1aa1d7677ee51334a6f8fde887"}
Jul 24 15:43:14.807645 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:14.807147    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hnnbz"
Jul 24 15:43:14.832917 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:14.832861    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hnnbz" podStartSLOduration=2.577667361 podStartE2EDuration="4.832844817s" totalImagesPullingTime="2.255177456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:43:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:43:14.832031875 +0000 UTC m=+423.242742495" watchObservedRunningTime="2026-07-24 15:43:14.832844817 +0000 UTC m=+423.243555435"
Jul 24 15:43:21.790879 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:21.790847    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"
Jul 24 15:43:22.796412 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:22.796385    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:24.748081 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.748045    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"]
Jul 24 15:43:24.748521 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.748282    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p" podUID="69dd3778-70d9-4b18-8934-a338c3dc937b" containerName="manager" containerID="cri-o://29fcdc0282d916d96e9d5c24b7cd237880dd8aa6f5e4d63475ac98e9ca755f90" gracePeriod=2
Jul 24 15:43:24.752208 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.752181    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"]
Jul 24 15:43:24.765259 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.765228    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"]
Jul 24 15:43:24.765552 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.765499    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h" podUID="ab6b960f-a7b2-4e76-8b3e-33f9a9124981" containerName="manager" containerID="cri-o://966ba17618acb6b3bb2bec7f8b69c48e768453b9f100dc4ebc615f7453219a6d" gracePeriod=2
Jul 24 15:43:24.771195 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.771147    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"]
Jul 24 15:43:24.773935 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:24.773907    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bxq7p\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="69dd3778-70d9-4b18-8934-a338c3dc937b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:24.780391 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.780365    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"]
Jul 24 15:43:24.780714 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.780699    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69dd3778-70d9-4b18-8934-a338c3dc937b" containerName="manager"
Jul 24 15:43:24.780791 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.780717    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="69dd3778-70d9-4b18-8934-a338c3dc937b" containerName="manager"
Jul 24 15:43:24.780791 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.780736    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab6b960f-a7b2-4e76-8b3e-33f9a9124981" containerName="manager"
Jul 24 15:43:24.780791 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.780744    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab6b960f-a7b2-4e76-8b3e-33f9a9124981" containerName="manager"
Jul 24 15:43:24.780946 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.780848    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="69dd3778-70d9-4b18-8934-a338c3dc937b" containerName="manager"
Jul 24 15:43:24.780946 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.780865    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab6b960f-a7b2-4e76-8b3e-33f9a9124981" containerName="manager"
Jul 24 15:43:24.784148 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.784130    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"
Jul 24 15:43:24.786381 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:24.786355    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6qr6h\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="ab6b960f-a7b2-4e76-8b3e-33f9a9124981" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"
Jul 24 15:43:24.788335 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:24.788271    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bxq7p\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="69dd3778-70d9-4b18-8934-a338c3dc937b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:24.789219 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.789197    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbt4w"]
Jul 24 15:43:24.792316 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.792297    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbt4w"
Jul 24 15:43:24.808251 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.808218    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"]
Jul 24 15:43:24.810188 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.810163    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbt4w"]
Jul 24 15:43:24.825237 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:24.825206    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bxq7p\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="69dd3778-70d9-4b18-8934-a338c3dc937b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:24.846717 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:24.846680    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6qr6h\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="ab6b960f-a7b2-4e76-8b3e-33f9a9124981" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"
Jul 24 15:43:24.857860 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.857820    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl9qw\" (UniqueName: \"kubernetes.io/projected/52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d-kube-api-access-rl9qw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lrkjk\" (UID: \"52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"
Jul 24 15:43:24.858034 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.857869    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p69s\" (UniqueName: \"kubernetes.io/projected/7b21dbee-df26-48c4-a68d-25a8feb04e6a-kube-api-access-7p69s\") pod \"limitador-operator-controller-manager-859d7b78bd-bbt4w\" (UID: \"7b21dbee-df26-48c4-a68d-25a8feb04e6a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbt4w"
Jul 24 15:43:24.858034 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.857947    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lrkjk\" (UID: \"52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"
Jul 24 15:43:24.959198 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.959165    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lrkjk\" (UID: \"52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"
Jul 24 15:43:24.959198 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.959214    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl9qw\" (UniqueName: \"kubernetes.io/projected/52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d-kube-api-access-rl9qw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lrkjk\" (UID: \"52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"
Jul 24 15:43:24.959408 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.959239    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p69s\" (UniqueName: \"kubernetes.io/projected/7b21dbee-df26-48c4-a68d-25a8feb04e6a-kube-api-access-7p69s\") pod \"limitador-operator-controller-manager-859d7b78bd-bbt4w\" (UID: \"7b21dbee-df26-48c4-a68d-25a8feb04e6a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbt4w"
Jul 24 15:43:24.959703 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.959654    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lrkjk\" (UID: \"52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"
Jul 24 15:43:24.972652 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.972619    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl9qw\" (UniqueName: \"kubernetes.io/projected/52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d-kube-api-access-rl9qw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lrkjk\" (UID: \"52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"
Jul 24 15:43:24.972797 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:24.972624    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p69s\" (UniqueName: \"kubernetes.io/projected/7b21dbee-df26-48c4-a68d-25a8feb04e6a-kube-api-access-7p69s\") pod \"limitador-operator-controller-manager-859d7b78bd-bbt4w\" (UID: \"7b21dbee-df26-48c4-a68d-25a8feb04e6a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbt4w"
Jul 24 15:43:25.002023 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.001970    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:25.004039 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.004016    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"
Jul 24 15:43:25.004764 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:25.004734    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6qr6h\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="ab6b960f-a7b2-4e76-8b3e-33f9a9124981" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"
Jul 24 15:43:25.006542 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:25.006519    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bxq7p\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="69dd3778-70d9-4b18-8934-a338c3dc937b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:25.008779 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:25.008752    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6qr6h\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="ab6b960f-a7b2-4e76-8b3e-33f9a9124981" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"
Jul 24 15:43:25.010515 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:25.010495    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bxq7p\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="69dd3778-70d9-4b18-8934-a338c3dc937b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:25.023994 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.023961    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"]
Jul 24 15:43:25.027083 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.027064    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"
Jul 24 15:43:25.029270 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:25.029248    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bxq7p\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="69dd3778-70d9-4b18-8934-a338c3dc937b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:25.031102 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:25.031083    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6qr6h\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="ab6b960f-a7b2-4e76-8b3e-33f9a9124981" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"
Jul 24 15:43:25.040833 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.040808    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"]
Jul 24 15:43:25.060390 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.060359    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2w68\" (UniqueName: \"kubernetes.io/projected/ab6b960f-a7b2-4e76-8b3e-33f9a9124981-kube-api-access-h2w68\") pod \"ab6b960f-a7b2-4e76-8b3e-33f9a9124981\" (UID: \"ab6b960f-a7b2-4e76-8b3e-33f9a9124981\") "
Jul 24 15:43:25.060390 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.060392    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w88fc\" (UniqueName: \"kubernetes.io/projected/69dd3778-70d9-4b18-8934-a338c3dc937b-kube-api-access-w88fc\") pod \"69dd3778-70d9-4b18-8934-a338c3dc937b\" (UID: \"69dd3778-70d9-4b18-8934-a338c3dc937b\") "
Jul 24 15:43:25.060719 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.060430    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69dd3778-70d9-4b18-8934-a338c3dc937b-extensions-socket-volume\") pod \"69dd3778-70d9-4b18-8934-a338c3dc937b\" (UID: \"69dd3778-70d9-4b18-8934-a338c3dc937b\") "
Jul 24 15:43:25.060992 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.060966    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69dd3778-70d9-4b18-8934-a338c3dc937b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "69dd3778-70d9-4b18-8934-a338c3dc937b" (UID: "69dd3778-70d9-4b18-8934-a338c3dc937b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:43:25.062583 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.062560    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab6b960f-a7b2-4e76-8b3e-33f9a9124981-kube-api-access-h2w68" (OuterVolumeSpecName: "kube-api-access-h2w68") pod "ab6b960f-a7b2-4e76-8b3e-33f9a9124981" (UID: "ab6b960f-a7b2-4e76-8b3e-33f9a9124981"). InnerVolumeSpecName "kube-api-access-h2w68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:43:25.062648 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.062573    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69dd3778-70d9-4b18-8934-a338c3dc937b-kube-api-access-w88fc" (OuterVolumeSpecName: "kube-api-access-w88fc") pod "69dd3778-70d9-4b18-8934-a338c3dc937b" (UID: "69dd3778-70d9-4b18-8934-a338c3dc937b"). InnerVolumeSpecName "kube-api-access-w88fc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:43:25.161504 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.161470    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnwtb\" (UniqueName: \"kubernetes.io/projected/b7dbdb02-a636-4be2-addf-70d7f4e1ed87-kube-api-access-gnwtb\") pod \"kuadrant-operator-controller-manager-86878c769f-2fgg5\" (UID: \"b7dbdb02-a636-4be2-addf-70d7f4e1ed87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"
Jul 24 15:43:25.161736 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.161539    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7dbdb02-a636-4be2-addf-70d7f4e1ed87-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2fgg5\" (UID: \"b7dbdb02-a636-4be2-addf-70d7f4e1ed87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"
Jul 24 15:43:25.161736 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.161619    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2w68\" (UniqueName: \"kubernetes.io/projected/ab6b960f-a7b2-4e76-8b3e-33f9a9124981-kube-api-access-h2w68\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:43:25.161736 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.161639    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w88fc\" (UniqueName: \"kubernetes.io/projected/69dd3778-70d9-4b18-8934-a338c3dc937b-kube-api-access-w88fc\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:43:25.161736 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.161648    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69dd3778-70d9-4b18-8934-a338c3dc937b-extensions-socket-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:43:25.180513 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.180481    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"
Jul 24 15:43:25.186643 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.186624    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbt4w"
Jul 24 15:43:25.263312 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.263053    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7dbdb02-a636-4be2-addf-70d7f4e1ed87-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2fgg5\" (UID: \"b7dbdb02-a636-4be2-addf-70d7f4e1ed87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"
Jul 24 15:43:25.263312 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.263098    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnwtb\" (UniqueName: \"kubernetes.io/projected/b7dbdb02-a636-4be2-addf-70d7f4e1ed87-kube-api-access-gnwtb\") pod \"kuadrant-operator-controller-manager-86878c769f-2fgg5\" (UID: \"b7dbdb02-a636-4be2-addf-70d7f4e1ed87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"
Jul 24 15:43:25.263514 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.263426    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7dbdb02-a636-4be2-addf-70d7f4e1ed87-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2fgg5\" (UID: \"b7dbdb02-a636-4be2-addf-70d7f4e1ed87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"
Jul 24 15:43:25.274568 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.274511    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnwtb\" (UniqueName: \"kubernetes.io/projected/b7dbdb02-a636-4be2-addf-70d7f4e1ed87-kube-api-access-gnwtb\") pod \"kuadrant-operator-controller-manager-86878c769f-2fgg5\" (UID: \"b7dbdb02-a636-4be2-addf-70d7f4e1ed87\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"
Jul 24 15:43:25.328299 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.328277    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"]
Jul 24 15:43:25.331263 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:43:25.331229    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f0c4f3_ba07_4c67_99dc_f478bb1c5b1d.slice/crio-aae41b538fecd4e5c468afdc26304a1ae5d90b95614c25c1dc8629aa87d17789 WatchSource:0}: Error finding container aae41b538fecd4e5c468afdc26304a1ae5d90b95614c25c1dc8629aa87d17789: Status 404 returned error can't find the container with id aae41b538fecd4e5c468afdc26304a1ae5d90b95614c25c1dc8629aa87d17789
Jul 24 15:43:25.336822 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.336803    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"
Jul 24 15:43:25.360455 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.360430    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbt4w"]
Jul 24 15:43:25.366854 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:43:25.366824    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b21dbee_df26_48c4_a68d_25a8feb04e6a.slice/crio-ebb02b88a171cdb2d61a2d3fd25023ac82df9f3455badef3428dc421d4c8e32e WatchSource:0}: Error finding container ebb02b88a171cdb2d61a2d3fd25023ac82df9f3455badef3428dc421d4c8e32e: Status 404 returned error can't find the container with id ebb02b88a171cdb2d61a2d3fd25023ac82df9f3455badef3428dc421d4c8e32e
Jul 24 15:43:25.501611 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.501585    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"]
Jul 24 15:43:25.504082 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:43:25.504059    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7dbdb02_a636_4be2_addf_70d7f4e1ed87.slice/crio-19e606a49a48d2fafe2d4d626fb93a027cdcf516a5d93bef28a51f11509c71b0 WatchSource:0}: Error finding container 19e606a49a48d2fafe2d4d626fb93a027cdcf516a5d93bef28a51f11509c71b0: Status 404 returned error can't find the container with id 19e606a49a48d2fafe2d4d626fb93a027cdcf516a5d93bef28a51f11509c71b0
Jul 24 15:43:25.813810 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.813777    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-hnnbz"
Jul 24 15:43:25.817017 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:25.816994    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6qr6h\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="ab6b960f-a7b2-4e76-8b3e-33f9a9124981" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"
Jul 24 15:43:25.839325 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:25.839282    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bxq7p\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="69dd3778-70d9-4b18-8934-a338c3dc937b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:25.858851 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.858802    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk" event={"ID":"52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d","Type":"ContainerStarted","Data":"c5d1f900fe2c1dad27c305a9c44d03bc602382295275adee33ee34bc1d28d246"}
Jul 24 15:43:25.859037 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.858880    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk" event={"ID":"52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d","Type":"ContainerStarted","Data":"aae41b538fecd4e5c468afdc26304a1ae5d90b95614c25c1dc8629aa87d17789"}
Jul 24 15:43:25.859037 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.858937    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"
Jul 24 15:43:25.860836 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.860804    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5" event={"ID":"b7dbdb02-a636-4be2-addf-70d7f4e1ed87","Type":"ContainerStarted","Data":"fe5096cd621ea0d221c6b861c1962b11199cd74dfdf5d5855848d2788ab7a458"}
Jul 24 15:43:25.860989 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.860869    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"
Jul 24 15:43:25.860989 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.860893    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5" event={"ID":"b7dbdb02-a636-4be2-addf-70d7f4e1ed87","Type":"ContainerStarted","Data":"19e606a49a48d2fafe2d4d626fb93a027cdcf516a5d93bef28a51f11509c71b0"}
Jul 24 15:43:25.861260 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:25.861235    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bxq7p\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="69dd3778-70d9-4b18-8934-a338c3dc937b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:25.862302 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.862279    2561 generic.go:356] "Generic (PLEG): container finished" podID="69dd3778-70d9-4b18-8934-a338c3dc937b" containerID="29fcdc0282d916d96e9d5c24b7cd237880dd8aa6f5e4d63475ac98e9ca755f90" exitCode=0
Jul 24 15:43:25.862399 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.862310    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:25.862399 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.862379    2561 scope.go:117] "RemoveContainer" containerID="29fcdc0282d916d96e9d5c24b7cd237880dd8aa6f5e4d63475ac98e9ca755f90"
Jul 24 15:43:25.864688 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.864566    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbt4w" event={"ID":"7b21dbee-df26-48c4-a68d-25a8feb04e6a","Type":"ContainerStarted","Data":"1ebaa5f419f632fe10bf0cba202671dbe131ff9de07016cc36c74616a0bfcc41"}
Jul 24 15:43:25.864688 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.864621    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbt4w"
Jul 24 15:43:25.864688 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.864646    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbt4w" event={"ID":"7b21dbee-df26-48c4-a68d-25a8feb04e6a","Type":"ContainerStarted","Data":"ebb02b88a171cdb2d61a2d3fd25023ac82df9f3455badef3428dc421d4c8e32e"}
Jul 24 15:43:25.866453 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.866430    2561 generic.go:356] "Generic (PLEG): container finished" podID="ab6b960f-a7b2-4e76-8b3e-33f9a9124981" containerID="966ba17618acb6b3bb2bec7f8b69c48e768453b9f100dc4ebc615f7453219a6d" exitCode=0
Jul 24 15:43:25.866563 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.866479    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"
Jul 24 15:43:25.873885 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.873483    2561 scope.go:117] "RemoveContainer" containerID="29fcdc0282d916d96e9d5c24b7cd237880dd8aa6f5e4d63475ac98e9ca755f90"
Jul 24 15:43:25.873994 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:25.873964    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29fcdc0282d916d96e9d5c24b7cd237880dd8aa6f5e4d63475ac98e9ca755f90\": container with ID starting with 29fcdc0282d916d96e9d5c24b7cd237880dd8aa6f5e4d63475ac98e9ca755f90 not found: ID does not exist" containerID="29fcdc0282d916d96e9d5c24b7cd237880dd8aa6f5e4d63475ac98e9ca755f90"
Jul 24 15:43:25.874108 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.873995    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29fcdc0282d916d96e9d5c24b7cd237880dd8aa6f5e4d63475ac98e9ca755f90"} err="failed to get container status \"29fcdc0282d916d96e9d5c24b7cd237880dd8aa6f5e4d63475ac98e9ca755f90\": rpc error: code = NotFound desc = could not find container \"29fcdc0282d916d96e9d5c24b7cd237880dd8aa6f5e4d63475ac98e9ca755f90\": container with ID starting with 29fcdc0282d916d96e9d5c24b7cd237880dd8aa6f5e4d63475ac98e9ca755f90 not found: ID does not exist"
Jul 24 15:43:25.874108 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.874025    2561 scope.go:117] "RemoveContainer" containerID="966ba17618acb6b3bb2bec7f8b69c48e768453b9f100dc4ebc615f7453219a6d"
Jul 24 15:43:25.882611 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.882590    2561 scope.go:117] "RemoveContainer" containerID="966ba17618acb6b3bb2bec7f8b69c48e768453b9f100dc4ebc615f7453219a6d"
Jul 24 15:43:25.882985 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:25.882966    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"966ba17618acb6b3bb2bec7f8b69c48e768453b9f100dc4ebc615f7453219a6d\": container with ID starting with 966ba17618acb6b3bb2bec7f8b69c48e768453b9f100dc4ebc615f7453219a6d not found: ID does not exist" containerID="966ba17618acb6b3bb2bec7f8b69c48e768453b9f100dc4ebc615f7453219a6d"
Jul 24 15:43:25.883050 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.882995    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"966ba17618acb6b3bb2bec7f8b69c48e768453b9f100dc4ebc615f7453219a6d"} err="failed to get container status \"966ba17618acb6b3bb2bec7f8b69c48e768453b9f100dc4ebc615f7453219a6d\": rpc error: code = NotFound desc = could not find container \"966ba17618acb6b3bb2bec7f8b69c48e768453b9f100dc4ebc615f7453219a6d\": container with ID starting with 966ba17618acb6b3bb2bec7f8b69c48e768453b9f100dc4ebc615f7453219a6d not found: ID does not exist"
Jul 24 15:43:25.886094 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.886050    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk" podStartSLOduration=1.886036227 podStartE2EDuration="1.886036227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:43:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:43:25.884891233 +0000 UTC m=+434.295601887" watchObservedRunningTime="2026-07-24 15:43:25.886036227 +0000 UTC m=+434.296746843"
Jul 24 15:43:25.887076 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:25.887048    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6qr6h\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="ab6b960f-a7b2-4e76-8b3e-33f9a9124981" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"
Jul 24 15:43:25.888996 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:25.888972    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6qr6h\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="ab6b960f-a7b2-4e76-8b3e-33f9a9124981" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6qr6h"
Jul 24 15:43:25.912200 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.911856    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5" podStartSLOduration=1.911836437 podStartE2EDuration="1.911836437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:43:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:43:25.911175853 +0000 UTC m=+434.321886493" watchObservedRunningTime="2026-07-24 15:43:25.911836437 +0000 UTC m=+434.322547054"
Jul 24 15:43:25.913844 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:25.913806    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-bxq7p\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="69dd3778-70d9-4b18-8934-a338c3dc937b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-bxq7p"
Jul 24 15:43:25.937738 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:25.936346    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbt4w" podStartSLOduration=1.936323649 podStartE2EDuration="1.936323649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:43:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:43:25.935866525 +0000 UTC m=+434.346577181" watchObservedRunningTime="2026-07-24 15:43:25.936323649 +0000 UTC m=+434.347034267"
Jul 24 15:43:26.106885 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:26.106804    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69dd3778-70d9-4b18-8934-a338c3dc937b" path="/var/lib/kubelet/pods/69dd3778-70d9-4b18-8934-a338c3dc937b/volumes"
Jul 24 15:43:26.107119 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:26.107106    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab6b960f-a7b2-4e76-8b3e-33f9a9124981" path="/var/lib/kubelet/pods/ab6b960f-a7b2-4e76-8b3e-33f9a9124981/volumes"
Jul 24 15:43:36.875498 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:36.875410    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bbt4w"
Jul 24 15:43:36.875498 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:36.875495    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"
Jul 24 15:43:36.876047 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:36.875538    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"
Jul 24 15:43:36.961281 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:36.961248    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"]
Jul 24 15:43:36.961498 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:36.961477    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk" podUID="52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d" containerName="manager" containerID="cri-o://c5d1f900fe2c1dad27c305a9c44d03bc602382295275adee33ee34bc1d28d246" gracePeriod=10
Jul 24 15:43:37.210334 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:37.210312    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"
Jul 24 15:43:37.362833 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:37.362797    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d-extensions-socket-volume\") pod \"52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d\" (UID: \"52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d\") "
Jul 24 15:43:37.362987 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:37.362845    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rl9qw\" (UniqueName: \"kubernetes.io/projected/52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d-kube-api-access-rl9qw\") pod \"52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d\" (UID: \"52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d\") "
Jul 24 15:43:37.363224 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:37.363202    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d" (UID: "52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:43:37.365128 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:37.365105    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d-kube-api-access-rl9qw" (OuterVolumeSpecName: "kube-api-access-rl9qw") pod "52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d" (UID: "52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d"). InnerVolumeSpecName "kube-api-access-rl9qw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:43:37.463694 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:37.463627    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d-extensions-socket-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:43:37.463694 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:37.463694    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rl9qw\" (UniqueName: \"kubernetes.io/projected/52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d-kube-api-access-rl9qw\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:43:37.933636 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:37.933549    2561 generic.go:356] "Generic (PLEG): container finished" podID="52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d" containerID="c5d1f900fe2c1dad27c305a9c44d03bc602382295275adee33ee34bc1d28d246" exitCode=0
Jul 24 15:43:37.933636 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:37.933592    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"
Jul 24 15:43:37.934081 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:37.933635    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk" event={"ID":"52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d","Type":"ContainerDied","Data":"c5d1f900fe2c1dad27c305a9c44d03bc602382295275adee33ee34bc1d28d246"}
Jul 24 15:43:37.934081 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:37.933701    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk" event={"ID":"52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d","Type":"ContainerDied","Data":"aae41b538fecd4e5c468afdc26304a1ae5d90b95614c25c1dc8629aa87d17789"}
Jul 24 15:43:37.934081 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:37.933724    2561 scope.go:117] "RemoveContainer" containerID="c5d1f900fe2c1dad27c305a9c44d03bc602382295275adee33ee34bc1d28d246"
Jul 24 15:43:37.942643 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:37.942626    2561 scope.go:117] "RemoveContainer" containerID="c5d1f900fe2c1dad27c305a9c44d03bc602382295275adee33ee34bc1d28d246"
Jul 24 15:43:37.942955 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:43:37.942937    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5d1f900fe2c1dad27c305a9c44d03bc602382295275adee33ee34bc1d28d246\": container with ID starting with c5d1f900fe2c1dad27c305a9c44d03bc602382295275adee33ee34bc1d28d246 not found: ID does not exist" containerID="c5d1f900fe2c1dad27c305a9c44d03bc602382295275adee33ee34bc1d28d246"
Jul 24 15:43:37.943016 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:37.942963    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5d1f900fe2c1dad27c305a9c44d03bc602382295275adee33ee34bc1d28d246"} err="failed to get container status \"c5d1f900fe2c1dad27c305a9c44d03bc602382295275adee33ee34bc1d28d246\": rpc error: code = NotFound desc = could not find container \"c5d1f900fe2c1dad27c305a9c44d03bc602382295275adee33ee34bc1d28d246\": container with ID starting with c5d1f900fe2c1dad27c305a9c44d03bc602382295275adee33ee34bc1d28d246 not found: ID does not exist"
Jul 24 15:43:37.960346 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:37.960314    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"]
Jul 24 15:43:37.970785 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:37.970757    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lrkjk"]
Jul 24 15:43:38.105897 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:38.105857    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d" path="/var/lib/kubelet/pods/52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d/volumes"
Jul 24 15:43:55.419970 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:55.419933    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"]
Jul 24 15:43:55.420395 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:55.420238    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d" containerName="manager"
Jul 24 15:43:55.420395 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:55.420250    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d" containerName="manager"
Jul 24 15:43:55.420395 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:55.420314    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="52f0c4f3-ba07-4c67-99dc-f478bb1c5b1d" containerName="manager"
Jul 24 15:43:55.425902 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:55.425880    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"
Jul 24 15:43:55.428250 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:55.428226    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 24 15:43:55.428378 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:55.428229    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9ds7q\""
Jul 24 15:43:55.431934 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:55.431908    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"]
Jul 24 15:43:55.518075 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:55.518040    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"]
Jul 24 15:43:55.617983 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:55.617949    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ad13dc8f-055d-41fa-ba91-13d119ecfbb7-config-file\") pod \"limitador-limitador-64c94fd64f-w5b9p\" (UID: \"ad13dc8f-055d-41fa-ba91-13d119ecfbb7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"
Jul 24 15:43:55.618196 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:55.617988    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj92p\" (UniqueName: \"kubernetes.io/projected/ad13dc8f-055d-41fa-ba91-13d119ecfbb7-kube-api-access-jj92p\") pod \"limitador-limitador-64c94fd64f-w5b9p\" (UID: \"ad13dc8f-055d-41fa-ba91-13d119ecfbb7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"
Jul 24 15:43:55.718597 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:55.718563    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ad13dc8f-055d-41fa-ba91-13d119ecfbb7-config-file\") pod \"limitador-limitador-64c94fd64f-w5b9p\" (UID: \"ad13dc8f-055d-41fa-ba91-13d119ecfbb7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"
Jul 24 15:43:55.718597 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:55.718598    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj92p\" (UniqueName: \"kubernetes.io/projected/ad13dc8f-055d-41fa-ba91-13d119ecfbb7-kube-api-access-jj92p\") pod \"limitador-limitador-64c94fd64f-w5b9p\" (UID: \"ad13dc8f-055d-41fa-ba91-13d119ecfbb7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"
Jul 24 15:43:55.719237 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:55.719217    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ad13dc8f-055d-41fa-ba91-13d119ecfbb7-config-file\") pod \"limitador-limitador-64c94fd64f-w5b9p\" (UID: \"ad13dc8f-055d-41fa-ba91-13d119ecfbb7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"
Jul 24 15:43:55.727455 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:55.727431    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj92p\" (UniqueName: \"kubernetes.io/projected/ad13dc8f-055d-41fa-ba91-13d119ecfbb7-kube-api-access-jj92p\") pod \"limitador-limitador-64c94fd64f-w5b9p\" (UID: \"ad13dc8f-055d-41fa-ba91-13d119ecfbb7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"
Jul 24 15:43:55.738273 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:55.738250    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"
Jul 24 15:43:55.879519 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:55.879486    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"]
Jul 24 15:43:55.882724 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:43:55.882697    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad13dc8f_055d_41fa_ba91_13d119ecfbb7.slice/crio-1e2886d6e4363f04e499fc1fff0919bc6a5c8eb104d5fa2d2fba8039cd053e16 WatchSource:0}: Error finding container 1e2886d6e4363f04e499fc1fff0919bc6a5c8eb104d5fa2d2fba8039cd053e16: Status 404 returned error can't find the container with id 1e2886d6e4363f04e499fc1fff0919bc6a5c8eb104d5fa2d2fba8039cd053e16
Jul 24 15:43:56.018821 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:56.018735    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p" event={"ID":"ad13dc8f-055d-41fa-ba91-13d119ecfbb7","Type":"ContainerStarted","Data":"1e2886d6e4363f04e499fc1fff0919bc6a5c8eb104d5fa2d2fba8039cd053e16"}
Jul 24 15:43:56.550359 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:56.550322    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-t62xq"]
Jul 24 15:43:56.553964 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:56.553925    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t62xq"
Jul 24 15:43:56.556572 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:56.556542    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bkphj\""
Jul 24 15:43:56.562261 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:56.562206    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t62xq"]
Jul 24 15:43:56.727210 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:56.727175    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvpzv\" (UniqueName: \"kubernetes.io/projected/c102a681-b5f3-48ea-9bbd-8297e91f7a0c-kube-api-access-gvpzv\") pod \"authorino-75d556bd77-t62xq\" (UID: \"c102a681-b5f3-48ea-9bbd-8297e91f7a0c\") " pod="kuadrant-system/authorino-75d556bd77-t62xq"
Jul 24 15:43:56.828830 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:56.828740    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvpzv\" (UniqueName: \"kubernetes.io/projected/c102a681-b5f3-48ea-9bbd-8297e91f7a0c-kube-api-access-gvpzv\") pod \"authorino-75d556bd77-t62xq\" (UID: \"c102a681-b5f3-48ea-9bbd-8297e91f7a0c\") " pod="kuadrant-system/authorino-75d556bd77-t62xq"
Jul 24 15:43:56.841631 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:56.841599    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvpzv\" (UniqueName: \"kubernetes.io/projected/c102a681-b5f3-48ea-9bbd-8297e91f7a0c-kube-api-access-gvpzv\") pod \"authorino-75d556bd77-t62xq\" (UID: \"c102a681-b5f3-48ea-9bbd-8297e91f7a0c\") " pod="kuadrant-system/authorino-75d556bd77-t62xq"
Jul 24 15:43:56.869338 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:56.869301    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t62xq"
Jul 24 15:43:57.052418 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:57.052113    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t62xq"]
Jul 24 15:43:57.055365 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:43:57.055333    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc102a681_b5f3_48ea_9bbd_8297e91f7a0c.slice/crio-826ac4403b12474c4352288617bddafde25fab66b7069176d0cd7fa82131eeb9 WatchSource:0}: Error finding container 826ac4403b12474c4352288617bddafde25fab66b7069176d0cd7fa82131eeb9: Status 404 returned error can't find the container with id 826ac4403b12474c4352288617bddafde25fab66b7069176d0cd7fa82131eeb9
Jul 24 15:43:58.030214 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:58.030184    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t62xq" event={"ID":"c102a681-b5f3-48ea-9bbd-8297e91f7a0c","Type":"ContainerStarted","Data":"826ac4403b12474c4352288617bddafde25fab66b7069176d0cd7fa82131eeb9"}
Jul 24 15:43:59.037822 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:59.037785    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p" event={"ID":"ad13dc8f-055d-41fa-ba91-13d119ecfbb7","Type":"ContainerStarted","Data":"e99ad835e3d1b081f98032e8260680dde0b2cfe3ac06d120a81b0daffdbeb689"}
Jul 24 15:43:59.038313 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:59.037982    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"
Jul 24 15:43:59.055307 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:43:59.055195    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p" podStartSLOduration=1.130907333 podStartE2EDuration="4.055172647s" totalImagesPullingTime="2.924265314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:43:59.053342064 +0000 UTC m=+467.464052694" watchObservedRunningTime="2026-07-24 15:43:59.055172647 +0000 UTC m=+467.465883265"
Jul 24 15:44:01.047671 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:01.047617    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t62xq" event={"ID":"c102a681-b5f3-48ea-9bbd-8297e91f7a0c","Type":"ContainerStarted","Data":"c8007e7324413eb6664e28b0bae3972ac9eaa492add508d4f1e7690bf0167a5f"}
Jul 24 15:44:01.060796 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:01.060746    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-t62xq" podStartSLOduration=1.274777756 podStartE2EDuration="5.060729586s" totalImagesPullingTime="3.78595183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:43:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:44:01.059786036 +0000 UTC m=+469.470496652" watchObservedRunningTime="2026-07-24 15:44:01.060729586 +0000 UTC m=+469.471440204"
Jul 24 15:44:10.042931 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:10.042894    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"
Jul 24 15:44:10.470899 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:10.470862    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"]
Jul 24 15:44:10.471085 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:10.471046    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p" podUID="ad13dc8f-055d-41fa-ba91-13d119ecfbb7" containerName="limitador" containerID="cri-o://e99ad835e3d1b081f98032e8260680dde0b2cfe3ac06d120a81b0daffdbeb689" gracePeriod=30
Jul 24 15:44:11.021403 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:11.021377    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"
Jul 24 15:44:11.055992 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:11.055902    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jj92p\" (UniqueName: \"kubernetes.io/projected/ad13dc8f-055d-41fa-ba91-13d119ecfbb7-kube-api-access-jj92p\") pod \"ad13dc8f-055d-41fa-ba91-13d119ecfbb7\" (UID: \"ad13dc8f-055d-41fa-ba91-13d119ecfbb7\") "
Jul 24 15:44:11.055992 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:11.055966    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ad13dc8f-055d-41fa-ba91-13d119ecfbb7-config-file\") pod \"ad13dc8f-055d-41fa-ba91-13d119ecfbb7\" (UID: \"ad13dc8f-055d-41fa-ba91-13d119ecfbb7\") "
Jul 24 15:44:11.056386 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:11.056362    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad13dc8f-055d-41fa-ba91-13d119ecfbb7-config-file" (OuterVolumeSpecName: "config-file") pod "ad13dc8f-055d-41fa-ba91-13d119ecfbb7" (UID: "ad13dc8f-055d-41fa-ba91-13d119ecfbb7"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:44:11.058247 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:11.058222    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad13dc8f-055d-41fa-ba91-13d119ecfbb7-kube-api-access-jj92p" (OuterVolumeSpecName: "kube-api-access-jj92p") pod "ad13dc8f-055d-41fa-ba91-13d119ecfbb7" (UID: "ad13dc8f-055d-41fa-ba91-13d119ecfbb7"). InnerVolumeSpecName "kube-api-access-jj92p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:44:11.092465 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:11.092435    2561 generic.go:356] "Generic (PLEG): container finished" podID="ad13dc8f-055d-41fa-ba91-13d119ecfbb7" containerID="e99ad835e3d1b081f98032e8260680dde0b2cfe3ac06d120a81b0daffdbeb689" exitCode=0
Jul 24 15:44:11.092632 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:11.092480    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"
Jul 24 15:44:11.092632 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:11.092494    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p" event={"ID":"ad13dc8f-055d-41fa-ba91-13d119ecfbb7","Type":"ContainerDied","Data":"e99ad835e3d1b081f98032e8260680dde0b2cfe3ac06d120a81b0daffdbeb689"}
Jul 24 15:44:11.092632 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:11.092536    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-w5b9p" event={"ID":"ad13dc8f-055d-41fa-ba91-13d119ecfbb7","Type":"ContainerDied","Data":"1e2886d6e4363f04e499fc1fff0919bc6a5c8eb104d5fa2d2fba8039cd053e16"}
Jul 24 15:44:11.092632 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:11.092556    2561 scope.go:117] "RemoveContainer" containerID="e99ad835e3d1b081f98032e8260680dde0b2cfe3ac06d120a81b0daffdbeb689"
Jul 24 15:44:11.101480 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:11.101455    2561 scope.go:117] "RemoveContainer" containerID="e99ad835e3d1b081f98032e8260680dde0b2cfe3ac06d120a81b0daffdbeb689"
Jul 24 15:44:11.101834 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:44:11.101816    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e99ad835e3d1b081f98032e8260680dde0b2cfe3ac06d120a81b0daffdbeb689\": container with ID starting with e99ad835e3d1b081f98032e8260680dde0b2cfe3ac06d120a81b0daffdbeb689 not found: ID does not exist" containerID="e99ad835e3d1b081f98032e8260680dde0b2cfe3ac06d120a81b0daffdbeb689"
Jul 24 15:44:11.101909 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:11.101856    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99ad835e3d1b081f98032e8260680dde0b2cfe3ac06d120a81b0daffdbeb689"} err="failed to get container status \"e99ad835e3d1b081f98032e8260680dde0b2cfe3ac06d120a81b0daffdbeb689\": rpc error: code = NotFound desc = could not find container \"e99ad835e3d1b081f98032e8260680dde0b2cfe3ac06d120a81b0daffdbeb689\": container with ID starting with e99ad835e3d1b081f98032e8260680dde0b2cfe3ac06d120a81b0daffdbeb689 not found: ID does not exist"
Jul 24 15:44:11.113108 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:11.113078    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"]
Jul 24 15:44:11.116517 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:11.116491    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-w5b9p"]
Jul 24 15:44:11.157530 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:11.157495    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jj92p\" (UniqueName: \"kubernetes.io/projected/ad13dc8f-055d-41fa-ba91-13d119ecfbb7-kube-api-access-jj92p\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:44:11.157530 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:11.157524    2561 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ad13dc8f-055d-41fa-ba91-13d119ecfbb7-config-file\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:44:12.106804 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.106774    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad13dc8f-055d-41fa-ba91-13d119ecfbb7" path="/var/lib/kubelet/pods/ad13dc8f-055d-41fa-ba91-13d119ecfbb7/volumes"
Jul 24 15:44:12.107154 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.107031    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-f798k"]
Jul 24 15:44:12.107324 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.107312    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad13dc8f-055d-41fa-ba91-13d119ecfbb7" containerName="limitador"
Jul 24 15:44:12.107359 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.107327    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad13dc8f-055d-41fa-ba91-13d119ecfbb7" containerName="limitador"
Jul 24 15:44:12.107392 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.107383    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad13dc8f-055d-41fa-ba91-13d119ecfbb7" containerName="limitador"
Jul 24 15:44:12.111464 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.111440    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f798k"
Jul 24 15:44:12.114357 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.114331    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 24 15:44:12.114803 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.114436    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 15:44:12.115178 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.115160    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 24 15:44:12.115305 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.115283    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-fd92n\""
Jul 24 15:44:12.120867 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.120831    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-f798k"]
Jul 24 15:44:12.165825 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.165789    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnhjt\" (UniqueName: \"kubernetes.io/projected/e863f716-e88f-4b89-9fe8-9a47157fed96-kube-api-access-gnhjt\") pod \"postgres-69fb4594cf-f798k\" (UID: \"e863f716-e88f-4b89-9fe8-9a47157fed96\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f798k"
Jul 24 15:44:12.166008 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.165833    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e863f716-e88f-4b89-9fe8-9a47157fed96-data\") pod \"postgres-69fb4594cf-f798k\" (UID: \"e863f716-e88f-4b89-9fe8-9a47157fed96\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f798k"
Jul 24 15:44:12.266247 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.266204    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnhjt\" (UniqueName: \"kubernetes.io/projected/e863f716-e88f-4b89-9fe8-9a47157fed96-kube-api-access-gnhjt\") pod \"postgres-69fb4594cf-f798k\" (UID: \"e863f716-e88f-4b89-9fe8-9a47157fed96\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f798k"
Jul 24 15:44:12.266438 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.266257    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e863f716-e88f-4b89-9fe8-9a47157fed96-data\") pod \"postgres-69fb4594cf-f798k\" (UID: \"e863f716-e88f-4b89-9fe8-9a47157fed96\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f798k"
Jul 24 15:44:12.266730 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.266708    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e863f716-e88f-4b89-9fe8-9a47157fed96-data\") pod \"postgres-69fb4594cf-f798k\" (UID: \"e863f716-e88f-4b89-9fe8-9a47157fed96\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f798k"
Jul 24 15:44:12.274429 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.274394    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnhjt\" (UniqueName: \"kubernetes.io/projected/e863f716-e88f-4b89-9fe8-9a47157fed96-kube-api-access-gnhjt\") pod \"postgres-69fb4594cf-f798k\" (UID: \"e863f716-e88f-4b89-9fe8-9a47157fed96\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f798k"
Jul 24 15:44:12.428125 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.428075    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f798k"
Jul 24 15:44:12.571628 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:12.571591    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-f798k"]
Jul 24 15:44:12.574520 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:44:12.574490    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode863f716_e88f_4b89_9fe8_9a47157fed96.slice/crio-1edca91912f6e8744c71ac23e95f8a3dc936ea83c4d0db3c91c37c633a6156da WatchSource:0}: Error finding container 1edca91912f6e8744c71ac23e95f8a3dc936ea83c4d0db3c91c37c633a6156da: Status 404 returned error can't find the container with id 1edca91912f6e8744c71ac23e95f8a3dc936ea83c4d0db3c91c37c633a6156da
Jul 24 15:44:13.103322 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:13.103280    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f798k" event={"ID":"e863f716-e88f-4b89-9fe8-9a47157fed96","Type":"ContainerStarted","Data":"1edca91912f6e8744c71ac23e95f8a3dc936ea83c4d0db3c91c37c633a6156da"}
Jul 24 15:44:18.129004 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:18.128961    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f798k" event={"ID":"e863f716-e88f-4b89-9fe8-9a47157fed96","Type":"ContainerStarted","Data":"41aec94a0b5a1ad6c4fd5c003e0732efe6b7825caa7fc13bd38d5c9ffae3f5dd"}
Jul 24 15:44:18.129452 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:18.129115    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f798k"
Jul 24 15:44:18.145481 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:18.145429    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f798k" podStartSLOduration=1.175243957 podStartE2EDuration="6.145412343s" totalImagesPullingTime="4.970168386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:44:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:44:18.144584684 +0000 UTC m=+486.555295301" watchObservedRunningTime="2026-07-24 15:44:18.145412343 +0000 UTC m=+486.556122959"
Jul 24 15:44:24.165233 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:24.165204    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f798k"
Jul 24 15:44:24.718934 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:24.718887    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-htlrj"]
Jul 24 15:44:24.723780 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:24.723752    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-htlrj"
Jul 24 15:44:24.728271 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:24.728241    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-htlrj"]
Jul 24 15:44:24.872206 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:24.872161    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlw8q\" (UniqueName: \"kubernetes.io/projected/32ea6b6f-bd45-4756-bcc9-f62350cc35e7-kube-api-access-xlw8q\") pod \"authorino-5d4b67c446-htlrj\" (UID: \"32ea6b6f-bd45-4756-bcc9-f62350cc35e7\") " pod="kuadrant-system/authorino-5d4b67c446-htlrj"
Jul 24 15:44:24.962095 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:24.962060    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-htlrj"]
Jul 24 15:44:24.962319 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:44:24.962300    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xlw8q], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-htlrj" podUID="32ea6b6f-bd45-4756-bcc9-f62350cc35e7"
Jul 24 15:44:24.972696 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:24.972618    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlw8q\" (UniqueName: \"kubernetes.io/projected/32ea6b6f-bd45-4756-bcc9-f62350cc35e7-kube-api-access-xlw8q\") pod \"authorino-5d4b67c446-htlrj\" (UID: \"32ea6b6f-bd45-4756-bcc9-f62350cc35e7\") " pod="kuadrant-system/authorino-5d4b67c446-htlrj"
Jul 24 15:44:24.986000 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:24.985970    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlw8q\" (UniqueName: \"kubernetes.io/projected/32ea6b6f-bd45-4756-bcc9-f62350cc35e7-kube-api-access-xlw8q\") pod \"authorino-5d4b67c446-htlrj\" (UID: \"32ea6b6f-bd45-4756-bcc9-f62350cc35e7\") " pod="kuadrant-system/authorino-5d4b67c446-htlrj"
Jul 24 15:44:24.990183 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:24.990155    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76697668cd-wxkql"]
Jul 24 15:44:24.993729 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:24.993708    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76697668cd-wxkql"
Jul 24 15:44:24.999721 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:24.999688    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76697668cd-wxkql"]
Jul 24 15:44:25.073483 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.073447    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rq2z\" (UniqueName: \"kubernetes.io/projected/d325a76d-f599-4dd7-b49e-21bcd139c89c-kube-api-access-5rq2z\") pod \"authorino-76697668cd-wxkql\" (UID: \"d325a76d-f599-4dd7-b49e-21bcd139c89c\") " pod="kuadrant-system/authorino-76697668cd-wxkql"
Jul 24 15:44:25.160577 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.160545    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-htlrj"
Jul 24 15:44:25.167263 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.167230    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76697668cd-wxkql"]
Jul 24 15:44:25.167631 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:44:25.167471    2561 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-5rq2z], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76697668cd-wxkql" podUID="d325a76d-f599-4dd7-b49e-21bcd139c89c"
Jul 24 15:44:25.167702 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.167686    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-htlrj"
Jul 24 15:44:25.173875 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.173837    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rq2z\" (UniqueName: \"kubernetes.io/projected/d325a76d-f599-4dd7-b49e-21bcd139c89c-kube-api-access-5rq2z\") pod \"authorino-76697668cd-wxkql\" (UID: \"d325a76d-f599-4dd7-b49e-21bcd139c89c\") " pod="kuadrant-system/authorino-76697668cd-wxkql"
Jul 24 15:44:25.188443 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.188411    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rq2z\" (UniqueName: \"kubernetes.io/projected/d325a76d-f599-4dd7-b49e-21bcd139c89c-kube-api-access-5rq2z\") pod \"authorino-76697668cd-wxkql\" (UID: \"d325a76d-f599-4dd7-b49e-21bcd139c89c\") " pod="kuadrant-system/authorino-76697668cd-wxkql"
Jul 24 15:44:25.196937 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.196906    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6648ddffb6-78xf4"]
Jul 24 15:44:25.200359 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.200336    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6648ddffb6-78xf4"
Jul 24 15:44:25.202689 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.202652    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 24 15:44:25.207927 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.207757    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6648ddffb6-78xf4"]
Jul 24 15:44:25.274913 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.274819    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlw8q\" (UniqueName: \"kubernetes.io/projected/32ea6b6f-bd45-4756-bcc9-f62350cc35e7-kube-api-access-xlw8q\") pod \"32ea6b6f-bd45-4756-bcc9-f62350cc35e7\" (UID: \"32ea6b6f-bd45-4756-bcc9-f62350cc35e7\") "
Jul 24 15:44:25.275153 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.275087    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4f8d534f-0d05-466c-bb7b-72c601a95fe5-tls-cert\") pod \"authorino-6648ddffb6-78xf4\" (UID: \"4f8d534f-0d05-466c-bb7b-72c601a95fe5\") " pod="kuadrant-system/authorino-6648ddffb6-78xf4"
Jul 24 15:44:25.275153 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.275119    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgs4c\" (UniqueName: \"kubernetes.io/projected/4f8d534f-0d05-466c-bb7b-72c601a95fe5-kube-api-access-tgs4c\") pod \"authorino-6648ddffb6-78xf4\" (UID: \"4f8d534f-0d05-466c-bb7b-72c601a95fe5\") " pod="kuadrant-system/authorino-6648ddffb6-78xf4"
Jul 24 15:44:25.277311 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.277284    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32ea6b6f-bd45-4756-bcc9-f62350cc35e7-kube-api-access-xlw8q" (OuterVolumeSpecName: "kube-api-access-xlw8q") pod "32ea6b6f-bd45-4756-bcc9-f62350cc35e7" (UID: "32ea6b6f-bd45-4756-bcc9-f62350cc35e7"). InnerVolumeSpecName "kube-api-access-xlw8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:44:25.375594 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.375559    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4f8d534f-0d05-466c-bb7b-72c601a95fe5-tls-cert\") pod \"authorino-6648ddffb6-78xf4\" (UID: \"4f8d534f-0d05-466c-bb7b-72c601a95fe5\") " pod="kuadrant-system/authorino-6648ddffb6-78xf4"
Jul 24 15:44:25.375594 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.375599    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgs4c\" (UniqueName: \"kubernetes.io/projected/4f8d534f-0d05-466c-bb7b-72c601a95fe5-kube-api-access-tgs4c\") pod \"authorino-6648ddffb6-78xf4\" (UID: \"4f8d534f-0d05-466c-bb7b-72c601a95fe5\") " pod="kuadrant-system/authorino-6648ddffb6-78xf4"
Jul 24 15:44:25.375868 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.375699    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlw8q\" (UniqueName: \"kubernetes.io/projected/32ea6b6f-bd45-4756-bcc9-f62350cc35e7-kube-api-access-xlw8q\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:44:25.378265 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.378244    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4f8d534f-0d05-466c-bb7b-72c601a95fe5-tls-cert\") pod \"authorino-6648ddffb6-78xf4\" (UID: \"4f8d534f-0d05-466c-bb7b-72c601a95fe5\") " pod="kuadrant-system/authorino-6648ddffb6-78xf4"
Jul 24 15:44:25.384429 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.384399    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgs4c\" (UniqueName: \"kubernetes.io/projected/4f8d534f-0d05-466c-bb7b-72c601a95fe5-kube-api-access-tgs4c\") pod \"authorino-6648ddffb6-78xf4\" (UID: \"4f8d534f-0d05-466c-bb7b-72c601a95fe5\") " pod="kuadrant-system/authorino-6648ddffb6-78xf4"
Jul 24 15:44:25.511449 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.511387    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6648ddffb6-78xf4"
Jul 24 15:44:25.651169 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:25.651142    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6648ddffb6-78xf4"]
Jul 24 15:44:25.653958 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:44:25.653931    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f8d534f_0d05_466c_bb7b_72c601a95fe5.slice/crio-f1d66aca2d88d63dbd5911b543b7aac768c3f7934a0dddb9ff5a001557c53e9c WatchSource:0}: Error finding container f1d66aca2d88d63dbd5911b543b7aac768c3f7934a0dddb9ff5a001557c53e9c: Status 404 returned error can't find the container with id f1d66aca2d88d63dbd5911b543b7aac768c3f7934a0dddb9ff5a001557c53e9c
Jul 24 15:44:26.169504 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:26.169462    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6648ddffb6-78xf4" event={"ID":"4f8d534f-0d05-466c-bb7b-72c601a95fe5","Type":"ContainerStarted","Data":"ef0005a5fd73c3872872c00333113f7b9655eb4914135938f8917aea4d373000"}
Jul 24 15:44:26.169992 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:26.169514    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6648ddffb6-78xf4" event={"ID":"4f8d534f-0d05-466c-bb7b-72c601a95fe5","Type":"ContainerStarted","Data":"f1d66aca2d88d63dbd5911b543b7aac768c3f7934a0dddb9ff5a001557c53e9c"}
Jul 24 15:44:26.169992 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:26.169556    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76697668cd-wxkql"
Jul 24 15:44:26.169992 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:26.169595    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-htlrj"
Jul 24 15:44:26.180722 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:26.180695    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76697668cd-wxkql"
Jul 24 15:44:26.186602 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:26.186542    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6648ddffb6-78xf4" podStartSLOduration=0.819337398 podStartE2EDuration="1.186525332s" totalImagesPullingTime="367.187934ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:44:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:44:26.185614149 +0000 UTC m=+494.596324769" watchObservedRunningTime="2026-07-24 15:44:26.186525332 +0000 UTC m=+494.597235949"
Jul 24 15:44:26.207105 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:26.207072    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-htlrj"]
Jul 24 15:44:26.211555 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:26.211525    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-htlrj"]
Jul 24 15:44:26.213013 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:26.212985    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t62xq"]
Jul 24 15:44:26.213198 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:26.213177    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-t62xq" podUID="c102a681-b5f3-48ea-9bbd-8297e91f7a0c" containerName="authorino" containerID="cri-o://c8007e7324413eb6664e28b0bae3972ac9eaa492add508d4f1e7690bf0167a5f" gracePeriod=30
Jul 24 15:44:26.284142 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:26.284056    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rq2z\" (UniqueName: \"kubernetes.io/projected/d325a76d-f599-4dd7-b49e-21bcd139c89c-kube-api-access-5rq2z\") pod \"d325a76d-f599-4dd7-b49e-21bcd139c89c\" (UID: \"d325a76d-f599-4dd7-b49e-21bcd139c89c\") "
Jul 24 15:44:26.286350 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:26.286315    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d325a76d-f599-4dd7-b49e-21bcd139c89c-kube-api-access-5rq2z" (OuterVolumeSpecName: "kube-api-access-5rq2z") pod "d325a76d-f599-4dd7-b49e-21bcd139c89c" (UID: "d325a76d-f599-4dd7-b49e-21bcd139c89c"). InnerVolumeSpecName "kube-api-access-5rq2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:44:26.385111 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:26.385081    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rq2z\" (UniqueName: \"kubernetes.io/projected/d325a76d-f599-4dd7-b49e-21bcd139c89c-kube-api-access-5rq2z\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:44:26.466550 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:26.466513    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t62xq"
Jul 24 15:44:26.587150 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:26.587054    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gvpzv\" (UniqueName: \"kubernetes.io/projected/c102a681-b5f3-48ea-9bbd-8297e91f7a0c-kube-api-access-gvpzv\") pod \"c102a681-b5f3-48ea-9bbd-8297e91f7a0c\" (UID: \"c102a681-b5f3-48ea-9bbd-8297e91f7a0c\") "
Jul 24 15:44:26.589214 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:26.589187    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c102a681-b5f3-48ea-9bbd-8297e91f7a0c-kube-api-access-gvpzv" (OuterVolumeSpecName: "kube-api-access-gvpzv") pod "c102a681-b5f3-48ea-9bbd-8297e91f7a0c" (UID: "c102a681-b5f3-48ea-9bbd-8297e91f7a0c"). InnerVolumeSpecName "kube-api-access-gvpzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:44:26.688225 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:26.688176    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvpzv\" (UniqueName: \"kubernetes.io/projected/c102a681-b5f3-48ea-9bbd-8297e91f7a0c-kube-api-access-gvpzv\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:44:27.175929 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:27.175894    2561 generic.go:356] "Generic (PLEG): container finished" podID="c102a681-b5f3-48ea-9bbd-8297e91f7a0c" containerID="c8007e7324413eb6664e28b0bae3972ac9eaa492add508d4f1e7690bf0167a5f" exitCode=0
Jul 24 15:44:27.176348 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:27.175940    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t62xq"
Jul 24 15:44:27.176348 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:27.175985    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t62xq" event={"ID":"c102a681-b5f3-48ea-9bbd-8297e91f7a0c","Type":"ContainerDied","Data":"c8007e7324413eb6664e28b0bae3972ac9eaa492add508d4f1e7690bf0167a5f"}
Jul 24 15:44:27.176348 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:27.176025    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t62xq" event={"ID":"c102a681-b5f3-48ea-9bbd-8297e91f7a0c","Type":"ContainerDied","Data":"826ac4403b12474c4352288617bddafde25fab66b7069176d0cd7fa82131eeb9"}
Jul 24 15:44:27.176348 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:27.176040    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76697668cd-wxkql"
Jul 24 15:44:27.176348 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:27.176041    2561 scope.go:117] "RemoveContainer" containerID="c8007e7324413eb6664e28b0bae3972ac9eaa492add508d4f1e7690bf0167a5f"
Jul 24 15:44:27.185449 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:27.185395    2561 scope.go:117] "RemoveContainer" containerID="c8007e7324413eb6664e28b0bae3972ac9eaa492add508d4f1e7690bf0167a5f"
Jul 24 15:44:27.185800 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:44:27.185779    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8007e7324413eb6664e28b0bae3972ac9eaa492add508d4f1e7690bf0167a5f\": container with ID starting with c8007e7324413eb6664e28b0bae3972ac9eaa492add508d4f1e7690bf0167a5f not found: ID does not exist" containerID="c8007e7324413eb6664e28b0bae3972ac9eaa492add508d4f1e7690bf0167a5f"
Jul 24 15:44:27.185906 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:27.185808    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8007e7324413eb6664e28b0bae3972ac9eaa492add508d4f1e7690bf0167a5f"} err="failed to get container status \"c8007e7324413eb6664e28b0bae3972ac9eaa492add508d4f1e7690bf0167a5f\": rpc error: code = NotFound desc = could not find container \"c8007e7324413eb6664e28b0bae3972ac9eaa492add508d4f1e7690bf0167a5f\": container with ID starting with c8007e7324413eb6664e28b0bae3972ac9eaa492add508d4f1e7690bf0167a5f not found: ID does not exist"
Jul 24 15:44:27.205380 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:27.205347    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76697668cd-wxkql"]
Jul 24 15:44:27.209148 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:27.209123    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76697668cd-wxkql"]
Jul 24 15:44:27.219602 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:27.219549    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t62xq"]
Jul 24 15:44:27.220880 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:27.220860    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t62xq"]
Jul 24 15:44:28.106779 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:28.106744    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32ea6b6f-bd45-4756-bcc9-f62350cc35e7" path="/var/lib/kubelet/pods/32ea6b6f-bd45-4756-bcc9-f62350cc35e7/volumes"
Jul 24 15:44:28.106987 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:28.106975    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c102a681-b5f3-48ea-9bbd-8297e91f7a0c" path="/var/lib/kubelet/pods/c102a681-b5f3-48ea-9bbd-8297e91f7a0c/volumes"
Jul 24 15:44:28.107323 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:28.107310    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d325a76d-f599-4dd7-b49e-21bcd139c89c" path="/var/lib/kubelet/pods/d325a76d-f599-4dd7-b49e-21bcd139c89c/volumes"
Jul 24 15:44:31.035275 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:31.035233    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7d5b5f6484-cv672"]
Jul 24 15:44:31.035895 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:31.035743    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c102a681-b5f3-48ea-9bbd-8297e91f7a0c" containerName="authorino"
Jul 24 15:44:31.035895 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:31.035762    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c102a681-b5f3-48ea-9bbd-8297e91f7a0c" containerName="authorino"
Jul 24 15:44:31.035895 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:31.035848    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c102a681-b5f3-48ea-9bbd-8297e91f7a0c" containerName="authorino"
Jul 24 15:44:31.236214 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:31.236181    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d5b5f6484-cv672"]
Jul 24 15:44:31.236380 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:31.236304    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d5b5f6484-cv672"
Jul 24 15:44:31.238798 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:31.238772    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 15:44:31.239309 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:31.239291    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 15:44:31.239388 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:31.239339    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-g5tpr\""
Jul 24 15:44:31.330312 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:31.330208    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzhcx\" (UniqueName: \"kubernetes.io/projected/2dc1cf5b-be3c-4c32-a8a6-14328afe056d-kube-api-access-nzhcx\") pod \"maas-controller-7d5b5f6484-cv672\" (UID: \"2dc1cf5b-be3c-4c32-a8a6-14328afe056d\") " pod="opendatahub/maas-controller-7d5b5f6484-cv672"
Jul 24 15:44:31.330312 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:31.330258    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2dc1cf5b-be3c-4c32-a8a6-14328afe056d-webhook-cert\") pod \"maas-controller-7d5b5f6484-cv672\" (UID: \"2dc1cf5b-be3c-4c32-a8a6-14328afe056d\") " pod="opendatahub/maas-controller-7d5b5f6484-cv672"
Jul 24 15:44:31.430863 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:31.430831    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzhcx\" (UniqueName: \"kubernetes.io/projected/2dc1cf5b-be3c-4c32-a8a6-14328afe056d-kube-api-access-nzhcx\") pod \"maas-controller-7d5b5f6484-cv672\" (UID: \"2dc1cf5b-be3c-4c32-a8a6-14328afe056d\") " pod="opendatahub/maas-controller-7d5b5f6484-cv672"
Jul 24 15:44:31.431029 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:31.430870    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2dc1cf5b-be3c-4c32-a8a6-14328afe056d-webhook-cert\") pod \"maas-controller-7d5b5f6484-cv672\" (UID: \"2dc1cf5b-be3c-4c32-a8a6-14328afe056d\") " pod="opendatahub/maas-controller-7d5b5f6484-cv672"
Jul 24 15:44:31.433434 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:31.433404    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2dc1cf5b-be3c-4c32-a8a6-14328afe056d-webhook-cert\") pod \"maas-controller-7d5b5f6484-cv672\" (UID: \"2dc1cf5b-be3c-4c32-a8a6-14328afe056d\") " pod="opendatahub/maas-controller-7d5b5f6484-cv672"
Jul 24 15:44:31.438654 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:31.438626    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzhcx\" (UniqueName: \"kubernetes.io/projected/2dc1cf5b-be3c-4c32-a8a6-14328afe056d-kube-api-access-nzhcx\") pod \"maas-controller-7d5b5f6484-cv672\" (UID: \"2dc1cf5b-be3c-4c32-a8a6-14328afe056d\") " pod="opendatahub/maas-controller-7d5b5f6484-cv672"
Jul 24 15:44:31.546923 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:31.546885    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d5b5f6484-cv672"
Jul 24 15:44:31.702018 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:31.701985    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7d5b5f6484-cv672"]
Jul 24 15:44:31.705598 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:44:31.705565    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dc1cf5b_be3c_4c32_a8a6_14328afe056d.slice/crio-e28bc246858c746fd100c04680b93bf78dbe97781738f47b8c0b20e4d583efc4 WatchSource:0}: Error finding container e28bc246858c746fd100c04680b93bf78dbe97781738f47b8c0b20e4d583efc4: Status 404 returned error can't find the container with id e28bc246858c746fd100c04680b93bf78dbe97781738f47b8c0b20e4d583efc4
Jul 24 15:44:32.200222 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:32.200186    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d5b5f6484-cv672" event={"ID":"2dc1cf5b-be3c-4c32-a8a6-14328afe056d","Type":"ContainerStarted","Data":"e28bc246858c746fd100c04680b93bf78dbe97781738f47b8c0b20e4d583efc4"}
Jul 24 15:44:35.215970 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:35.215930    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d5b5f6484-cv672" event={"ID":"2dc1cf5b-be3c-4c32-a8a6-14328afe056d","Type":"ContainerStarted","Data":"0abbe99ceef1266139feff823503a45412951000df483effe59486fca7f8cd3b"}
Jul 24 15:44:35.216409 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:35.216007    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7d5b5f6484-cv672"
Jul 24 15:44:35.238222 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:35.238156    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7d5b5f6484-cv672" podStartSLOduration=1.664972777 podStartE2EDuration="4.238135351s" totalImagesPullingTime="2.573162574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:44:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:44:35.237806473 +0000 UTC m=+503.648517091" watchObservedRunningTime="2026-07-24 15:44:35.238135351 +0000 UTC m=+503.648845970"
Jul 24 15:44:45.830839 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:45.830803    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d5b5f6484-cv672"]
Jul 24 15:44:45.831266 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:45.831006    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7d5b5f6484-cv672" podUID="2dc1cf5b-be3c-4c32-a8a6-14328afe056d" containerName="manager" containerID="cri-o://0abbe99ceef1266139feff823503a45412951000df483effe59486fca7f8cd3b" gracePeriod=30
Jul 24 15:44:45.832900 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:45.832872    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7d5b5f6484-cv672"
Jul 24 15:44:46.086913 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:46.086845    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d5b5f6484-cv672"
Jul 24 15:44:46.262810 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:46.262779    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzhcx\" (UniqueName: \"kubernetes.io/projected/2dc1cf5b-be3c-4c32-a8a6-14328afe056d-kube-api-access-nzhcx\") pod \"2dc1cf5b-be3c-4c32-a8a6-14328afe056d\" (UID: \"2dc1cf5b-be3c-4c32-a8a6-14328afe056d\") "
Jul 24 15:44:46.263008 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:46.262824    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2dc1cf5b-be3c-4c32-a8a6-14328afe056d-webhook-cert\") pod \"2dc1cf5b-be3c-4c32-a8a6-14328afe056d\" (UID: \"2dc1cf5b-be3c-4c32-a8a6-14328afe056d\") "
Jul 24 15:44:46.264092 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:46.264041    2561 generic.go:356] "Generic (PLEG): container finished" podID="2dc1cf5b-be3c-4c32-a8a6-14328afe056d" containerID="0abbe99ceef1266139feff823503a45412951000df483effe59486fca7f8cd3b" exitCode=0
Jul 24 15:44:46.264210 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:46.264100    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7d5b5f6484-cv672"
Jul 24 15:44:46.264210 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:46.264123    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d5b5f6484-cv672" event={"ID":"2dc1cf5b-be3c-4c32-a8a6-14328afe056d","Type":"ContainerDied","Data":"0abbe99ceef1266139feff823503a45412951000df483effe59486fca7f8cd3b"}
Jul 24 15:44:46.264210 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:46.264182    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7d5b5f6484-cv672" event={"ID":"2dc1cf5b-be3c-4c32-a8a6-14328afe056d","Type":"ContainerDied","Data":"e28bc246858c746fd100c04680b93bf78dbe97781738f47b8c0b20e4d583efc4"}
Jul 24 15:44:46.264323 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:46.264209    2561 scope.go:117] "RemoveContainer" containerID="0abbe99ceef1266139feff823503a45412951000df483effe59486fca7f8cd3b"
Jul 24 15:44:46.265433 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:46.265410    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dc1cf5b-be3c-4c32-a8a6-14328afe056d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "2dc1cf5b-be3c-4c32-a8a6-14328afe056d" (UID: "2dc1cf5b-be3c-4c32-a8a6-14328afe056d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:44:46.265502 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:46.265441    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dc1cf5b-be3c-4c32-a8a6-14328afe056d-kube-api-access-nzhcx" (OuterVolumeSpecName: "kube-api-access-nzhcx") pod "2dc1cf5b-be3c-4c32-a8a6-14328afe056d" (UID: "2dc1cf5b-be3c-4c32-a8a6-14328afe056d"). InnerVolumeSpecName "kube-api-access-nzhcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:44:46.276058 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:46.276037    2561 scope.go:117] "RemoveContainer" containerID="0abbe99ceef1266139feff823503a45412951000df483effe59486fca7f8cd3b"
Jul 24 15:44:46.276378 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:44:46.276359    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0abbe99ceef1266139feff823503a45412951000df483effe59486fca7f8cd3b\": container with ID starting with 0abbe99ceef1266139feff823503a45412951000df483effe59486fca7f8cd3b not found: ID does not exist" containerID="0abbe99ceef1266139feff823503a45412951000df483effe59486fca7f8cd3b"
Jul 24 15:44:46.276425 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:46.276390    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0abbe99ceef1266139feff823503a45412951000df483effe59486fca7f8cd3b"} err="failed to get container status \"0abbe99ceef1266139feff823503a45412951000df483effe59486fca7f8cd3b\": rpc error: code = NotFound desc = could not find container \"0abbe99ceef1266139feff823503a45412951000df483effe59486fca7f8cd3b\": container with ID starting with 0abbe99ceef1266139feff823503a45412951000df483effe59486fca7f8cd3b not found: ID does not exist"
Jul 24 15:44:46.364023 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:46.363935    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzhcx\" (UniqueName: \"kubernetes.io/projected/2dc1cf5b-be3c-4c32-a8a6-14328afe056d-kube-api-access-nzhcx\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:44:46.364023 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:46.363967    2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2dc1cf5b-be3c-4c32-a8a6-14328afe056d-webhook-cert\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:44:46.587450 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:46.587409    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7d5b5f6484-cv672"]
Jul 24 15:44:46.591959 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:46.591926    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7d5b5f6484-cv672"]
Jul 24 15:44:48.106606 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:44:48.106572    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dc1cf5b-be3c-4c32-a8a6-14328afe056d" path="/var/lib/kubelet/pods/2dc1cf5b-be3c-4c32-a8a6-14328afe056d/volumes"
Jul 24 15:45:10.769371 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:10.769289    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-qf26j"]
Jul 24 15:45:10.769871 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:10.769701    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dc1cf5b-be3c-4c32-a8a6-14328afe056d" containerName="manager"
Jul 24 15:45:10.769871 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:10.769715    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dc1cf5b-be3c-4c32-a8a6-14328afe056d" containerName="manager"
Jul 24 15:45:10.769871 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:10.769782    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dc1cf5b-be3c-4c32-a8a6-14328afe056d" containerName="manager"
Jul 24 15:45:10.777465 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:10.777435    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-qf26j"
Jul 24 15:45:10.779932 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:10.779904    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 24 15:45:10.779932 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:10.779922    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-kgmvx\""
Jul 24 15:45:10.785573 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:10.785546    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-qf26j"]
Jul 24 15:45:10.857873 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:10.857820    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f22st\" (UniqueName: \"kubernetes.io/projected/fa01d160-d791-465e-9488-0ae89aca50b3-kube-api-access-f22st\") pod \"payload-processing-5cc6cd444b-qf26j\" (UID: \"fa01d160-d791-465e-9488-0ae89aca50b3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-qf26j"
Jul 24 15:45:10.858054 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:10.857906    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa01d160-d791-465e-9488-0ae89aca50b3-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-qf26j\" (UID: \"fa01d160-d791-465e-9488-0ae89aca50b3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-qf26j"
Jul 24 15:45:10.959170 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:10.959136    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa01d160-d791-465e-9488-0ae89aca50b3-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-qf26j\" (UID: \"fa01d160-d791-465e-9488-0ae89aca50b3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-qf26j"
Jul 24 15:45:10.959361 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:10.959197    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f22st\" (UniqueName: \"kubernetes.io/projected/fa01d160-d791-465e-9488-0ae89aca50b3-kube-api-access-f22st\") pod \"payload-processing-5cc6cd444b-qf26j\" (UID: \"fa01d160-d791-465e-9488-0ae89aca50b3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-qf26j"
Jul 24 15:45:10.959896 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:10.959876    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa01d160-d791-465e-9488-0ae89aca50b3-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-qf26j\" (UID: \"fa01d160-d791-465e-9488-0ae89aca50b3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-qf26j"
Jul 24 15:45:10.967136 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:10.967103    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f22st\" (UniqueName: \"kubernetes.io/projected/fa01d160-d791-465e-9488-0ae89aca50b3-kube-api-access-f22st\") pod \"payload-processing-5cc6cd444b-qf26j\" (UID: \"fa01d160-d791-465e-9488-0ae89aca50b3\") " pod="openshift-ingress/payload-processing-5cc6cd444b-qf26j"
Jul 24 15:45:11.089930 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:11.089836    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-qf26j"
Jul 24 15:45:11.232702 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:11.232676    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-qf26j"]
Jul 24 15:45:11.235573 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:45:11.235540    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa01d160_d791_465e_9488_0ae89aca50b3.slice/crio-d8a7ae74a921b3c7df540052fc4045cf569a6d6e517aad19673c4d1502ff8d1b WatchSource:0}: Error finding container d8a7ae74a921b3c7df540052fc4045cf569a6d6e517aad19673c4d1502ff8d1b: Status 404 returned error can't find the container with id d8a7ae74a921b3c7df540052fc4045cf569a6d6e517aad19673c4d1502ff8d1b
Jul 24 15:45:11.373934 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:11.373858    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-qf26j" event={"ID":"fa01d160-d791-465e-9488-0ae89aca50b3","Type":"ContainerStarted","Data":"d8a7ae74a921b3c7df540052fc4045cf569a6d6e517aad19673c4d1502ff8d1b"}
Jul 24 15:45:14.391533 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:14.391490    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-qf26j" event={"ID":"fa01d160-d791-465e-9488-0ae89aca50b3","Type":"ContainerStarted","Data":"4567adb44fc97006539109db904dfddaa4860588945bb21d7062acd1a144dde1"}
Jul 24 15:45:14.391984 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:14.391597    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-qf26j"
Jul 24 15:45:14.409093 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:14.409037    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-qf26j" podStartSLOduration=1.830699895 podStartE2EDuration="4.409017392s" totalImagesPullingTime="2.578317497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:45:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:45:14.408568588 +0000 UTC m=+542.819279208" watchObservedRunningTime="2026-07-24 15:45:14.409017392 +0000 UTC m=+542.819728011"
Jul 24 15:45:25.397606 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:25.397578    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-qf26j"
Jul 24 15:45:38.381988 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.381951    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"]
Jul 24 15:45:38.385427 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.385403    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.388194 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.388144    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 15:45:38.388363 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.388241    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mjhhv\""
Jul 24 15:45:38.389069 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.389051    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 15:45:38.389132 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.389119    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 24 15:45:38.396911 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.396885    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"]
Jul 24 15:45:38.499824 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.499786    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/160829c5-e8bf-4ed8-9380-cc0234f2023b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.500038 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.499838    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/160829c5-e8bf-4ed8-9380-cc0234f2023b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.500038 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.499860    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/160829c5-e8bf-4ed8-9380-cc0234f2023b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.500038 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.499936    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/160829c5-e8bf-4ed8-9380-cc0234f2023b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.500038 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.499972    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gtkq\" (UniqueName: \"kubernetes.io/projected/160829c5-e8bf-4ed8-9380-cc0234f2023b-kube-api-access-8gtkq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.500187 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.500078    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/160829c5-e8bf-4ed8-9380-cc0234f2023b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.600875 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.600827    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/160829c5-e8bf-4ed8-9380-cc0234f2023b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.600875 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.600870    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/160829c5-e8bf-4ed8-9380-cc0234f2023b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.601143 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.600900    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/160829c5-e8bf-4ed8-9380-cc0234f2023b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.601143 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.600915    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/160829c5-e8bf-4ed8-9380-cc0234f2023b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.601143 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.600937    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/160829c5-e8bf-4ed8-9380-cc0234f2023b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.601143 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.600957    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gtkq\" (UniqueName: \"kubernetes.io/projected/160829c5-e8bf-4ed8-9380-cc0234f2023b-kube-api-access-8gtkq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.601336 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.601313    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/160829c5-e8bf-4ed8-9380-cc0234f2023b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.601391 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.601314    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/160829c5-e8bf-4ed8-9380-cc0234f2023b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.601453 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.601424    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/160829c5-e8bf-4ed8-9380-cc0234f2023b-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.603574 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.603539    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/160829c5-e8bf-4ed8-9380-cc0234f2023b-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.603754 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.603738    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/160829c5-e8bf-4ed8-9380-cc0234f2023b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.609280 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.609231    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gtkq\" (UniqueName: \"kubernetes.io/projected/160829c5-e8bf-4ed8-9380-cc0234f2023b-kube-api-access-8gtkq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5\" (UID: \"160829c5-e8bf-4ed8-9380-cc0234f2023b\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.696798 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.696705    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:38.844084 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:38.844035    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"]
Jul 24 15:45:38.846952 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:45:38.846921    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod160829c5_e8bf_4ed8_9380_cc0234f2023b.slice/crio-854114f06558cb980c4657bef8e276aec863ac680101f0b1d1463bd74bffa505 WatchSource:0}: Error finding container 854114f06558cb980c4657bef8e276aec863ac680101f0b1d1463bd74bffa505: Status 404 returned error can't find the container with id 854114f06558cb980c4657bef8e276aec863ac680101f0b1d1463bd74bffa505
Jul 24 15:45:39.504897 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:39.504851    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5" event={"ID":"160829c5-e8bf-4ed8-9380-cc0234f2023b","Type":"ContainerStarted","Data":"854114f06558cb980c4657bef8e276aec863ac680101f0b1d1463bd74bffa505"}
Jul 24 15:45:40.510229 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:40.510192    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5" event={"ID":"160829c5-e8bf-4ed8-9380-cc0234f2023b","Type":"ContainerStarted","Data":"56e9f8e6ca17c8ac350f609414f6edca3af1c1190c951d1e2e4ff05af3735d15"}
Jul 24 15:45:40.529379 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:40.529321    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5" podStartSLOduration=1.3857584840000001 podStartE2EDuration="2.52930598s" totalImagesPullingTime="1.143547496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:45:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:45:40.528192087 +0000 UTC m=+568.938902703" watchObservedRunningTime="2026-07-24 15:45:40.52930598 +0000 UTC m=+568.940016596"
Jul 24 15:45:48.696903 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:48.696868    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:48.697421 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:48.696952    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:48.709773 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:48.709733    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:49.562718 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:49.562685    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5"
Jul 24 15:45:53.891490 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:53.891457    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"]
Jul 24 15:45:53.898934 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:53.898913    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:53.901496 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:53.901473    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 24 15:45:53.904633 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:53.904604    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"]
Jul 24 15:45:53.924932 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:53.924553    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:53.924932 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:53.924605    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tng8k\" (UniqueName: \"kubernetes.io/projected/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-kube-api-access-tng8k\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:53.924932 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:53.924645    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:53.924932 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:53.924727    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:53.924932 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:53.924858    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:53.925232 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:53.924948    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:54.026144 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:54.026107    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:54.026347 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:54.026169    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:54.026347 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:54.026224    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:54.026347 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:54.026275    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:54.026347 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:54.026299    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tng8k\" (UniqueName: \"kubernetes.io/projected/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-kube-api-access-tng8k\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:54.026347 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:54.026335    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:54.026621 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:54.026601    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:54.026705 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:54.026652    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:54.026781 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:54.026763    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:54.028806 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:54.028790    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:54.028956 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:54.028940    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:54.034716 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:54.034634    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tng8k\" (UniqueName: \"kubernetes.io/projected/38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c-kube-api-access-tng8k\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wwgsn\" (UID: \"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:54.211010 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:54.210958    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:45:54.346633 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:54.346608    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"]
Jul 24 15:45:54.349206 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:45:54.349174    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38f9eab2_2be5_4fb3_b7e8_eb8001a4a09c.slice/crio-b8a49c2b03bd879178f26cc0dc5c18bc908b93d2ddfb65d01f087f6562528e9f WatchSource:0}: Error finding container b8a49c2b03bd879178f26cc0dc5c18bc908b93d2ddfb65d01f087f6562528e9f: Status 404 returned error can't find the container with id b8a49c2b03bd879178f26cc0dc5c18bc908b93d2ddfb65d01f087f6562528e9f
Jul 24 15:45:54.575383 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:54.575352    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn" event={"ID":"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c","Type":"ContainerStarted","Data":"b8a49c2b03bd879178f26cc0dc5c18bc908b93d2ddfb65d01f087f6562528e9f"}
Jul 24 15:45:55.584074 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:55.584034    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn" event={"ID":"38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c","Type":"ContainerStarted","Data":"415f72640bd69b1765bf6304cffb8c97a42fbc6801c8a31c3066e60446c8493c"}
Jul 24 15:45:55.603877 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:45:55.603821    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn" podStartSLOduration=2.4042188749999998 podStartE2EDuration="2.603804812s" totalImagesPullingTime="199.585937ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:45:55.602121253 +0000 UTC m=+584.012831871" watchObservedRunningTime="2026-07-24 15:45:55.603804812 +0000 UTC m=+584.014515427"
Jul 24 15:46:04.211219 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:46:04.211175    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:46:04.211635 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:46:04.211239    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:46:04.223652 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:46:04.223625    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:46:04.634573 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:46:04.634483    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wwgsn"
Jul 24 15:47:24.392409 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:24.392374    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f468f5-ztsjj"]
Jul 24 15:47:24.395686 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:24.395650    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f468f5-ztsjj"
Jul 24 15:47:24.402207 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:24.402181    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f468f5-ztsjj"]
Jul 24 15:47:24.426163 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:24.426115    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bv5gf\" (UniqueName: \"kubernetes.io/projected/e96645d7-a4df-4e93-bd25-19b1ad71f158-kube-api-access-bv5gf\") pod \"authorino-6f468f5-ztsjj\" (UID: \"e96645d7-a4df-4e93-bd25-19b1ad71f158\") " pod="kuadrant-system/authorino-6f468f5-ztsjj"
Jul 24 15:47:24.426345 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:24.426300    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e96645d7-a4df-4e93-bd25-19b1ad71f158-tls-cert\") pod \"authorino-6f468f5-ztsjj\" (UID: \"e96645d7-a4df-4e93-bd25-19b1ad71f158\") " pod="kuadrant-system/authorino-6f468f5-ztsjj"
Jul 24 15:47:24.527746 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:24.527701    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e96645d7-a4df-4e93-bd25-19b1ad71f158-tls-cert\") pod \"authorino-6f468f5-ztsjj\" (UID: \"e96645d7-a4df-4e93-bd25-19b1ad71f158\") " pod="kuadrant-system/authorino-6f468f5-ztsjj"
Jul 24 15:47:24.527958 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:24.527756    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bv5gf\" (UniqueName: \"kubernetes.io/projected/e96645d7-a4df-4e93-bd25-19b1ad71f158-kube-api-access-bv5gf\") pod \"authorino-6f468f5-ztsjj\" (UID: \"e96645d7-a4df-4e93-bd25-19b1ad71f158\") " pod="kuadrant-system/authorino-6f468f5-ztsjj"
Jul 24 15:47:24.530362 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:24.530333    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e96645d7-a4df-4e93-bd25-19b1ad71f158-tls-cert\") pod \"authorino-6f468f5-ztsjj\" (UID: \"e96645d7-a4df-4e93-bd25-19b1ad71f158\") " pod="kuadrant-system/authorino-6f468f5-ztsjj"
Jul 24 15:47:24.536177 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:24.536142    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bv5gf\" (UniqueName: \"kubernetes.io/projected/e96645d7-a4df-4e93-bd25-19b1ad71f158-kube-api-access-bv5gf\") pod \"authorino-6f468f5-ztsjj\" (UID: \"e96645d7-a4df-4e93-bd25-19b1ad71f158\") " pod="kuadrant-system/authorino-6f468f5-ztsjj"
Jul 24 15:47:24.705393 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:24.705346    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f468f5-ztsjj"
Jul 24 15:47:24.852468 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:24.852440    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f468f5-ztsjj"]
Jul 24 15:47:24.855529 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:47:24.855487    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode96645d7_a4df_4e93_bd25_19b1ad71f158.slice/crio-c6c6dbe7d5fc64f40c2b993c13d127b0ca546162dbeb68918767bb1eb25b70ef WatchSource:0}: Error finding container c6c6dbe7d5fc64f40c2b993c13d127b0ca546162dbeb68918767bb1eb25b70ef: Status 404 returned error can't find the container with id c6c6dbe7d5fc64f40c2b993c13d127b0ca546162dbeb68918767bb1eb25b70ef
Jul 24 15:47:24.856820 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:24.856802    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:47:24.987357 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:24.987256    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f468f5-ztsjj" event={"ID":"e96645d7-a4df-4e93-bd25-19b1ad71f158","Type":"ContainerStarted","Data":"c6c6dbe7d5fc64f40c2b993c13d127b0ca546162dbeb68918767bb1eb25b70ef"}
Jul 24 15:47:25.997064 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:25.997030    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f468f5-ztsjj" event={"ID":"e96645d7-a4df-4e93-bd25-19b1ad71f158","Type":"ContainerStarted","Data":"4618825886bb867e25820e1ed9f7b4e669f20d9654946025fa56775971e15e6e"}
Jul 24 15:47:26.019055 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:26.018955    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6f468f5-ztsjj" podStartSLOduration=1.475146251 podStartE2EDuration="2.018930091s" totalImagesPullingTime="543.78384ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:47:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:26.017532011 +0000 UTC m=+674.428242627" watchObservedRunningTime="2026-07-24 15:47:26.018930091 +0000 UTC m=+674.429640700"
Jul 24 15:47:26.050381 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:26.050345    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6648ddffb6-78xf4"]
Jul 24 15:47:26.050831 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:26.050800    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6648ddffb6-78xf4" podUID="4f8d534f-0d05-466c-bb7b-72c601a95fe5" containerName="authorino" containerID="cri-o://ef0005a5fd73c3872872c00333113f7b9655eb4914135938f8917aea4d373000" gracePeriod=30
Jul 24 15:47:26.302596 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:26.302572    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6648ddffb6-78xf4"
Jul 24 15:47:26.345796 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:26.345762    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgs4c\" (UniqueName: \"kubernetes.io/projected/4f8d534f-0d05-466c-bb7b-72c601a95fe5-kube-api-access-tgs4c\") pod \"4f8d534f-0d05-466c-bb7b-72c601a95fe5\" (UID: \"4f8d534f-0d05-466c-bb7b-72c601a95fe5\") "
Jul 24 15:47:26.346021 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:26.345913    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4f8d534f-0d05-466c-bb7b-72c601a95fe5-tls-cert\") pod \"4f8d534f-0d05-466c-bb7b-72c601a95fe5\" (UID: \"4f8d534f-0d05-466c-bb7b-72c601a95fe5\") "
Jul 24 15:47:26.348697 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:26.348644    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f8d534f-0d05-466c-bb7b-72c601a95fe5-kube-api-access-tgs4c" (OuterVolumeSpecName: "kube-api-access-tgs4c") pod "4f8d534f-0d05-466c-bb7b-72c601a95fe5" (UID: "4f8d534f-0d05-466c-bb7b-72c601a95fe5"). InnerVolumeSpecName "kube-api-access-tgs4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:47:26.357962 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:26.357935    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f8d534f-0d05-466c-bb7b-72c601a95fe5-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "4f8d534f-0d05-466c-bb7b-72c601a95fe5" (UID: "4f8d534f-0d05-466c-bb7b-72c601a95fe5"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:47:26.446903 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:26.446865    2561 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4f8d534f-0d05-466c-bb7b-72c601a95fe5-tls-cert\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:47:26.446903 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:26.446905    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgs4c\" (UniqueName: \"kubernetes.io/projected/4f8d534f-0d05-466c-bb7b-72c601a95fe5-kube-api-access-tgs4c\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:47:27.004805 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:27.004767    2561 generic.go:356] "Generic (PLEG): container finished" podID="4f8d534f-0d05-466c-bb7b-72c601a95fe5" containerID="ef0005a5fd73c3872872c00333113f7b9655eb4914135938f8917aea4d373000" exitCode=0
Jul 24 15:47:27.005277 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:27.004811    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6648ddffb6-78xf4"
Jul 24 15:47:27.005277 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:27.004861    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6648ddffb6-78xf4" event={"ID":"4f8d534f-0d05-466c-bb7b-72c601a95fe5","Type":"ContainerDied","Data":"ef0005a5fd73c3872872c00333113f7b9655eb4914135938f8917aea4d373000"}
Jul 24 15:47:27.005277 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:27.004898    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6648ddffb6-78xf4" event={"ID":"4f8d534f-0d05-466c-bb7b-72c601a95fe5","Type":"ContainerDied","Data":"f1d66aca2d88d63dbd5911b543b7aac768c3f7934a0dddb9ff5a001557c53e9c"}
Jul 24 15:47:27.005277 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:27.004917    2561 scope.go:117] "RemoveContainer" containerID="ef0005a5fd73c3872872c00333113f7b9655eb4914135938f8917aea4d373000"
Jul 24 15:47:27.014639 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:27.014622    2561 scope.go:117] "RemoveContainer" containerID="ef0005a5fd73c3872872c00333113f7b9655eb4914135938f8917aea4d373000"
Jul 24 15:47:27.014970 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:47:27.014952    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef0005a5fd73c3872872c00333113f7b9655eb4914135938f8917aea4d373000\": container with ID starting with ef0005a5fd73c3872872c00333113f7b9655eb4914135938f8917aea4d373000 not found: ID does not exist" containerID="ef0005a5fd73c3872872c00333113f7b9655eb4914135938f8917aea4d373000"
Jul 24 15:47:27.015060 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:27.014983    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef0005a5fd73c3872872c00333113f7b9655eb4914135938f8917aea4d373000"} err="failed to get container status \"ef0005a5fd73c3872872c00333113f7b9655eb4914135938f8917aea4d373000\": rpc error: code = NotFound desc = could not find container \"ef0005a5fd73c3872872c00333113f7b9655eb4914135938f8917aea4d373000\": container with ID starting with ef0005a5fd73c3872872c00333113f7b9655eb4914135938f8917aea4d373000 not found: ID does not exist"
Jul 24 15:47:27.032701 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:27.032653    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6648ddffb6-78xf4"]
Jul 24 15:47:27.040545 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:27.040507    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6648ddffb6-78xf4"]
Jul 24 15:47:28.107132 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:47:28.107100    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f8d534f-0d05-466c-bb7b-72c601a95fe5" path="/var/lib/kubelet/pods/4f8d534f-0d05-466c-bb7b-72c601a95fe5/volumes"
Jul 24 15:49:15.809092 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:15.809055    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67bb47cc5d-d8rxq"]
Jul 24 15:49:15.809550 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:15.809431    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f8d534f-0d05-466c-bb7b-72c601a95fe5" containerName="authorino"
Jul 24 15:49:15.809550 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:15.809447    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f8d534f-0d05-466c-bb7b-72c601a95fe5" containerName="authorino"
Jul 24 15:49:15.809550 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:15.809526    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f8d534f-0d05-466c-bb7b-72c601a95fe5" containerName="authorino"
Jul 24 15:49:15.812588 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:15.812566    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67bb47cc5d-d8rxq"
Jul 24 15:49:15.815599 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:15.815576    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 15:49:15.815740 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:15.815594    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-g5tpr\""
Jul 24 15:49:15.815740 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:15.815594    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 15:49:15.819589 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:15.819548    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67bb47cc5d-d8rxq"]
Jul 24 15:49:15.923408 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:15.923369    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smzv8\" (UniqueName: \"kubernetes.io/projected/ba80b77b-b0a2-4f47-ab6f-697c4d521425-kube-api-access-smzv8\") pod \"maas-controller-67bb47cc5d-d8rxq\" (UID: \"ba80b77b-b0a2-4f47-ab6f-697c4d521425\") " pod="opendatahub/maas-controller-67bb47cc5d-d8rxq"
Jul 24 15:49:15.923605 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:15.923473    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba80b77b-b0a2-4f47-ab6f-697c4d521425-webhook-cert\") pod \"maas-controller-67bb47cc5d-d8rxq\" (UID: \"ba80b77b-b0a2-4f47-ab6f-697c4d521425\") " pod="opendatahub/maas-controller-67bb47cc5d-d8rxq"
Jul 24 15:49:16.024565 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:16.024526    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smzv8\" (UniqueName: \"kubernetes.io/projected/ba80b77b-b0a2-4f47-ab6f-697c4d521425-kube-api-access-smzv8\") pod \"maas-controller-67bb47cc5d-d8rxq\" (UID: \"ba80b77b-b0a2-4f47-ab6f-697c4d521425\") " pod="opendatahub/maas-controller-67bb47cc5d-d8rxq"
Jul 24 15:49:16.024769 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:16.024623    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba80b77b-b0a2-4f47-ab6f-697c4d521425-webhook-cert\") pod \"maas-controller-67bb47cc5d-d8rxq\" (UID: \"ba80b77b-b0a2-4f47-ab6f-697c4d521425\") " pod="opendatahub/maas-controller-67bb47cc5d-d8rxq"
Jul 24 15:49:16.027244 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:16.027227    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba80b77b-b0a2-4f47-ab6f-697c4d521425-webhook-cert\") pod \"maas-controller-67bb47cc5d-d8rxq\" (UID: \"ba80b77b-b0a2-4f47-ab6f-697c4d521425\") " pod="opendatahub/maas-controller-67bb47cc5d-d8rxq"
Jul 24 15:49:16.033154 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:16.033127    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smzv8\" (UniqueName: \"kubernetes.io/projected/ba80b77b-b0a2-4f47-ab6f-697c4d521425-kube-api-access-smzv8\") pod \"maas-controller-67bb47cc5d-d8rxq\" (UID: \"ba80b77b-b0a2-4f47-ab6f-697c4d521425\") " pod="opendatahub/maas-controller-67bb47cc5d-d8rxq"
Jul 24 15:49:16.125434 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:16.125336    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67bb47cc5d-d8rxq"
Jul 24 15:49:16.263516 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:16.263480    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67bb47cc5d-d8rxq"]
Jul 24 15:49:16.266932 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:49:16.266895    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba80b77b_b0a2_4f47_ab6f_697c4d521425.slice/crio-625a2f9782e77bac901336f56b0adf3159aff649fdb1d5d8f22ed731c1f57742 WatchSource:0}: Error finding container 625a2f9782e77bac901336f56b0adf3159aff649fdb1d5d8f22ed731c1f57742: Status 404 returned error can't find the container with id 625a2f9782e77bac901336f56b0adf3159aff649fdb1d5d8f22ed731c1f57742
Jul 24 15:49:16.514025 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:16.513994    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67bb47cc5d-d8rxq" event={"ID":"ba80b77b-b0a2-4f47-ab6f-697c4d521425","Type":"ContainerStarted","Data":"625a2f9782e77bac901336f56b0adf3159aff649fdb1d5d8f22ed731c1f57742"}
Jul 24 15:49:17.520254 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:17.520220    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67bb47cc5d-d8rxq" event={"ID":"ba80b77b-b0a2-4f47-ab6f-697c4d521425","Type":"ContainerStarted","Data":"11114b633d91e993c4090350f6d965cf42d391f5cf10f26ad2565f1f017c47e3"}
Jul 24 15:49:17.520725 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:17.520356    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67bb47cc5d-d8rxq"
Jul 24 15:49:17.537335 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:17.537271    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67bb47cc5d-d8rxq" podStartSLOduration=2.094914847 podStartE2EDuration="2.537252434s" totalImagesPullingTime="442.337587ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:49:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:49:17.53528702 +0000 UTC m=+785.945997637" watchObservedRunningTime="2026-07-24 15:49:17.537252434 +0000 UTC m=+785.947963049"
Jul 24 15:49:28.527511 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:28.527481    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67bb47cc5d-d8rxq"
Jul 24 15:49:33.194968 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.194873    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67bb47cc5d-d8rxq"]
Jul 24 15:49:33.195436 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.195078    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67bb47cc5d-d8rxq" podUID="ba80b77b-b0a2-4f47-ab6f-697c4d521425" containerName="manager" containerID="cri-o://11114b633d91e993c4090350f6d965cf42d391f5cf10f26ad2565f1f017c47e3" gracePeriod=30
Jul 24 15:49:33.442841 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.442811    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67bb47cc5d-d8rxq"
Jul 24 15:49:33.571624 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.571586    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba80b77b-b0a2-4f47-ab6f-697c4d521425-webhook-cert\") pod \"ba80b77b-b0a2-4f47-ab6f-697c4d521425\" (UID: \"ba80b77b-b0a2-4f47-ab6f-697c4d521425\") "
Jul 24 15:49:33.571811 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.571643    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smzv8\" (UniqueName: \"kubernetes.io/projected/ba80b77b-b0a2-4f47-ab6f-697c4d521425-kube-api-access-smzv8\") pod \"ba80b77b-b0a2-4f47-ab6f-697c4d521425\" (UID: \"ba80b77b-b0a2-4f47-ab6f-697c4d521425\") "
Jul 24 15:49:33.573917 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.573886    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba80b77b-b0a2-4f47-ab6f-697c4d521425-kube-api-access-smzv8" (OuterVolumeSpecName: "kube-api-access-smzv8") pod "ba80b77b-b0a2-4f47-ab6f-697c4d521425" (UID: "ba80b77b-b0a2-4f47-ab6f-697c4d521425"). InnerVolumeSpecName "kube-api-access-smzv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:49:33.573917 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.573898    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba80b77b-b0a2-4f47-ab6f-697c4d521425-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ba80b77b-b0a2-4f47-ab6f-697c4d521425" (UID: "ba80b77b-b0a2-4f47-ab6f-697c4d521425"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:49:33.595173 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.595139    2561 generic.go:356] "Generic (PLEG): container finished" podID="ba80b77b-b0a2-4f47-ab6f-697c4d521425" containerID="11114b633d91e993c4090350f6d965cf42d391f5cf10f26ad2565f1f017c47e3" exitCode=0
Jul 24 15:49:33.595356 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.595192    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67bb47cc5d-d8rxq" event={"ID":"ba80b77b-b0a2-4f47-ab6f-697c4d521425","Type":"ContainerDied","Data":"11114b633d91e993c4090350f6d965cf42d391f5cf10f26ad2565f1f017c47e3"}
Jul 24 15:49:33.595356 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.595194    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67bb47cc5d-d8rxq"
Jul 24 15:49:33.595356 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.595234    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67bb47cc5d-d8rxq" event={"ID":"ba80b77b-b0a2-4f47-ab6f-697c4d521425","Type":"ContainerDied","Data":"625a2f9782e77bac901336f56b0adf3159aff649fdb1d5d8f22ed731c1f57742"}
Jul 24 15:49:33.595356 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.595256    2561 scope.go:117] "RemoveContainer" containerID="11114b633d91e993c4090350f6d965cf42d391f5cf10f26ad2565f1f017c47e3"
Jul 24 15:49:33.604370 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.604350    2561 scope.go:117] "RemoveContainer" containerID="11114b633d91e993c4090350f6d965cf42d391f5cf10f26ad2565f1f017c47e3"
Jul 24 15:49:33.604685 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:49:33.604649    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11114b633d91e993c4090350f6d965cf42d391f5cf10f26ad2565f1f017c47e3\": container with ID starting with 11114b633d91e993c4090350f6d965cf42d391f5cf10f26ad2565f1f017c47e3 not found: ID does not exist" containerID="11114b633d91e993c4090350f6d965cf42d391f5cf10f26ad2565f1f017c47e3"
Jul 24 15:49:33.604770 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.604692    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11114b633d91e993c4090350f6d965cf42d391f5cf10f26ad2565f1f017c47e3"} err="failed to get container status \"11114b633d91e993c4090350f6d965cf42d391f5cf10f26ad2565f1f017c47e3\": rpc error: code = NotFound desc = could not find container \"11114b633d91e993c4090350f6d965cf42d391f5cf10f26ad2565f1f017c47e3\": container with ID starting with 11114b633d91e993c4090350f6d965cf42d391f5cf10f26ad2565f1f017c47e3 not found: ID does not exist"
Jul 24 15:49:33.617105 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.617068    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67bb47cc5d-d8rxq"]
Jul 24 15:49:33.621057 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.621032    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67bb47cc5d-d8rxq"]
Jul 24 15:49:33.672508 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.672471    2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ba80b77b-b0a2-4f47-ab6f-697c4d521425-webhook-cert\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:49:33.672508 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:33.672498    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smzv8\" (UniqueName: \"kubernetes.io/projected/ba80b77b-b0a2-4f47-ab6f-697c4d521425-kube-api-access-smzv8\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:49:34.106486 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.106446    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba80b77b-b0a2-4f47-ab6f-697c4d521425" path="/var/lib/kubelet/pods/ba80b77b-b0a2-4f47-ab6f-697c4d521425/volumes"
Jul 24 15:49:34.441489 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.441413    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67bb47cc5d-9df4d"]
Jul 24 15:49:34.441889 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.441758    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba80b77b-b0a2-4f47-ab6f-697c4d521425" containerName="manager"
Jul 24 15:49:34.441889 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.441769    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba80b77b-b0a2-4f47-ab6f-697c4d521425" containerName="manager"
Jul 24 15:49:34.441889 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.441841    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba80b77b-b0a2-4f47-ab6f-697c4d521425" containerName="manager"
Jul 24 15:49:34.446361 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.446338    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67bb47cc5d-9df4d"
Jul 24 15:49:34.448884 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.448864    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 24 15:49:34.449644 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.449626    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 24 15:49:34.449760 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.449631    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-g5tpr\""
Jul 24 15:49:34.456394 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.456363    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67bb47cc5d-9df4d"]
Jul 24 15:49:34.580518 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.580478    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdv6w\" (UniqueName: \"kubernetes.io/projected/d11df53b-f05a-40f2-82af-22a66735a556-kube-api-access-sdv6w\") pod \"maas-controller-67bb47cc5d-9df4d\" (UID: \"d11df53b-f05a-40f2-82af-22a66735a556\") " pod="opendatahub/maas-controller-67bb47cc5d-9df4d"
Jul 24 15:49:34.580733 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.580544    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d11df53b-f05a-40f2-82af-22a66735a556-webhook-cert\") pod \"maas-controller-67bb47cc5d-9df4d\" (UID: \"d11df53b-f05a-40f2-82af-22a66735a556\") " pod="opendatahub/maas-controller-67bb47cc5d-9df4d"
Jul 24 15:49:34.681596 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.681553    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdv6w\" (UniqueName: \"kubernetes.io/projected/d11df53b-f05a-40f2-82af-22a66735a556-kube-api-access-sdv6w\") pod \"maas-controller-67bb47cc5d-9df4d\" (UID: \"d11df53b-f05a-40f2-82af-22a66735a556\") " pod="opendatahub/maas-controller-67bb47cc5d-9df4d"
Jul 24 15:49:34.681798 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.681612    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d11df53b-f05a-40f2-82af-22a66735a556-webhook-cert\") pod \"maas-controller-67bb47cc5d-9df4d\" (UID: \"d11df53b-f05a-40f2-82af-22a66735a556\") " pod="opendatahub/maas-controller-67bb47cc5d-9df4d"
Jul 24 15:49:34.684224 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.684203    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d11df53b-f05a-40f2-82af-22a66735a556-webhook-cert\") pod \"maas-controller-67bb47cc5d-9df4d\" (UID: \"d11df53b-f05a-40f2-82af-22a66735a556\") " pod="opendatahub/maas-controller-67bb47cc5d-9df4d"
Jul 24 15:49:34.690208 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.690182    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdv6w\" (UniqueName: \"kubernetes.io/projected/d11df53b-f05a-40f2-82af-22a66735a556-kube-api-access-sdv6w\") pod \"maas-controller-67bb47cc5d-9df4d\" (UID: \"d11df53b-f05a-40f2-82af-22a66735a556\") " pod="opendatahub/maas-controller-67bb47cc5d-9df4d"
Jul 24 15:49:34.758647 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.758597    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67bb47cc5d-9df4d"
Jul 24 15:49:34.903282 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:34.903249    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67bb47cc5d-9df4d"]
Jul 24 15:49:34.906011 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:49:34.905978    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd11df53b_f05a_40f2_82af_22a66735a556.slice/crio-938ea74579f7f75618d07e50beef54a2c4eb62e46478871abc8016e88578e274 WatchSource:0}: Error finding container 938ea74579f7f75618d07e50beef54a2c4eb62e46478871abc8016e88578e274: Status 404 returned error can't find the container with id 938ea74579f7f75618d07e50beef54a2c4eb62e46478871abc8016e88578e274
Jul 24 15:49:35.607582 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:35.607545    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67bb47cc5d-9df4d" event={"ID":"d11df53b-f05a-40f2-82af-22a66735a556","Type":"ContainerStarted","Data":"7e5949e547c909fa23ec8f8732ae923401c525498cf9e2b6e876c6f0037823b1"}
Jul 24 15:49:35.608028 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:35.607612    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67bb47cc5d-9df4d" event={"ID":"d11df53b-f05a-40f2-82af-22a66735a556","Type":"ContainerStarted","Data":"938ea74579f7f75618d07e50beef54a2c4eb62e46478871abc8016e88578e274"}
Jul 24 15:49:35.608028 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:35.607745    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67bb47cc5d-9df4d"
Jul 24 15:49:35.624298 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:35.624246    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67bb47cc5d-9df4d" podStartSLOduration=1.053076181 podStartE2EDuration="1.624225354s" totalImagesPullingTime="571.149173ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:49:35.62255462 +0000 UTC m=+804.033265237" watchObservedRunningTime="2026-07-24 15:49:35.624225354 +0000 UTC m=+804.034935969"
Jul 24 15:49:46.614180 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:49:46.614144    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67bb47cc5d-9df4d"
Jul 24 15:54:28.045024 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:28.044981    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"]
Jul 24 15:54:28.047694 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:28.045255    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5" podUID="b7dbdb02-a636-4be2-addf-70d7f4e1ed87" containerName="manager" containerID="cri-o://fe5096cd621ea0d221c6b861c1962b11199cd74dfdf5d5855848d2788ab7a458" gracePeriod=10
Jul 24 15:54:31.198163 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:31.198132    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"
Jul 24 15:54:31.269287 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:31.269258    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnwtb\" (UniqueName: \"kubernetes.io/projected/b7dbdb02-a636-4be2-addf-70d7f4e1ed87-kube-api-access-gnwtb\") pod \"b7dbdb02-a636-4be2-addf-70d7f4e1ed87\" (UID: \"b7dbdb02-a636-4be2-addf-70d7f4e1ed87\") "
Jul 24 15:54:31.269444 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:31.269313    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7dbdb02-a636-4be2-addf-70d7f4e1ed87-extensions-socket-volume\") pod \"b7dbdb02-a636-4be2-addf-70d7f4e1ed87\" (UID: \"b7dbdb02-a636-4be2-addf-70d7f4e1ed87\") "
Jul 24 15:54:31.269684 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:31.269636    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7dbdb02-a636-4be2-addf-70d7f4e1ed87-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b7dbdb02-a636-4be2-addf-70d7f4e1ed87" (UID: "b7dbdb02-a636-4be2-addf-70d7f4e1ed87"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:54:31.271478 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:31.271456    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7dbdb02-a636-4be2-addf-70d7f4e1ed87-kube-api-access-gnwtb" (OuterVolumeSpecName: "kube-api-access-gnwtb") pod "b7dbdb02-a636-4be2-addf-70d7f4e1ed87" (UID: "b7dbdb02-a636-4be2-addf-70d7f4e1ed87"). InnerVolumeSpecName "kube-api-access-gnwtb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:54:31.370280 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:31.370178    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnwtb\" (UniqueName: \"kubernetes.io/projected/b7dbdb02-a636-4be2-addf-70d7f4e1ed87-kube-api-access-gnwtb\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:54:31.370280 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:31.370218    2561 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7dbdb02-a636-4be2-addf-70d7f4e1ed87-extensions-socket-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:54:31.989949 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:31.989914    2561 generic.go:356] "Generic (PLEG): container finished" podID="b7dbdb02-a636-4be2-addf-70d7f4e1ed87" containerID="fe5096cd621ea0d221c6b861c1962b11199cd74dfdf5d5855848d2788ab7a458" exitCode=0
Jul 24 15:54:31.990125 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:31.989962    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"
Jul 24 15:54:31.990125 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:31.990000    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5" event={"ID":"b7dbdb02-a636-4be2-addf-70d7f4e1ed87","Type":"ContainerDied","Data":"fe5096cd621ea0d221c6b861c1962b11199cd74dfdf5d5855848d2788ab7a458"}
Jul 24 15:54:31.990125 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:31.990058    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5" event={"ID":"b7dbdb02-a636-4be2-addf-70d7f4e1ed87","Type":"ContainerDied","Data":"19e606a49a48d2fafe2d4d626fb93a027cdcf516a5d93bef28a51f11509c71b0"}
Jul 24 15:54:31.990125 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:31.990084    2561 scope.go:117] "RemoveContainer" containerID="fe5096cd621ea0d221c6b861c1962b11199cd74dfdf5d5855848d2788ab7a458"
Jul 24 15:54:32.001135 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:32.001113    2561 scope.go:117] "RemoveContainer" containerID="fe5096cd621ea0d221c6b861c1962b11199cd74dfdf5d5855848d2788ab7a458"
Jul 24 15:54:32.001499 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:54:32.001477    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe5096cd621ea0d221c6b861c1962b11199cd74dfdf5d5855848d2788ab7a458\": container with ID starting with fe5096cd621ea0d221c6b861c1962b11199cd74dfdf5d5855848d2788ab7a458 not found: ID does not exist" containerID="fe5096cd621ea0d221c6b861c1962b11199cd74dfdf5d5855848d2788ab7a458"
Jul 24 15:54:32.001570 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:32.001512    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe5096cd621ea0d221c6b861c1962b11199cd74dfdf5d5855848d2788ab7a458"} err="failed to get container status \"fe5096cd621ea0d221c6b861c1962b11199cd74dfdf5d5855848d2788ab7a458\": rpc error: code = NotFound desc = could not find container \"fe5096cd621ea0d221c6b861c1962b11199cd74dfdf5d5855848d2788ab7a458\": container with ID starting with fe5096cd621ea0d221c6b861c1962b11199cd74dfdf5d5855848d2788ab7a458 not found: ID does not exist"
Jul 24 15:54:32.013681 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:32.013614    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"]
Jul 24 15:54:32.019688 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:32.019635    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2fgg5"]
Jul 24 15:54:32.107209 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:54:32.107169    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7dbdb02-a636-4be2-addf-70d7f4e1ed87" path="/var/lib/kubelet/pods/b7dbdb02-a636-4be2-addf-70d7f4e1ed87/volumes"
Jul 24 15:55:29.261970 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:29.261926    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zcg6b"]
Jul 24 15:55:29.262547 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:29.262427    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7dbdb02-a636-4be2-addf-70d7f4e1ed87" containerName="manager"
Jul 24 15:55:29.262547 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:29.262445    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7dbdb02-a636-4be2-addf-70d7f4e1ed87" containerName="manager"
Jul 24 15:55:29.262547 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:29.262545    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7dbdb02-a636-4be2-addf-70d7f4e1ed87" containerName="manager"
Jul 24 15:55:29.266761 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:29.266738    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zcg6b"
Jul 24 15:55:29.269622 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:29.269597    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-ssnjg\""
Jul 24 15:55:29.281601 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:29.281570    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zcg6b"]
Jul 24 15:55:29.391821 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:29.391780    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b5tp\" (UniqueName: \"kubernetes.io/projected/12ea3d30-44e2-460e-813a-fa7f80aae5ba-kube-api-access-6b5tp\") pod \"kuadrant-operator-controller-manager-86878c769f-zcg6b\" (UID: \"12ea3d30-44e2-460e-813a-fa7f80aae5ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zcg6b"
Jul 24 15:55:29.391821 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:29.391841    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12ea3d30-44e2-460e-813a-fa7f80aae5ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zcg6b\" (UID: \"12ea3d30-44e2-460e-813a-fa7f80aae5ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zcg6b"
Jul 24 15:55:29.492617 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:29.492573    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b5tp\" (UniqueName: \"kubernetes.io/projected/12ea3d30-44e2-460e-813a-fa7f80aae5ba-kube-api-access-6b5tp\") pod \"kuadrant-operator-controller-manager-86878c769f-zcg6b\" (UID: \"12ea3d30-44e2-460e-813a-fa7f80aae5ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zcg6b"
Jul 24 15:55:29.492617 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:29.492623    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12ea3d30-44e2-460e-813a-fa7f80aae5ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zcg6b\" (UID: \"12ea3d30-44e2-460e-813a-fa7f80aae5ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zcg6b"
Jul 24 15:55:29.492997 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:29.492979    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12ea3d30-44e2-460e-813a-fa7f80aae5ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zcg6b\" (UID: \"12ea3d30-44e2-460e-813a-fa7f80aae5ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zcg6b"
Jul 24 15:55:29.505633 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:29.505598    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b5tp\" (UniqueName: \"kubernetes.io/projected/12ea3d30-44e2-460e-813a-fa7f80aae5ba-kube-api-access-6b5tp\") pod \"kuadrant-operator-controller-manager-86878c769f-zcg6b\" (UID: \"12ea3d30-44e2-460e-813a-fa7f80aae5ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zcg6b"
Jul 24 15:55:29.577439 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:29.577345    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zcg6b"
Jul 24 15:55:29.725515 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:29.725483    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zcg6b"]
Jul 24 15:55:29.729153 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:55:29.729129    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12ea3d30_44e2_460e_813a_fa7f80aae5ba.slice/crio-7f06eb63cee2e6e4f90050720b8bd3ad5bcbe1ce4976fa0647576cfcdc566f5a WatchSource:0}: Error finding container 7f06eb63cee2e6e4f90050720b8bd3ad5bcbe1ce4976fa0647576cfcdc566f5a: Status 404 returned error can't find the container with id 7f06eb63cee2e6e4f90050720b8bd3ad5bcbe1ce4976fa0647576cfcdc566f5a
Jul 24 15:55:30.254631 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:30.254592    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zcg6b" event={"ID":"12ea3d30-44e2-460e-813a-fa7f80aae5ba","Type":"ContainerStarted","Data":"9a43a237f4ff3c161e53573f4813acdf2523600eb7abcedb47f9c122522e2770"}
Jul 24 15:55:30.254829 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:30.254674    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zcg6b" event={"ID":"12ea3d30-44e2-460e-813a-fa7f80aae5ba","Type":"ContainerStarted","Data":"7f06eb63cee2e6e4f90050720b8bd3ad5bcbe1ce4976fa0647576cfcdc566f5a"}
Jul 24 15:55:30.254829 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:30.254707    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zcg6b"
Jul 24 15:55:30.287821 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:30.284623    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zcg6b" podStartSLOduration=1.284600626 podStartE2EDuration="1.284600626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:55:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:55:30.279923512 +0000 UTC m=+1158.690634128" watchObservedRunningTime="2026-07-24 15:55:30.284600626 +0000 UTC m=+1158.695311244"
Jul 24 15:55:41.260989 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:55:41.260957    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zcg6b"
Jul 24 15:58:09.803374 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.803338    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"]
Jul 24 15:58:09.806645 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.806623    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.809248 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.809227    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-ab1ff5eb-kserve-self-signed-certs\""
Jul 24 15:58:09.823096 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.823043    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"]
Jul 24 15:58:09.880046 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.880009    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-tmp-dir\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.880046 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.880049    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-dshm\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.880309 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.880070    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-model-cache\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.880309 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.880090    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6a2cca2-235e-4264-a796-e7c17ee34e57-tls-certs\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.880309 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.880234    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-home\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.880309 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.880271    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5988w\" (UniqueName: \"kubernetes.io/projected/e6a2cca2-235e-4264-a796-e7c17ee34e57-kube-api-access-5988w\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.981230 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.981192    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6a2cca2-235e-4264-a796-e7c17ee34e57-tls-certs\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.981406 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.981254    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-home\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.981406 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.981279    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5988w\" (UniqueName: \"kubernetes.io/projected/e6a2cca2-235e-4264-a796-e7c17ee34e57-kube-api-access-5988w\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.981406 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.981337    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-tmp-dir\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.981406 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.981369    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-dshm\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.981683 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.981571    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-model-cache\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.981806 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.981745    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-home\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.981870 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.981809    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-tmp-dir\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.981936 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.981917    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-model-cache\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.983772 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.983751    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-dshm\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.984064 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.984044    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6a2cca2-235e-4264-a796-e7c17ee34e57-tls-certs\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:09.989259 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:09.989236    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5988w\" (UniqueName: \"kubernetes.io/projected/e6a2cca2-235e-4264-a796-e7c17ee34e57-kube-api-access-5988w\") pod \"e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") " pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:10.116743 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:10.116635    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:10.263224 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:10.263187    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"]
Jul 24 15:58:10.266480 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:58:10.266453    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6a2cca2_235e_4264_a796_e7c17ee34e57.slice/crio-8566d9471e1497216ef10a9d2b5e5640c13d73b2e966233cf22c746b0de3161a WatchSource:0}: Error finding container 8566d9471e1497216ef10a9d2b5e5640c13d73b2e966233cf22c746b0de3161a: Status 404 returned error can't find the container with id 8566d9471e1497216ef10a9d2b5e5640c13d73b2e966233cf22c746b0de3161a
Jul 24 15:58:11.013939 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:11.013888    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x" event={"ID":"e6a2cca2-235e-4264-a796-e7c17ee34e57","Type":"ContainerStarted","Data":"0c78876f99b948afad7a82ff5ab9e844ba4ec3d103c5fdd037bd3780b52d2100"}
Jul 24 15:58:11.013939 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:11.013946    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x" event={"ID":"e6a2cca2-235e-4264-a796-e7c17ee34e57","Type":"ContainerStarted","Data":"8566d9471e1497216ef10a9d2b5e5640c13d73b2e966233cf22c746b0de3161a"}
Jul 24 15:58:11.031928 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:11.031874    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x" podStartSLOduration=2.031857607 podStartE2EDuration="2.031857607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:58:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:58:11.030021961 +0000 UTC m=+1319.440732569" watchObservedRunningTime="2026-07-24 15:58:11.031857607 +0000 UTC m=+1319.442568223"
Jul 24 15:58:12.790438 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:12.790398    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"]
Jul 24 15:58:12.794108 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:12.794087    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:12.796617 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:12.796594    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-ab1ff5eb-kserve-self-signed-certs\""
Jul 24 15:58:12.802530 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:12.802503    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"]
Jul 24 15:58:12.909264 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:12.909225    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-model-cache\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:12.909264 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:12.909264    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjtf6\" (UniqueName: \"kubernetes.io/projected/48f62950-f7ed-4363-b7c4-2b78ad93dedf-kube-api-access-mjtf6\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:12.909495 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:12.909288    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-dshm\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:12.909495 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:12.909303    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-tmp-dir\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:12.909495 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:12.909415    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-home\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:12.909495 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:12.909482    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48f62950-f7ed-4363-b7c4-2b78ad93dedf-tls-certs\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:13.010802 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:13.010765    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-home\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:13.011012 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:13.010847    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48f62950-f7ed-4363-b7c4-2b78ad93dedf-tls-certs\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:13.011012 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:13.010902    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-model-cache\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:13.011012 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:13.010929    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjtf6\" (UniqueName: \"kubernetes.io/projected/48f62950-f7ed-4363-b7c4-2b78ad93dedf-kube-api-access-mjtf6\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:13.011012 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:13.010960    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-dshm\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:13.011012 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:13.010982    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-tmp-dir\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:13.011291 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:13.011239    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-home\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:13.011291 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:13.011250    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-model-cache\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:13.011424 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:13.011406    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-tmp-dir\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:13.013512 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:13.013487    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-dshm\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:13.013705 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:13.013686    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48f62950-f7ed-4363-b7c4-2b78ad93dedf-tls-certs\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:13.019567 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:13.019539    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjtf6\" (UniqueName: \"kubernetes.io/projected/48f62950-f7ed-4363-b7c4-2b78ad93dedf-kube-api-access-mjtf6\") pod \"e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") " pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:13.104333 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:13.104240    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:13.248201 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:13.248175    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"]
Jul 24 15:58:13.251032 ip-10-0-135-164 kubenswrapper[2561]: W0724 15:58:13.251004    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48f62950_f7ed_4363_b7c4_2b78ad93dedf.slice/crio-816869cbf2d98bfe4c924004d84604250e7eb1523552ba06e6df47e4aad291fb WatchSource:0}: Error finding container 816869cbf2d98bfe4c924004d84604250e7eb1523552ba06e6df47e4aad291fb: Status 404 returned error can't find the container with id 816869cbf2d98bfe4c924004d84604250e7eb1523552ba06e6df47e4aad291fb
Jul 24 15:58:14.030425 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:14.030387    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc" event={"ID":"48f62950-f7ed-4363-b7c4-2b78ad93dedf","Type":"ContainerStarted","Data":"dc1526519d8ff73eda6d874eff651f47f634299497ceb9a2fee10f3ba123b231"}
Jul 24 15:58:14.030425 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:14.030428    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc" event={"ID":"48f62950-f7ed-4363-b7c4-2b78ad93dedf","Type":"ContainerStarted","Data":"816869cbf2d98bfe4c924004d84604250e7eb1523552ba06e6df47e4aad291fb"}
Jul 24 15:58:14.047024 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:14.046972    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc" podStartSLOduration=2.046955005 podStartE2EDuration="2.046955005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:58:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:58:14.04554311 +0000 UTC m=+1322.456253726" watchObservedRunningTime="2026-07-24 15:58:14.046955005 +0000 UTC m=+1322.457665623"
Jul 24 15:58:20.116877 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:20.116842    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:20.117307 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:20.116892    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:20.130194 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:20.130161    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:21.079149 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:21.079121    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:23.105339 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:23.105305    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:23.105339 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:23.105347    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:23.118222 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:23.118198    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:24.097675 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:24.097629    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:42.682108 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:42.682069    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"]
Jul 24 15:58:43.783402 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:43.783364    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"]
Jul 24 15:58:57.703150 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:57.703100    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x" podUID="e6a2cca2-235e-4264-a796-e7c17ee34e57" containerName="main" containerID="cri-o://0c78876f99b948afad7a82ff5ab9e844ba4ec3d103c5fdd037bd3780b52d2100" gracePeriod=45
Jul 24 15:58:57.963803 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:57.963778    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:58.112269 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.112230    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-tmp-dir\") pod \"e6a2cca2-235e-4264-a796-e7c17ee34e57\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") "
Jul 24 15:58:58.112457 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.112277    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5988w\" (UniqueName: \"kubernetes.io/projected/e6a2cca2-235e-4264-a796-e7c17ee34e57-kube-api-access-5988w\") pod \"e6a2cca2-235e-4264-a796-e7c17ee34e57\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") "
Jul 24 15:58:58.112457 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.112318    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6a2cca2-235e-4264-a796-e7c17ee34e57-tls-certs\") pod \"e6a2cca2-235e-4264-a796-e7c17ee34e57\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") "
Jul 24 15:58:58.112457 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.112351    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-home\") pod \"e6a2cca2-235e-4264-a796-e7c17ee34e57\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") "
Jul 24 15:58:58.112457 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.112374    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-dshm\") pod \"e6a2cca2-235e-4264-a796-e7c17ee34e57\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") "
Jul 24 15:58:58.112457 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.112398    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-model-cache\") pod \"e6a2cca2-235e-4264-a796-e7c17ee34e57\" (UID: \"e6a2cca2-235e-4264-a796-e7c17ee34e57\") "
Jul 24 15:58:58.112767 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.112508    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e6a2cca2-235e-4264-a796-e7c17ee34e57" (UID: "e6a2cca2-235e-4264-a796-e7c17ee34e57"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:58:58.112767 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.112587    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-tmp-dir\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:58:58.112767 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.112630    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-home" (OuterVolumeSpecName: "home") pod "e6a2cca2-235e-4264-a796-e7c17ee34e57" (UID: "e6a2cca2-235e-4264-a796-e7c17ee34e57"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:58:58.112863 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.112835    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-model-cache" (OuterVolumeSpecName: "model-cache") pod "e6a2cca2-235e-4264-a796-e7c17ee34e57" (UID: "e6a2cca2-235e-4264-a796-e7c17ee34e57"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:58:58.114843 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.114809    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6a2cca2-235e-4264-a796-e7c17ee34e57-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e6a2cca2-235e-4264-a796-e7c17ee34e57" (UID: "e6a2cca2-235e-4264-a796-e7c17ee34e57"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:58:58.114843 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.114826    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-dshm" (OuterVolumeSpecName: "dshm") pod "e6a2cca2-235e-4264-a796-e7c17ee34e57" (UID: "e6a2cca2-235e-4264-a796-e7c17ee34e57"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:58:58.114975 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.114841    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6a2cca2-235e-4264-a796-e7c17ee34e57-kube-api-access-5988w" (OuterVolumeSpecName: "kube-api-access-5988w") pod "e6a2cca2-235e-4264-a796-e7c17ee34e57" (UID: "e6a2cca2-235e-4264-a796-e7c17ee34e57"). InnerVolumeSpecName "kube-api-access-5988w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:58:58.213379 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.213342    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6a2cca2-235e-4264-a796-e7c17ee34e57-tls-certs\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:58:58.213379 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.213371    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-home\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:58:58.213379 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.213378    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-dshm\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:58:58.213379 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.213390    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6a2cca2-235e-4264-a796-e7c17ee34e57-model-cache\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:58:58.213639 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.213403    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5988w\" (UniqueName: \"kubernetes.io/projected/e6a2cca2-235e-4264-a796-e7c17ee34e57-kube-api-access-5988w\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:58:58.241723 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.241593    2561 generic.go:356] "Generic (PLEG): container finished" podID="e6a2cca2-235e-4264-a796-e7c17ee34e57" containerID="0c78876f99b948afad7a82ff5ab9e844ba4ec3d103c5fdd037bd3780b52d2100" exitCode=0
Jul 24 15:58:58.241723 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.241642    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"
Jul 24 15:58:58.241723 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.241642    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x" event={"ID":"e6a2cca2-235e-4264-a796-e7c17ee34e57","Type":"ContainerDied","Data":"0c78876f99b948afad7a82ff5ab9e844ba4ec3d103c5fdd037bd3780b52d2100"}
Jul 24 15:58:58.241723 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.241707    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x" event={"ID":"e6a2cca2-235e-4264-a796-e7c17ee34e57","Type":"ContainerDied","Data":"8566d9471e1497216ef10a9d2b5e5640c13d73b2e966233cf22c746b0de3161a"}
Jul 24 15:58:58.242092 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.241730    2561 scope.go:117] "RemoveContainer" containerID="0c78876f99b948afad7a82ff5ab9e844ba4ec3d103c5fdd037bd3780b52d2100"
Jul 24 15:58:58.251901 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.251879    2561 scope.go:117] "RemoveContainer" containerID="0c78876f99b948afad7a82ff5ab9e844ba4ec3d103c5fdd037bd3780b52d2100"
Jul 24 15:58:58.252264 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:58:58.252242    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c78876f99b948afad7a82ff5ab9e844ba4ec3d103c5fdd037bd3780b52d2100\": container with ID starting with 0c78876f99b948afad7a82ff5ab9e844ba4ec3d103c5fdd037bd3780b52d2100 not found: ID does not exist" containerID="0c78876f99b948afad7a82ff5ab9e844ba4ec3d103c5fdd037bd3780b52d2100"
Jul 24 15:58:58.252337 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.252277    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c78876f99b948afad7a82ff5ab9e844ba4ec3d103c5fdd037bd3780b52d2100"} err="failed to get container status \"0c78876f99b948afad7a82ff5ab9e844ba4ec3d103c5fdd037bd3780b52d2100\": rpc error: code = NotFound desc = could not find container \"0c78876f99b948afad7a82ff5ab9e844ba4ec3d103c5fdd037bd3780b52d2100\": container with ID starting with 0c78876f99b948afad7a82ff5ab9e844ba4ec3d103c5fdd037bd3780b52d2100 not found: ID does not exist"
Jul 24 15:58:58.263440 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.263405    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"]
Jul 24 15:58:58.268418 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.268384    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-ab1ff5eb-kserve-855659d599-mnh9x"]
Jul 24 15:58:58.804399 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:58.804357    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc" podUID="48f62950-f7ed-4363-b7c4-2b78ad93dedf" containerName="main" containerID="cri-o://dc1526519d8ff73eda6d874eff651f47f634299497ceb9a2fee10f3ba123b231" gracePeriod=45
Jul 24 15:58:59.069191 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.069128    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:59.222941 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.222910    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-home\") pod \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") "
Jul 24 15:58:59.223122 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.222984    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-dshm\") pod \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") "
Jul 24 15:58:59.223122 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.223019    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-tmp-dir\") pod \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") "
Jul 24 15:58:59.223122 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.223080    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48f62950-f7ed-4363-b7c4-2b78ad93dedf-tls-certs\") pod \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") "
Jul 24 15:58:59.223122 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.223120    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-model-cache\") pod \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") "
Jul 24 15:58:59.223324 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.223150    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjtf6\" (UniqueName: \"kubernetes.io/projected/48f62950-f7ed-4363-b7c4-2b78ad93dedf-kube-api-access-mjtf6\") pod \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\" (UID: \"48f62950-f7ed-4363-b7c4-2b78ad93dedf\") "
Jul 24 15:58:59.223324 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.223253    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-home" (OuterVolumeSpecName: "home") pod "48f62950-f7ed-4363-b7c4-2b78ad93dedf" (UID: "48f62950-f7ed-4363-b7c4-2b78ad93dedf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:58:59.223430 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.223406    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-home\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:58:59.223430 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.223413    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "48f62950-f7ed-4363-b7c4-2b78ad93dedf" (UID: "48f62950-f7ed-4363-b7c4-2b78ad93dedf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:58:59.223526 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.223418    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-model-cache" (OuterVolumeSpecName: "model-cache") pod "48f62950-f7ed-4363-b7c4-2b78ad93dedf" (UID: "48f62950-f7ed-4363-b7c4-2b78ad93dedf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:58:59.225206 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.225178    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-dshm" (OuterVolumeSpecName: "dshm") pod "48f62950-f7ed-4363-b7c4-2b78ad93dedf" (UID: "48f62950-f7ed-4363-b7c4-2b78ad93dedf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 15:58:59.225319 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.225267    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48f62950-f7ed-4363-b7c4-2b78ad93dedf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "48f62950-f7ed-4363-b7c4-2b78ad93dedf" (UID: "48f62950-f7ed-4363-b7c4-2b78ad93dedf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:58:59.225409 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.225350    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48f62950-f7ed-4363-b7c4-2b78ad93dedf-kube-api-access-mjtf6" (OuterVolumeSpecName: "kube-api-access-mjtf6") pod "48f62950-f7ed-4363-b7c4-2b78ad93dedf" (UID: "48f62950-f7ed-4363-b7c4-2b78ad93dedf"). InnerVolumeSpecName "kube-api-access-mjtf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:58:59.248612 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.248579    2561 generic.go:356] "Generic (PLEG): container finished" podID="48f62950-f7ed-4363-b7c4-2b78ad93dedf" containerID="dc1526519d8ff73eda6d874eff651f47f634299497ceb9a2fee10f3ba123b231" exitCode=0
Jul 24 15:58:59.248799 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.248624    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"
Jul 24 15:58:59.248799 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.248639    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc" event={"ID":"48f62950-f7ed-4363-b7c4-2b78ad93dedf","Type":"ContainerDied","Data":"dc1526519d8ff73eda6d874eff651f47f634299497ceb9a2fee10f3ba123b231"}
Jul 24 15:58:59.248799 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.248692    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc" event={"ID":"48f62950-f7ed-4363-b7c4-2b78ad93dedf","Type":"ContainerDied","Data":"816869cbf2d98bfe4c924004d84604250e7eb1523552ba06e6df47e4aad291fb"}
Jul 24 15:58:59.248799 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.248708    2561 scope.go:117] "RemoveContainer" containerID="dc1526519d8ff73eda6d874eff651f47f634299497ceb9a2fee10f3ba123b231"
Jul 24 15:58:59.260157 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.260132    2561 scope.go:117] "RemoveContainer" containerID="dc1526519d8ff73eda6d874eff651f47f634299497ceb9a2fee10f3ba123b231"
Jul 24 15:58:59.260796 ip-10-0-135-164 kubenswrapper[2561]: E0724 15:58:59.260770    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc1526519d8ff73eda6d874eff651f47f634299497ceb9a2fee10f3ba123b231\": container with ID starting with dc1526519d8ff73eda6d874eff651f47f634299497ceb9a2fee10f3ba123b231 not found: ID does not exist" containerID="dc1526519d8ff73eda6d874eff651f47f634299497ceb9a2fee10f3ba123b231"
Jul 24 15:58:59.260931 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.260805    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc1526519d8ff73eda6d874eff651f47f634299497ceb9a2fee10f3ba123b231"} err="failed to get container status \"dc1526519d8ff73eda6d874eff651f47f634299497ceb9a2fee10f3ba123b231\": rpc error: code = NotFound desc = could not find container \"dc1526519d8ff73eda6d874eff651f47f634299497ceb9a2fee10f3ba123b231\": container with ID starting with dc1526519d8ff73eda6d874eff651f47f634299497ceb9a2fee10f3ba123b231 not found: ID does not exist"
Jul 24 15:58:59.269679 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.269630    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"]
Jul 24 15:58:59.273833 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.273798    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-ab1ff5eb-kserve-84c69b759f-kphkc"]
Jul 24 15:58:59.324222 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.324135    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-dshm\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:58:59.324222 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.324165    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-tmp-dir\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:58:59.324222 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.324173    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48f62950-f7ed-4363-b7c4-2b78ad93dedf-tls-certs\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:58:59.324222 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.324181    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48f62950-f7ed-4363-b7c4-2b78ad93dedf-model-cache\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:58:59.324222 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:58:59.324190    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjtf6\" (UniqueName: \"kubernetes.io/projected/48f62950-f7ed-4363-b7c4-2b78ad93dedf-kube-api-access-mjtf6\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 15:59:00.106616 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:59:00.106578    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48f62950-f7ed-4363-b7c4-2b78ad93dedf" path="/var/lib/kubelet/pods/48f62950-f7ed-4363-b7c4-2b78ad93dedf/volumes"
Jul 24 15:59:00.107168 ip-10-0-135-164 kubenswrapper[2561]: I0724 15:59:00.107149    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6a2cca2-235e-4264-a796-e7c17ee34e57" path="/var/lib/kubelet/pods/e6a2cca2-235e-4264-a796-e7c17ee34e57/volumes"
Jul 24 16:02:32.059265 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.059228    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"]
Jul 24 16:02:32.059847 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.059676    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48f62950-f7ed-4363-b7c4-2b78ad93dedf" containerName="main"
Jul 24 16:02:32.059847 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.059693    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="48f62950-f7ed-4363-b7c4-2b78ad93dedf" containerName="main"
Jul 24 16:02:32.059847 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.059724    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6a2cca2-235e-4264-a796-e7c17ee34e57" containerName="main"
Jul 24 16:02:32.059847 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.059731    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6a2cca2-235e-4264-a796-e7c17ee34e57" containerName="main"
Jul 24 16:02:32.059847 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.059836    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="48f62950-f7ed-4363-b7c4-2b78ad93dedf" containerName="main"
Jul 24 16:02:32.059847 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.059850    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6a2cca2-235e-4264-a796-e7c17ee34e57" containerName="main"
Jul 24 16:02:32.063090 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.063073    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"
Jul 24 16:02:32.065417 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.065390    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-5ce23d5e-dockercfg-l88rh\""
Jul 24 16:02:32.065583 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.065442    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-5ce23d5e\""
Jul 24 16:02:32.072951 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.072924    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"]
Jul 24 16:02:32.107305 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.107269    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"]
Jul 24 16:02:32.112573 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.112544    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"
Jul 24 16:02:32.124643 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.124615    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"]
Jul 24 16:02:32.191973 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.191938    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdg7q\" (UniqueName: \"kubernetes.io/projected/f7b3e0ce-50d0-405a-bebe-28a99fa341f9-kube-api-access-wdg7q\") pod \"payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv\" (UID: \"f7b3e0ce-50d0-405a-bebe-28a99fa341f9\") " pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"
Jul 24 16:02:32.191973 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.191983    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34d178cf-ae3c-4629-83ac-d2666103c306-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw\" (UID: \"34d178cf-ae3c-4629-83ac-d2666103c306\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"
Jul 24 16:02:32.192211 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.192061    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgj9w\" (UniqueName: \"kubernetes.io/projected/34d178cf-ae3c-4629-83ac-d2666103c306-kube-api-access-qgj9w\") pod \"payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw\" (UID: \"34d178cf-ae3c-4629-83ac-d2666103c306\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"
Jul 24 16:02:32.192211 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.192116    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7b3e0ce-50d0-405a-bebe-28a99fa341f9-plugins-config-volume\") pod \"payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv\" (UID: \"f7b3e0ce-50d0-405a-bebe-28a99fa341f9\") " pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"
Jul 24 16:02:32.292882 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.292842    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdg7q\" (UniqueName: \"kubernetes.io/projected/f7b3e0ce-50d0-405a-bebe-28a99fa341f9-kube-api-access-wdg7q\") pod \"payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv\" (UID: \"f7b3e0ce-50d0-405a-bebe-28a99fa341f9\") " pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"
Jul 24 16:02:32.293095 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.292897    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34d178cf-ae3c-4629-83ac-d2666103c306-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw\" (UID: \"34d178cf-ae3c-4629-83ac-d2666103c306\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"
Jul 24 16:02:32.293095 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.292944    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgj9w\" (UniqueName: \"kubernetes.io/projected/34d178cf-ae3c-4629-83ac-d2666103c306-kube-api-access-qgj9w\") pod \"payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw\" (UID: \"34d178cf-ae3c-4629-83ac-d2666103c306\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"
Jul 24 16:02:32.293095 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.292982    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7b3e0ce-50d0-405a-bebe-28a99fa341f9-plugins-config-volume\") pod \"payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv\" (UID: \"f7b3e0ce-50d0-405a-bebe-28a99fa341f9\") " pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"
Jul 24 16:02:32.293735 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.293689    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7b3e0ce-50d0-405a-bebe-28a99fa341f9-plugins-config-volume\") pod \"payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv\" (UID: \"f7b3e0ce-50d0-405a-bebe-28a99fa341f9\") " pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"
Jul 24 16:02:32.293735 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.293718    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34d178cf-ae3c-4629-83ac-d2666103c306-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw\" (UID: \"34d178cf-ae3c-4629-83ac-d2666103c306\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"
Jul 24 16:02:32.301847 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.301806    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdg7q\" (UniqueName: \"kubernetes.io/projected/f7b3e0ce-50d0-405a-bebe-28a99fa341f9-kube-api-access-wdg7q\") pod \"payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv\" (UID: \"f7b3e0ce-50d0-405a-bebe-28a99fa341f9\") " pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"
Jul 24 16:02:32.303029 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.303009    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgj9w\" (UniqueName: \"kubernetes.io/projected/34d178cf-ae3c-4629-83ac-d2666103c306-kube-api-access-qgj9w\") pod \"payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw\" (UID: \"34d178cf-ae3c-4629-83ac-d2666103c306\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"
Jul 24 16:02:32.375357 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.375261    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"
Jul 24 16:02:32.423905 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.423875    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"
Jul 24 16:02:32.539948 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.535817    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"]
Jul 24 16:02:32.545079 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:02:32.544969    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34d178cf_ae3c_4629_83ac_d2666103c306.slice/crio-292f02318a2d6f7ce2e5e3d7a65b14637d47e6f23a73965c81a86e7e5d181f3f WatchSource:0}: Error finding container 292f02318a2d6f7ce2e5e3d7a65b14637d47e6f23a73965c81a86e7e5d181f3f: Status 404 returned error can't find the container with id 292f02318a2d6f7ce2e5e3d7a65b14637d47e6f23a73965c81a86e7e5d181f3f
Jul 24 16:02:32.598510 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:32.598479    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"]
Jul 24 16:02:32.603859 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:02:32.603830    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7b3e0ce_50d0_405a_bebe_28a99fa341f9.slice/crio-efa1d56eddbeb722d3f4aa0c66fd9b52a514bc8497b020bd1d8cd7eb38692733 WatchSource:0}: Error finding container efa1d56eddbeb722d3f4aa0c66fd9b52a514bc8497b020bd1d8cd7eb38692733: Status 404 returned error can't find the container with id efa1d56eddbeb722d3f4aa0c66fd9b52a514bc8497b020bd1d8cd7eb38692733
Jul 24 16:02:33.263065 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:33.263025    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw" event={"ID":"34d178cf-ae3c-4629-83ac-d2666103c306","Type":"ContainerStarted","Data":"6cbd104cea20143ff45ad51119839214015304e5736b27d7dc42d5ee2a28a5ad"}
Jul 24 16:02:33.263499 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:33.263211    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"
Jul 24 16:02:33.263499 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:33.263238    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw" event={"ID":"34d178cf-ae3c-4629-83ac-d2666103c306","Type":"ContainerStarted","Data":"292f02318a2d6f7ce2e5e3d7a65b14637d47e6f23a73965c81a86e7e5d181f3f"}
Jul 24 16:02:33.264675 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:33.264638    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv" event={"ID":"f7b3e0ce-50d0-405a-bebe-28a99fa341f9","Type":"ContainerStarted","Data":"8ea8bb363eead7a4c7c2a66059f8912e01bc6b364c82d5cd367ea4f507db9ef3"}
Jul 24 16:02:33.264792 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:33.264699    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv" event={"ID":"f7b3e0ce-50d0-405a-bebe-28a99fa341f9","Type":"ContainerStarted","Data":"efa1d56eddbeb722d3f4aa0c66fd9b52a514bc8497b020bd1d8cd7eb38692733"}
Jul 24 16:02:33.264792 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:33.264773    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"
Jul 24 16:02:33.279252 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:33.279190    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw" podStartSLOduration=1.27916955 podStartE2EDuration="1.27916955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:02:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:02:33.278171857 +0000 UTC m=+1581.688882475" watchObservedRunningTime="2026-07-24 16:02:33.27916955 +0000 UTC m=+1581.689880165"
Jul 24 16:02:33.295168 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:33.295116    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv" podStartSLOduration=1.295100812 podStartE2EDuration="1.295100812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:02:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:02:33.294148829 +0000 UTC m=+1581.704859447" watchObservedRunningTime="2026-07-24 16:02:33.295100812 +0000 UTC m=+1581.705811427"
Jul 24 16:02:44.272307 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:44.272267    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"
Jul 24 16:02:44.272710 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:02:44.272353    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"
Jul 24 16:03:07.375861 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.375763    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"]
Jul 24 16:03:07.376442 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.375992    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv" podUID="f7b3e0ce-50d0-405a-bebe-28a99fa341f9" containerName="payload-processing" containerID="cri-o://8ea8bb363eead7a4c7c2a66059f8912e01bc6b364c82d5cd367ea4f507db9ef3" gracePeriod=30
Jul 24 16:03:07.386379 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.386352    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"]
Jul 24 16:03:07.386631 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.386586    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw" podUID="34d178cf-ae3c-4629-83ac-d2666103c306" containerName="payload-pre-processing" containerID="cri-o://6cbd104cea20143ff45ad51119839214015304e5736b27d7dc42d5ee2a28a5ad" gracePeriod=30
Jul 24 16:03:07.662080 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.662054    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"
Jul 24 16:03:07.664906 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.664884    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"
Jul 24 16:03:07.717319 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.717283    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdg7q\" (UniqueName: \"kubernetes.io/projected/f7b3e0ce-50d0-405a-bebe-28a99fa341f9-kube-api-access-wdg7q\") pod \"f7b3e0ce-50d0-405a-bebe-28a99fa341f9\" (UID: \"f7b3e0ce-50d0-405a-bebe-28a99fa341f9\") "
Jul 24 16:03:07.717319 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.717338    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgj9w\" (UniqueName: \"kubernetes.io/projected/34d178cf-ae3c-4629-83ac-d2666103c306-kube-api-access-qgj9w\") pod \"34d178cf-ae3c-4629-83ac-d2666103c306\" (UID: \"34d178cf-ae3c-4629-83ac-d2666103c306\") "
Jul 24 16:03:07.717593 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.717382    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7b3e0ce-50d0-405a-bebe-28a99fa341f9-plugins-config-volume\") pod \"f7b3e0ce-50d0-405a-bebe-28a99fa341f9\" (UID: \"f7b3e0ce-50d0-405a-bebe-28a99fa341f9\") "
Jul 24 16:03:07.717593 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.717410    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34d178cf-ae3c-4629-83ac-d2666103c306-plugins-config-volume\") pod \"34d178cf-ae3c-4629-83ac-d2666103c306\" (UID: \"34d178cf-ae3c-4629-83ac-d2666103c306\") "
Jul 24 16:03:07.717908 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.717858    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7b3e0ce-50d0-405a-bebe-28a99fa341f9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f7b3e0ce-50d0-405a-bebe-28a99fa341f9" (UID: "f7b3e0ce-50d0-405a-bebe-28a99fa341f9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:03:07.718033 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.717947    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34d178cf-ae3c-4629-83ac-d2666103c306-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "34d178cf-ae3c-4629-83ac-d2666103c306" (UID: "34d178cf-ae3c-4629-83ac-d2666103c306"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:03:07.719762 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.719737    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7b3e0ce-50d0-405a-bebe-28a99fa341f9-kube-api-access-wdg7q" (OuterVolumeSpecName: "kube-api-access-wdg7q") pod "f7b3e0ce-50d0-405a-bebe-28a99fa341f9" (UID: "f7b3e0ce-50d0-405a-bebe-28a99fa341f9"). InnerVolumeSpecName "kube-api-access-wdg7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:03:07.719860 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.719841    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34d178cf-ae3c-4629-83ac-d2666103c306-kube-api-access-qgj9w" (OuterVolumeSpecName: "kube-api-access-qgj9w") pod "34d178cf-ae3c-4629-83ac-d2666103c306" (UID: "34d178cf-ae3c-4629-83ac-d2666103c306"). InnerVolumeSpecName "kube-api-access-qgj9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:03:07.818143 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.818106    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdg7q\" (UniqueName: \"kubernetes.io/projected/f7b3e0ce-50d0-405a-bebe-28a99fa341f9-kube-api-access-wdg7q\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:03:07.818143 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.818137    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgj9w\" (UniqueName: \"kubernetes.io/projected/34d178cf-ae3c-4629-83ac-d2666103c306-kube-api-access-qgj9w\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:03:07.818143 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.818146    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7b3e0ce-50d0-405a-bebe-28a99fa341f9-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:03:07.818143 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:07.818154    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34d178cf-ae3c-4629-83ac-d2666103c306-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:03:08.427636 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.427598    2561 generic.go:356] "Generic (PLEG): container finished" podID="34d178cf-ae3c-4629-83ac-d2666103c306" containerID="6cbd104cea20143ff45ad51119839214015304e5736b27d7dc42d5ee2a28a5ad" exitCode=0
Jul 24 16:03:08.427636 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.427640    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"
Jul 24 16:03:08.428149 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.427687    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw" event={"ID":"34d178cf-ae3c-4629-83ac-d2666103c306","Type":"ContainerDied","Data":"6cbd104cea20143ff45ad51119839214015304e5736b27d7dc42d5ee2a28a5ad"}
Jul 24 16:03:08.428149 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.427748    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw" event={"ID":"34d178cf-ae3c-4629-83ac-d2666103c306","Type":"ContainerDied","Data":"292f02318a2d6f7ce2e5e3d7a65b14637d47e6f23a73965c81a86e7e5d181f3f"}
Jul 24 16:03:08.428149 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.427772    2561 scope.go:117] "RemoveContainer" containerID="6cbd104cea20143ff45ad51119839214015304e5736b27d7dc42d5ee2a28a5ad"
Jul 24 16:03:08.429176 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.429156    2561 generic.go:356] "Generic (PLEG): container finished" podID="f7b3e0ce-50d0-405a-bebe-28a99fa341f9" containerID="8ea8bb363eead7a4c7c2a66059f8912e01bc6b364c82d5cd367ea4f507db9ef3" exitCode=0
Jul 24 16:03:08.429417 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.429192    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv" event={"ID":"f7b3e0ce-50d0-405a-bebe-28a99fa341f9","Type":"ContainerDied","Data":"8ea8bb363eead7a4c7c2a66059f8912e01bc6b364c82d5cd367ea4f507db9ef3"}
Jul 24 16:03:08.429417 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.429195    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"
Jul 24 16:03:08.429417 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.429231    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv" event={"ID":"f7b3e0ce-50d0-405a-bebe-28a99fa341f9","Type":"ContainerDied","Data":"efa1d56eddbeb722d3f4aa0c66fd9b52a514bc8497b020bd1d8cd7eb38692733"}
Jul 24 16:03:08.436766 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.436732    2561 scope.go:117] "RemoveContainer" containerID="6cbd104cea20143ff45ad51119839214015304e5736b27d7dc42d5ee2a28a5ad"
Jul 24 16:03:08.437160 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:03:08.437136    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cbd104cea20143ff45ad51119839214015304e5736b27d7dc42d5ee2a28a5ad\": container with ID starting with 6cbd104cea20143ff45ad51119839214015304e5736b27d7dc42d5ee2a28a5ad not found: ID does not exist" containerID="6cbd104cea20143ff45ad51119839214015304e5736b27d7dc42d5ee2a28a5ad"
Jul 24 16:03:08.437248 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.437168    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cbd104cea20143ff45ad51119839214015304e5736b27d7dc42d5ee2a28a5ad"} err="failed to get container status \"6cbd104cea20143ff45ad51119839214015304e5736b27d7dc42d5ee2a28a5ad\": rpc error: code = NotFound desc = could not find container \"6cbd104cea20143ff45ad51119839214015304e5736b27d7dc42d5ee2a28a5ad\": container with ID starting with 6cbd104cea20143ff45ad51119839214015304e5736b27d7dc42d5ee2a28a5ad not found: ID does not exist"
Jul 24 16:03:08.437248 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.437186    2561 scope.go:117] "RemoveContainer" containerID="8ea8bb363eead7a4c7c2a66059f8912e01bc6b364c82d5cd367ea4f507db9ef3"
Jul 24 16:03:08.446245 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.446220    2561 scope.go:117] "RemoveContainer" containerID="8ea8bb363eead7a4c7c2a66059f8912e01bc6b364c82d5cd367ea4f507db9ef3"
Jul 24 16:03:08.446583 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:03:08.446558    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ea8bb363eead7a4c7c2a66059f8912e01bc6b364c82d5cd367ea4f507db9ef3\": container with ID starting with 8ea8bb363eead7a4c7c2a66059f8912e01bc6b364c82d5cd367ea4f507db9ef3 not found: ID does not exist" containerID="8ea8bb363eead7a4c7c2a66059f8912e01bc6b364c82d5cd367ea4f507db9ef3"
Jul 24 16:03:08.446696 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.446592    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ea8bb363eead7a4c7c2a66059f8912e01bc6b364c82d5cd367ea4f507db9ef3"} err="failed to get container status \"8ea8bb363eead7a4c7c2a66059f8912e01bc6b364c82d5cd367ea4f507db9ef3\": rpc error: code = NotFound desc = could not find container \"8ea8bb363eead7a4c7c2a66059f8912e01bc6b364c82d5cd367ea4f507db9ef3\": container with ID starting with 8ea8bb363eead7a4c7c2a66059f8912e01bc6b364c82d5cd367ea4f507db9ef3 not found: ID does not exist"
Jul 24 16:03:08.446924 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.446907    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"]
Jul 24 16:03:08.452240 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.452215    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5ce23d5e-55c88f8cf6-56cfw"]
Jul 24 16:03:08.461302 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.461256    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"]
Jul 24 16:03:08.467322 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:08.467292    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5ce23d5e-7fc459d74d-pw4zv"]
Jul 24 16:03:10.106223 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:10.106188    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34d178cf-ae3c-4629-83ac-d2666103c306" path="/var/lib/kubelet/pods/34d178cf-ae3c-4629-83ac-d2666103c306/volumes"
Jul 24 16:03:10.106603 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:10.106521    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7b3e0ce-50d0-405a-bebe-28a99fa341f9" path="/var/lib/kubelet/pods/f7b3e0ce-50d0-405a-bebe-28a99fa341f9/volumes"
Jul 24 16:03:16.273124 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.273084    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"]
Jul 24 16:03:16.273499 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.273433    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34d178cf-ae3c-4629-83ac-d2666103c306" containerName="payload-pre-processing"
Jul 24 16:03:16.273499 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.273446    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="34d178cf-ae3c-4629-83ac-d2666103c306" containerName="payload-pre-processing"
Jul 24 16:03:16.273499 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.273456    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7b3e0ce-50d0-405a-bebe-28a99fa341f9" containerName="payload-processing"
Jul 24 16:03:16.273499 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.273461    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7b3e0ce-50d0-405a-bebe-28a99fa341f9" containerName="payload-processing"
Jul 24 16:03:16.273619 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.273525    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="34d178cf-ae3c-4629-83ac-d2666103c306" containerName="payload-pre-processing"
Jul 24 16:03:16.273619 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.273535    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7b3e0ce-50d0-405a-bebe-28a99fa341f9" containerName="payload-processing"
Jul 24 16:03:16.277793 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.277773    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"
Jul 24 16:03:16.280264 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.280240    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-09aa8ba8\""
Jul 24 16:03:16.280382 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.280290    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-09aa8ba8-dockercfg-fvklq\""
Jul 24 16:03:16.286888 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.286863    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"]
Jul 24 16:03:16.395363 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.395322    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-627nd\" (UniqueName: \"kubernetes.io/projected/2720a7dd-2da4-4887-9d00-e98f83287121-kube-api-access-627nd\") pod \"payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf\" (UID: \"2720a7dd-2da4-4887-9d00-e98f83287121\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"
Jul 24 16:03:16.395543 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.395380    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2720a7dd-2da4-4887-9d00-e98f83287121-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf\" (UID: \"2720a7dd-2da4-4887-9d00-e98f83287121\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"
Jul 24 16:03:16.426011 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.425978    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"]
Jul 24 16:03:16.429364 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.429333    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"
Jul 24 16:03:16.440067 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.439220    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"]
Jul 24 16:03:16.496252 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.496216    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2720a7dd-2da4-4887-9d00-e98f83287121-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf\" (UID: \"2720a7dd-2da4-4887-9d00-e98f83287121\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"
Jul 24 16:03:16.496444 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.496289    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bd46eb5-7aba-4f85-a306-a6c4da860405-plugins-config-volume\") pod \"payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl\" (UID: \"2bd46eb5-7aba-4f85-a306-a6c4da860405\") " pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"
Jul 24 16:03:16.496444 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.496309    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wccdw\" (UniqueName: \"kubernetes.io/projected/2bd46eb5-7aba-4f85-a306-a6c4da860405-kube-api-access-wccdw\") pod \"payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl\" (UID: \"2bd46eb5-7aba-4f85-a306-a6c4da860405\") " pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"
Jul 24 16:03:16.496444 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.496339    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-627nd\" (UniqueName: \"kubernetes.io/projected/2720a7dd-2da4-4887-9d00-e98f83287121-kube-api-access-627nd\") pod \"payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf\" (UID: \"2720a7dd-2da4-4887-9d00-e98f83287121\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"
Jul 24 16:03:16.497070 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.497050    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2720a7dd-2da4-4887-9d00-e98f83287121-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf\" (UID: \"2720a7dd-2da4-4887-9d00-e98f83287121\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"
Jul 24 16:03:16.505438 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.505414    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-627nd\" (UniqueName: \"kubernetes.io/projected/2720a7dd-2da4-4887-9d00-e98f83287121-kube-api-access-627nd\") pod \"payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf\" (UID: \"2720a7dd-2da4-4887-9d00-e98f83287121\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"
Jul 24 16:03:16.589371 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.589274    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"
Jul 24 16:03:16.596894 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.596851    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bd46eb5-7aba-4f85-a306-a6c4da860405-plugins-config-volume\") pod \"payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl\" (UID: \"2bd46eb5-7aba-4f85-a306-a6c4da860405\") " pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"
Jul 24 16:03:16.597041 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.596905    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wccdw\" (UniqueName: \"kubernetes.io/projected/2bd46eb5-7aba-4f85-a306-a6c4da860405-kube-api-access-wccdw\") pod \"payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl\" (UID: \"2bd46eb5-7aba-4f85-a306-a6c4da860405\") " pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"
Jul 24 16:03:16.597489 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.597466    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bd46eb5-7aba-4f85-a306-a6c4da860405-plugins-config-volume\") pod \"payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl\" (UID: \"2bd46eb5-7aba-4f85-a306-a6c4da860405\") " pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"
Jul 24 16:03:16.606832 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.606802    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wccdw\" (UniqueName: \"kubernetes.io/projected/2bd46eb5-7aba-4f85-a306-a6c4da860405-kube-api-access-wccdw\") pod \"payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl\" (UID: \"2bd46eb5-7aba-4f85-a306-a6c4da860405\") " pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"
Jul 24 16:03:16.744021 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.743985    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"]
Jul 24 16:03:16.745983 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.745956    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"
Jul 24 16:03:16.749597 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:03:16.749564    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2720a7dd_2da4_4887_9d00_e98f83287121.slice/crio-8b1b16d539f01dc821f96375e57bdb43ef3a6207bfe579b791c760ad3419cceb WatchSource:0}: Error finding container 8b1b16d539f01dc821f96375e57bdb43ef3a6207bfe579b791c760ad3419cceb: Status 404 returned error can't find the container with id 8b1b16d539f01dc821f96375e57bdb43ef3a6207bfe579b791c760ad3419cceb
Jul 24 16:03:16.903848 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:16.903802    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"]
Jul 24 16:03:16.906912 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:03:16.906876    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bd46eb5_7aba_4f85_a306_a6c4da860405.slice/crio-be33b8b7aec67b94462a94ec59840ef5c7e229e4d19734b05d2ceb7bcbd96f81 WatchSource:0}: Error finding container be33b8b7aec67b94462a94ec59840ef5c7e229e4d19734b05d2ceb7bcbd96f81: Status 404 returned error can't find the container with id be33b8b7aec67b94462a94ec59840ef5c7e229e4d19734b05d2ceb7bcbd96f81
Jul 24 16:03:17.481043 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:17.481007    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl" event={"ID":"2bd46eb5-7aba-4f85-a306-a6c4da860405","Type":"ContainerStarted","Data":"b23d498bd4c29a17baa6992aabfff461330af08b546a6c5f64bba5c2c53bb347"}
Jul 24 16:03:17.481475 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:17.481066    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl" event={"ID":"2bd46eb5-7aba-4f85-a306-a6c4da860405","Type":"ContainerStarted","Data":"be33b8b7aec67b94462a94ec59840ef5c7e229e4d19734b05d2ceb7bcbd96f81"}
Jul 24 16:03:17.481475 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:17.481198    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"
Jul 24 16:03:17.482555 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:17.482530    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf" event={"ID":"2720a7dd-2da4-4887-9d00-e98f83287121","Type":"ContainerStarted","Data":"b8581dac5f1775ad4402b07ce62d7d946ccae44fd3e5d5681b95f02d45ddbfbe"}
Jul 24 16:03:17.482653 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:17.482571    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"
Jul 24 16:03:17.482653 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:17.482586    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf" event={"ID":"2720a7dd-2da4-4887-9d00-e98f83287121","Type":"ContainerStarted","Data":"8b1b16d539f01dc821f96375e57bdb43ef3a6207bfe579b791c760ad3419cceb"}
Jul 24 16:03:17.498433 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:17.498373    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl" podStartSLOduration=1.498354059 podStartE2EDuration="1.498354059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:03:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:03:17.496345903 +0000 UTC m=+1625.907056517" watchObservedRunningTime="2026-07-24 16:03:17.498354059 +0000 UTC m=+1625.909064729"
Jul 24 16:03:17.514512 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:17.514444    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf" podStartSLOduration=1.514423114 podStartE2EDuration="1.514423114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:03:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:03:17.512096692 +0000 UTC m=+1625.922807308" watchObservedRunningTime="2026-07-24 16:03:17.514423114 +0000 UTC m=+1625.925133735"
Jul 24 16:03:28.489377 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:28.489346    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"
Jul 24 16:03:28.489789 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:28.489414    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"
Jul 24 16:03:51.093054 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.093018    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"]
Jul 24 16:03:51.093711 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.093245    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl" podUID="2bd46eb5-7aba-4f85-a306-a6c4da860405" containerName="payload-processing" containerID="cri-o://b23d498bd4c29a17baa6992aabfff461330af08b546a6c5f64bba5c2c53bb347" gracePeriod=30
Jul 24 16:03:51.098195 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.098139    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"]
Jul 24 16:03:51.099012 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.098390    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf" podUID="2720a7dd-2da4-4887-9d00-e98f83287121" containerName="payload-pre-processing" containerID="cri-o://b8581dac5f1775ad4402b07ce62d7d946ccae44fd3e5d5681b95f02d45ddbfbe" gracePeriod=30
Jul 24 16:03:51.374148 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.374123    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"
Jul 24 16:03:51.385341 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.385311    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"
Jul 24 16:03:51.385501 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.385468    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-627nd\" (UniqueName: \"kubernetes.io/projected/2720a7dd-2da4-4887-9d00-e98f83287121-kube-api-access-627nd\") pod \"2720a7dd-2da4-4887-9d00-e98f83287121\" (UID: \"2720a7dd-2da4-4887-9d00-e98f83287121\") "
Jul 24 16:03:51.385556 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.385508    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wccdw\" (UniqueName: \"kubernetes.io/projected/2bd46eb5-7aba-4f85-a306-a6c4da860405-kube-api-access-wccdw\") pod \"2bd46eb5-7aba-4f85-a306-a6c4da860405\" (UID: \"2bd46eb5-7aba-4f85-a306-a6c4da860405\") "
Jul 24 16:03:51.385556 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.385528    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2720a7dd-2da4-4887-9d00-e98f83287121-plugins-config-volume\") pod \"2720a7dd-2da4-4887-9d00-e98f83287121\" (UID: \"2720a7dd-2da4-4887-9d00-e98f83287121\") "
Jul 24 16:03:51.385960 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.385934    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2720a7dd-2da4-4887-9d00-e98f83287121-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2720a7dd-2da4-4887-9d00-e98f83287121" (UID: "2720a7dd-2da4-4887-9d00-e98f83287121"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:03:51.387793 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.387771    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2720a7dd-2da4-4887-9d00-e98f83287121-kube-api-access-627nd" (OuterVolumeSpecName: "kube-api-access-627nd") pod "2720a7dd-2da4-4887-9d00-e98f83287121" (UID: "2720a7dd-2da4-4887-9d00-e98f83287121"). InnerVolumeSpecName "kube-api-access-627nd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:03:51.387868 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.387798    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bd46eb5-7aba-4f85-a306-a6c4da860405-kube-api-access-wccdw" (OuterVolumeSpecName: "kube-api-access-wccdw") pod "2bd46eb5-7aba-4f85-a306-a6c4da860405" (UID: "2bd46eb5-7aba-4f85-a306-a6c4da860405"). InnerVolumeSpecName "kube-api-access-wccdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:03:51.486257 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.486217    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bd46eb5-7aba-4f85-a306-a6c4da860405-plugins-config-volume\") pod \"2bd46eb5-7aba-4f85-a306-a6c4da860405\" (UID: \"2bd46eb5-7aba-4f85-a306-a6c4da860405\") "
Jul 24 16:03:51.486441 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.486423    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-627nd\" (UniqueName: \"kubernetes.io/projected/2720a7dd-2da4-4887-9d00-e98f83287121-kube-api-access-627nd\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:03:51.486485 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.486449    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wccdw\" (UniqueName: \"kubernetes.io/projected/2bd46eb5-7aba-4f85-a306-a6c4da860405-kube-api-access-wccdw\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:03:51.486485 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.486462    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2720a7dd-2da4-4887-9d00-e98f83287121-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:03:51.486647 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.486629    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bd46eb5-7aba-4f85-a306-a6c4da860405-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2bd46eb5-7aba-4f85-a306-a6c4da860405" (UID: "2bd46eb5-7aba-4f85-a306-a6c4da860405"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:03:51.587327 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.587285    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2bd46eb5-7aba-4f85-a306-a6c4da860405-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:03:51.658809 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.658717    2561 generic.go:356] "Generic (PLEG): container finished" podID="2720a7dd-2da4-4887-9d00-e98f83287121" containerID="b8581dac5f1775ad4402b07ce62d7d946ccae44fd3e5d5681b95f02d45ddbfbe" exitCode=0
Jul 24 16:03:51.658809 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.658756    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"
Jul 24 16:03:51.659027 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.658809    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf" event={"ID":"2720a7dd-2da4-4887-9d00-e98f83287121","Type":"ContainerDied","Data":"b8581dac5f1775ad4402b07ce62d7d946ccae44fd3e5d5681b95f02d45ddbfbe"}
Jul 24 16:03:51.659027 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.658869    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf" event={"ID":"2720a7dd-2da4-4887-9d00-e98f83287121","Type":"ContainerDied","Data":"8b1b16d539f01dc821f96375e57bdb43ef3a6207bfe579b791c760ad3419cceb"}
Jul 24 16:03:51.659027 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.658893    2561 scope.go:117] "RemoveContainer" containerID="b8581dac5f1775ad4402b07ce62d7d946ccae44fd3e5d5681b95f02d45ddbfbe"
Jul 24 16:03:51.660193 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.660170    2561 generic.go:356] "Generic (PLEG): container finished" podID="2bd46eb5-7aba-4f85-a306-a6c4da860405" containerID="b23d498bd4c29a17baa6992aabfff461330af08b546a6c5f64bba5c2c53bb347" exitCode=0
Jul 24 16:03:51.660295 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.660197    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl" event={"ID":"2bd46eb5-7aba-4f85-a306-a6c4da860405","Type":"ContainerDied","Data":"b23d498bd4c29a17baa6992aabfff461330af08b546a6c5f64bba5c2c53bb347"}
Jul 24 16:03:51.660295 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.660219    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"
Jul 24 16:03:51.660295 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.660262    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl" event={"ID":"2bd46eb5-7aba-4f85-a306-a6c4da860405","Type":"ContainerDied","Data":"be33b8b7aec67b94462a94ec59840ef5c7e229e4d19734b05d2ceb7bcbd96f81"}
Jul 24 16:03:51.670477 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.670441    2561 scope.go:117] "RemoveContainer" containerID="b8581dac5f1775ad4402b07ce62d7d946ccae44fd3e5d5681b95f02d45ddbfbe"
Jul 24 16:03:51.670879 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:03:51.670860    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8581dac5f1775ad4402b07ce62d7d946ccae44fd3e5d5681b95f02d45ddbfbe\": container with ID starting with b8581dac5f1775ad4402b07ce62d7d946ccae44fd3e5d5681b95f02d45ddbfbe not found: ID does not exist" containerID="b8581dac5f1775ad4402b07ce62d7d946ccae44fd3e5d5681b95f02d45ddbfbe"
Jul 24 16:03:51.670947 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.670897    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8581dac5f1775ad4402b07ce62d7d946ccae44fd3e5d5681b95f02d45ddbfbe"} err="failed to get container status \"b8581dac5f1775ad4402b07ce62d7d946ccae44fd3e5d5681b95f02d45ddbfbe\": rpc error: code = NotFound desc = could not find container \"b8581dac5f1775ad4402b07ce62d7d946ccae44fd3e5d5681b95f02d45ddbfbe\": container with ID starting with b8581dac5f1775ad4402b07ce62d7d946ccae44fd3e5d5681b95f02d45ddbfbe not found: ID does not exist"
Jul 24 16:03:51.670947 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.670921    2561 scope.go:117] "RemoveContainer" containerID="b23d498bd4c29a17baa6992aabfff461330af08b546a6c5f64bba5c2c53bb347"
Jul 24 16:03:51.680240 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.680217    2561 scope.go:117] "RemoveContainer" containerID="b23d498bd4c29a17baa6992aabfff461330af08b546a6c5f64bba5c2c53bb347"
Jul 24 16:03:51.680566 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:03:51.680543    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b23d498bd4c29a17baa6992aabfff461330af08b546a6c5f64bba5c2c53bb347\": container with ID starting with b23d498bd4c29a17baa6992aabfff461330af08b546a6c5f64bba5c2c53bb347 not found: ID does not exist" containerID="b23d498bd4c29a17baa6992aabfff461330af08b546a6c5f64bba5c2c53bb347"
Jul 24 16:03:51.680620 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.680577    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b23d498bd4c29a17baa6992aabfff461330af08b546a6c5f64bba5c2c53bb347"} err="failed to get container status \"b23d498bd4c29a17baa6992aabfff461330af08b546a6c5f64bba5c2c53bb347\": rpc error: code = NotFound desc = could not find container \"b23d498bd4c29a17baa6992aabfff461330af08b546a6c5f64bba5c2c53bb347\": container with ID starting with b23d498bd4c29a17baa6992aabfff461330af08b546a6c5f64bba5c2c53bb347 not found: ID does not exist"
Jul 24 16:03:51.688178 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.688143    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"]
Jul 24 16:03:51.691509 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.691475    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-09aa8ba8-b6c696684-h4kxf"]
Jul 24 16:03:51.701536 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.701501    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"]
Jul 24 16:03:51.704701 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:51.704636    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-09aa8ba8-6ff786fb77-8s8jl"]
Jul 24 16:03:52.106880 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:52.106845    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2720a7dd-2da4-4887-9d00-e98f83287121" path="/var/lib/kubelet/pods/2720a7dd-2da4-4887-9d00-e98f83287121/volumes"
Jul 24 16:03:52.107253 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:52.107196    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bd46eb5-7aba-4f85-a306-a6c4da860405" path="/var/lib/kubelet/pods/2bd46eb5-7aba-4f85-a306-a6c4da860405/volumes"
Jul 24 16:03:59.528019 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.527977    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"]
Jul 24 16:03:59.528393 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.528337    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2720a7dd-2da4-4887-9d00-e98f83287121" containerName="payload-pre-processing"
Jul 24 16:03:59.528393 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.528349    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2720a7dd-2da4-4887-9d00-e98f83287121" containerName="payload-pre-processing"
Jul 24 16:03:59.528393 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.528356    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bd46eb5-7aba-4f85-a306-a6c4da860405" containerName="payload-processing"
Jul 24 16:03:59.528393 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.528361    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bd46eb5-7aba-4f85-a306-a6c4da860405" containerName="payload-processing"
Jul 24 16:03:59.528541 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.528442    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2720a7dd-2da4-4887-9d00-e98f83287121" containerName="payload-pre-processing"
Jul 24 16:03:59.528541 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.528450    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bd46eb5-7aba-4f85-a306-a6c4da860405" containerName="payload-processing"
Jul 24 16:03:59.533070 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.533048    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"]
Jul 24 16:03:59.533186 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.533166    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"
Jul 24 16:03:59.535446 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.535417    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-510804df\""
Jul 24 16:03:59.536318 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.536294    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"
Jul 24 16:03:59.546427 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.546399    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"]
Jul 24 16:03:59.546755 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.546734    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tlvn\" (UniqueName: \"kubernetes.io/projected/dbafcc20-ede6-43e3-a34a-eb9cb98e7f23-kube-api-access-2tlvn\") pod \"payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s\" (UID: \"dbafcc20-ede6-43e3-a34a-eb9cb98e7f23\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"
Jul 24 16:03:59.546870 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.546777    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd6tx\" (UniqueName: \"kubernetes.io/projected/df98ab9f-9104-4cbd-9ff0-bc130c3eda3c-kube-api-access-pd6tx\") pod \"payload-processing-e2e-ait-510804df-596f6f8776-46wh9\" (UID: \"df98ab9f-9104-4cbd-9ff0-bc130c3eda3c\") " pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"
Jul 24 16:03:59.546870 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.546848    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df98ab9f-9104-4cbd-9ff0-bc130c3eda3c-plugins-config-volume\") pod \"payload-processing-e2e-ait-510804df-596f6f8776-46wh9\" (UID: \"df98ab9f-9104-4cbd-9ff0-bc130c3eda3c\") " pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"
Jul 24 16:03:59.546982 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.546950    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbafcc20-ede6-43e3-a34a-eb9cb98e7f23-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s\" (UID: \"dbafcc20-ede6-43e3-a34a-eb9cb98e7f23\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"
Jul 24 16:03:59.548808 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.548782    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"]
Jul 24 16:03:59.647819 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.647775    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tlvn\" (UniqueName: \"kubernetes.io/projected/dbafcc20-ede6-43e3-a34a-eb9cb98e7f23-kube-api-access-2tlvn\") pod \"payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s\" (UID: \"dbafcc20-ede6-43e3-a34a-eb9cb98e7f23\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"
Jul 24 16:03:59.647819 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.647820    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd6tx\" (UniqueName: \"kubernetes.io/projected/df98ab9f-9104-4cbd-9ff0-bc130c3eda3c-kube-api-access-pd6tx\") pod \"payload-processing-e2e-ait-510804df-596f6f8776-46wh9\" (UID: \"df98ab9f-9104-4cbd-9ff0-bc130c3eda3c\") " pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"
Jul 24 16:03:59.648051 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.647947    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df98ab9f-9104-4cbd-9ff0-bc130c3eda3c-plugins-config-volume\") pod \"payload-processing-e2e-ait-510804df-596f6f8776-46wh9\" (UID: \"df98ab9f-9104-4cbd-9ff0-bc130c3eda3c\") " pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"
Jul 24 16:03:59.648123 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.648060    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbafcc20-ede6-43e3-a34a-eb9cb98e7f23-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s\" (UID: \"dbafcc20-ede6-43e3-a34a-eb9cb98e7f23\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"
Jul 24 16:03:59.648714 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.648691    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df98ab9f-9104-4cbd-9ff0-bc130c3eda3c-plugins-config-volume\") pod \"payload-processing-e2e-ait-510804df-596f6f8776-46wh9\" (UID: \"df98ab9f-9104-4cbd-9ff0-bc130c3eda3c\") " pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"
Jul 24 16:03:59.648816 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.648697    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbafcc20-ede6-43e3-a34a-eb9cb98e7f23-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s\" (UID: \"dbafcc20-ede6-43e3-a34a-eb9cb98e7f23\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"
Jul 24 16:03:59.656257 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.656230    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd6tx\" (UniqueName: \"kubernetes.io/projected/df98ab9f-9104-4cbd-9ff0-bc130c3eda3c-kube-api-access-pd6tx\") pod \"payload-processing-e2e-ait-510804df-596f6f8776-46wh9\" (UID: \"df98ab9f-9104-4cbd-9ff0-bc130c3eda3c\") " pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"
Jul 24 16:03:59.656818 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.656796    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tlvn\" (UniqueName: \"kubernetes.io/projected/dbafcc20-ede6-43e3-a34a-eb9cb98e7f23-kube-api-access-2tlvn\") pod \"payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s\" (UID: \"dbafcc20-ede6-43e3-a34a-eb9cb98e7f23\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"
Jul 24 16:03:59.847690 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.847588    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"
Jul 24 16:03:59.856014 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:03:59.855985    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"
Jul 24 16:04:00.008318 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:00.008278    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"]
Jul 24 16:04:00.010074 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:04:00.010034    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbafcc20_ede6_43e3_a34a_eb9cb98e7f23.slice/crio-9e571f29103276d96ddeb40eb7790fc9f4999e7e1e3d00838e2d6bfa004335dc WatchSource:0}: Error finding container 9e571f29103276d96ddeb40eb7790fc9f4999e7e1e3d00838e2d6bfa004335dc: Status 404 returned error can't find the container with id 9e571f29103276d96ddeb40eb7790fc9f4999e7e1e3d00838e2d6bfa004335dc
Jul 24 16:04:00.038140 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:00.038104    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"]
Jul 24 16:04:00.042173 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:04:00.042125    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf98ab9f_9104_4cbd_9ff0_bc130c3eda3c.slice/crio-1169051cdd94ecd6e13a7e16b05a9b2c18892556fdd1ffed5f097753b868cb3a WatchSource:0}: Error finding container 1169051cdd94ecd6e13a7e16b05a9b2c18892556fdd1ffed5f097753b868cb3a: Status 404 returned error can't find the container with id 1169051cdd94ecd6e13a7e16b05a9b2c18892556fdd1ffed5f097753b868cb3a
Jul 24 16:04:00.704600 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:00.704557    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9" event={"ID":"df98ab9f-9104-4cbd-9ff0-bc130c3eda3c","Type":"ContainerStarted","Data":"bef5f6de11e29dcf14752ebdb400680d2651e93829f5aa0ddf9b72a8507e47e1"}
Jul 24 16:04:00.705101 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:00.704624    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"
Jul 24 16:04:00.705101 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:00.704646    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9" event={"ID":"df98ab9f-9104-4cbd-9ff0-bc130c3eda3c","Type":"ContainerStarted","Data":"1169051cdd94ecd6e13a7e16b05a9b2c18892556fdd1ffed5f097753b868cb3a"}
Jul 24 16:04:00.706334 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:00.706311    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s" event={"ID":"dbafcc20-ede6-43e3-a34a-eb9cb98e7f23","Type":"ContainerStarted","Data":"692bf6909818e016643b21fb63b356601e9c8ffe5f1978fc485bf7d283200ab4"}
Jul 24 16:04:00.706412 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:00.706353    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s" event={"ID":"dbafcc20-ede6-43e3-a34a-eb9cb98e7f23","Type":"ContainerStarted","Data":"9e571f29103276d96ddeb40eb7790fc9f4999e7e1e3d00838e2d6bfa004335dc"}
Jul 24 16:04:00.706412 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:00.706372    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"
Jul 24 16:04:00.724114 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:00.724065    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9" podStartSLOduration=1.724046757 podStartE2EDuration="1.724046757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:03:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:04:00.723259543 +0000 UTC m=+1669.133970159" watchObservedRunningTime="2026-07-24 16:04:00.724046757 +0000 UTC m=+1669.134757367"
Jul 24 16:04:00.739110 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:00.739033    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s" podStartSLOduration=1.7390099540000001 podStartE2EDuration="1.739009954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:03:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:04:00.738565155 +0000 UTC m=+1669.149275772" watchObservedRunningTime="2026-07-24 16:04:00.739009954 +0000 UTC m=+1669.149720573"
Jul 24 16:04:11.713255 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:11.713222    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"
Jul 24 16:04:11.713735 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:11.713289    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"
Jul 24 16:04:35.028219 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.028074    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"]
Jul 24 16:04:35.029078 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.028343    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9" podUID="df98ab9f-9104-4cbd-9ff0-bc130c3eda3c" containerName="payload-processing" containerID="cri-o://bef5f6de11e29dcf14752ebdb400680d2651e93829f5aa0ddf9b72a8507e47e1" gracePeriod=30
Jul 24 16:04:35.035110 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.035006    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"]
Jul 24 16:04:35.035330 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.035275    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s" podUID="dbafcc20-ede6-43e3-a34a-eb9cb98e7f23" containerName="payload-pre-processing" containerID="cri-o://692bf6909818e016643b21fb63b356601e9c8ffe5f1978fc485bf7d283200ab4" gracePeriod=30
Jul 24 16:04:35.317278 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.317252    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"
Jul 24 16:04:35.320706 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.320678    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"
Jul 24 16:04:35.476581 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.476549    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tlvn\" (UniqueName: \"kubernetes.io/projected/dbafcc20-ede6-43e3-a34a-eb9cb98e7f23-kube-api-access-2tlvn\") pod \"dbafcc20-ede6-43e3-a34a-eb9cb98e7f23\" (UID: \"dbafcc20-ede6-43e3-a34a-eb9cb98e7f23\") "
Jul 24 16:04:35.476581 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.476596    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df98ab9f-9104-4cbd-9ff0-bc130c3eda3c-plugins-config-volume\") pod \"df98ab9f-9104-4cbd-9ff0-bc130c3eda3c\" (UID: \"df98ab9f-9104-4cbd-9ff0-bc130c3eda3c\") "
Jul 24 16:04:35.476883 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.476622    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbafcc20-ede6-43e3-a34a-eb9cb98e7f23-plugins-config-volume\") pod \"dbafcc20-ede6-43e3-a34a-eb9cb98e7f23\" (UID: \"dbafcc20-ede6-43e3-a34a-eb9cb98e7f23\") "
Jul 24 16:04:35.476883 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.476690    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd6tx\" (UniqueName: \"kubernetes.io/projected/df98ab9f-9104-4cbd-9ff0-bc130c3eda3c-kube-api-access-pd6tx\") pod \"df98ab9f-9104-4cbd-9ff0-bc130c3eda3c\" (UID: \"df98ab9f-9104-4cbd-9ff0-bc130c3eda3c\") "
Jul 24 16:04:35.477082 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.477053    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df98ab9f-9104-4cbd-9ff0-bc130c3eda3c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "df98ab9f-9104-4cbd-9ff0-bc130c3eda3c" (UID: "df98ab9f-9104-4cbd-9ff0-bc130c3eda3c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:04:35.477137 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.477056    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbafcc20-ede6-43e3-a34a-eb9cb98e7f23-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dbafcc20-ede6-43e3-a34a-eb9cb98e7f23" (UID: "dbafcc20-ede6-43e3-a34a-eb9cb98e7f23"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:04:35.479065 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.479038    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbafcc20-ede6-43e3-a34a-eb9cb98e7f23-kube-api-access-2tlvn" (OuterVolumeSpecName: "kube-api-access-2tlvn") pod "dbafcc20-ede6-43e3-a34a-eb9cb98e7f23" (UID: "dbafcc20-ede6-43e3-a34a-eb9cb98e7f23"). InnerVolumeSpecName "kube-api-access-2tlvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:04:35.479182 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.479095    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df98ab9f-9104-4cbd-9ff0-bc130c3eda3c-kube-api-access-pd6tx" (OuterVolumeSpecName: "kube-api-access-pd6tx") pod "df98ab9f-9104-4cbd-9ff0-bc130c3eda3c" (UID: "df98ab9f-9104-4cbd-9ff0-bc130c3eda3c"). InnerVolumeSpecName "kube-api-access-pd6tx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:04:35.578059 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.577952    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df98ab9f-9104-4cbd-9ff0-bc130c3eda3c-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:04:35.578059 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.578000    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbafcc20-ede6-43e3-a34a-eb9cb98e7f23-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:04:35.578059 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.578010    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd6tx\" (UniqueName: \"kubernetes.io/projected/df98ab9f-9104-4cbd-9ff0-bc130c3eda3c-kube-api-access-pd6tx\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:04:35.578059 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.578018    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tlvn\" (UniqueName: \"kubernetes.io/projected/dbafcc20-ede6-43e3-a34a-eb9cb98e7f23-kube-api-access-2tlvn\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:04:35.875589 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.875500    2561 generic.go:356] "Generic (PLEG): container finished" podID="df98ab9f-9104-4cbd-9ff0-bc130c3eda3c" containerID="bef5f6de11e29dcf14752ebdb400680d2651e93829f5aa0ddf9b72a8507e47e1" exitCode=0
Jul 24 16:04:35.875589 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.875550    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"
Jul 24 16:04:35.875854 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.875591    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9" event={"ID":"df98ab9f-9104-4cbd-9ff0-bc130c3eda3c","Type":"ContainerDied","Data":"bef5f6de11e29dcf14752ebdb400680d2651e93829f5aa0ddf9b72a8507e47e1"}
Jul 24 16:04:35.875854 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.875646    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9" event={"ID":"df98ab9f-9104-4cbd-9ff0-bc130c3eda3c","Type":"ContainerDied","Data":"1169051cdd94ecd6e13a7e16b05a9b2c18892556fdd1ffed5f097753b868cb3a"}
Jul 24 16:04:35.875854 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.875701    2561 scope.go:117] "RemoveContainer" containerID="bef5f6de11e29dcf14752ebdb400680d2651e93829f5aa0ddf9b72a8507e47e1"
Jul 24 16:04:35.877137 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.877108    2561 generic.go:356] "Generic (PLEG): container finished" podID="dbafcc20-ede6-43e3-a34a-eb9cb98e7f23" containerID="692bf6909818e016643b21fb63b356601e9c8ffe5f1978fc485bf7d283200ab4" exitCode=0
Jul 24 16:04:35.877246 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.877172    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"
Jul 24 16:04:35.877246 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.877198    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s" event={"ID":"dbafcc20-ede6-43e3-a34a-eb9cb98e7f23","Type":"ContainerDied","Data":"692bf6909818e016643b21fb63b356601e9c8ffe5f1978fc485bf7d283200ab4"}
Jul 24 16:04:35.877342 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.877254    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s" event={"ID":"dbafcc20-ede6-43e3-a34a-eb9cb98e7f23","Type":"ContainerDied","Data":"9e571f29103276d96ddeb40eb7790fc9f4999e7e1e3d00838e2d6bfa004335dc"}
Jul 24 16:04:35.885101 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.885075    2561 scope.go:117] "RemoveContainer" containerID="bef5f6de11e29dcf14752ebdb400680d2651e93829f5aa0ddf9b72a8507e47e1"
Jul 24 16:04:35.885456 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:04:35.885437    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bef5f6de11e29dcf14752ebdb400680d2651e93829f5aa0ddf9b72a8507e47e1\": container with ID starting with bef5f6de11e29dcf14752ebdb400680d2651e93829f5aa0ddf9b72a8507e47e1 not found: ID does not exist" containerID="bef5f6de11e29dcf14752ebdb400680d2651e93829f5aa0ddf9b72a8507e47e1"
Jul 24 16:04:35.885549 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.885465    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bef5f6de11e29dcf14752ebdb400680d2651e93829f5aa0ddf9b72a8507e47e1"} err="failed to get container status \"bef5f6de11e29dcf14752ebdb400680d2651e93829f5aa0ddf9b72a8507e47e1\": rpc error: code = NotFound desc = could not find container \"bef5f6de11e29dcf14752ebdb400680d2651e93829f5aa0ddf9b72a8507e47e1\": container with ID starting with bef5f6de11e29dcf14752ebdb400680d2651e93829f5aa0ddf9b72a8507e47e1 not found: ID does not exist"
Jul 24 16:04:35.885549 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.885483    2561 scope.go:117] "RemoveContainer" containerID="692bf6909818e016643b21fb63b356601e9c8ffe5f1978fc485bf7d283200ab4"
Jul 24 16:04:35.896723 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.896692    2561 scope.go:117] "RemoveContainer" containerID="692bf6909818e016643b21fb63b356601e9c8ffe5f1978fc485bf7d283200ab4"
Jul 24 16:04:35.897063 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:04:35.897045    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"692bf6909818e016643b21fb63b356601e9c8ffe5f1978fc485bf7d283200ab4\": container with ID starting with 692bf6909818e016643b21fb63b356601e9c8ffe5f1978fc485bf7d283200ab4 not found: ID does not exist" containerID="692bf6909818e016643b21fb63b356601e9c8ffe5f1978fc485bf7d283200ab4"
Jul 24 16:04:35.897118 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.897070    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"692bf6909818e016643b21fb63b356601e9c8ffe5f1978fc485bf7d283200ab4"} err="failed to get container status \"692bf6909818e016643b21fb63b356601e9c8ffe5f1978fc485bf7d283200ab4\": rpc error: code = NotFound desc = could not find container \"692bf6909818e016643b21fb63b356601e9c8ffe5f1978fc485bf7d283200ab4\": container with ID starting with 692bf6909818e016643b21fb63b356601e9c8ffe5f1978fc485bf7d283200ab4 not found: ID does not exist"
Jul 24 16:04:35.902510 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.902473    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"]
Jul 24 16:04:35.906777 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.906743    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-510804df-596f6f8776-46wh9"]
Jul 24 16:04:35.922992 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.922942    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"]
Jul 24 16:04:35.926384 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:35.926344    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-510804df-5db78d494d-5wc7s"]
Jul 24 16:04:36.106412 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:36.106376    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbafcc20-ede6-43e3-a34a-eb9cb98e7f23" path="/var/lib/kubelet/pods/dbafcc20-ede6-43e3-a34a-eb9cb98e7f23/volumes"
Jul 24 16:04:36.106803 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:36.106747    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df98ab9f-9104-4cbd-9ff0-bc130c3eda3c" path="/var/lib/kubelet/pods/df98ab9f-9104-4cbd-9ff0-bc130c3eda3c/volumes"
Jul 24 16:04:44.680235 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.680191    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"]
Jul 24 16:04:44.682749 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.680543    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbafcc20-ede6-43e3-a34a-eb9cb98e7f23" containerName="payload-pre-processing"
Jul 24 16:04:44.682749 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.680555    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbafcc20-ede6-43e3-a34a-eb9cb98e7f23" containerName="payload-pre-processing"
Jul 24 16:04:44.682749 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.680574    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df98ab9f-9104-4cbd-9ff0-bc130c3eda3c" containerName="payload-processing"
Jul 24 16:04:44.682749 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.680579    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="df98ab9f-9104-4cbd-9ff0-bc130c3eda3c" containerName="payload-processing"
Jul 24 16:04:44.682749 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.680631    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="df98ab9f-9104-4cbd-9ff0-bc130c3eda3c" containerName="payload-processing"
Jul 24 16:04:44.682749 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.680640    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbafcc20-ede6-43e3-a34a-eb9cb98e7f23" containerName="payload-pre-processing"
Jul 24 16:04:44.683630 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.683612    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"
Jul 24 16:04:44.686375 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.686351    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-977c6dda\""
Jul 24 16:04:44.686829 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.686793    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-977c6dda-dockercfg-4lhff\""
Jul 24 16:04:44.705367 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.705335    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"]
Jul 24 16:04:44.716366 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.716333    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"]
Jul 24 16:04:44.719948 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.719922    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"
Jul 24 16:04:44.733033 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.732999    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"]
Jul 24 16:04:44.745081 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.745044    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrzk7\" (UniqueName: \"kubernetes.io/projected/2af0f0f3-ad1f-4334-a48d-776d4dd95ea5-kube-api-access-qrzk7\") pod \"payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv\" (UID: \"2af0f0f3-ad1f-4334-a48d-776d4dd95ea5\") " pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"
Jul 24 16:04:44.745081 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.745088    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmkmc\" (UniqueName: \"kubernetes.io/projected/e418f176-5b92-4415-be70-6fd15f8b8489-kube-api-access-dmkmc\") pod \"payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq\" (UID: \"e418f176-5b92-4415-be70-6fd15f8b8489\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"
Jul 24 16:04:44.745370 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.745113    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e418f176-5b92-4415-be70-6fd15f8b8489-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq\" (UID: \"e418f176-5b92-4415-be70-6fd15f8b8489\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"
Jul 24 16:04:44.745370 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.745280    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2af0f0f3-ad1f-4334-a48d-776d4dd95ea5-plugins-config-volume\") pod \"payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv\" (UID: \"2af0f0f3-ad1f-4334-a48d-776d4dd95ea5\") " pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"
Jul 24 16:04:44.846212 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.846173    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2af0f0f3-ad1f-4334-a48d-776d4dd95ea5-plugins-config-volume\") pod \"payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv\" (UID: \"2af0f0f3-ad1f-4334-a48d-776d4dd95ea5\") " pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"
Jul 24 16:04:44.846423 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.846247    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrzk7\" (UniqueName: \"kubernetes.io/projected/2af0f0f3-ad1f-4334-a48d-776d4dd95ea5-kube-api-access-qrzk7\") pod \"payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv\" (UID: \"2af0f0f3-ad1f-4334-a48d-776d4dd95ea5\") " pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"
Jul 24 16:04:44.846423 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.846329    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmkmc\" (UniqueName: \"kubernetes.io/projected/e418f176-5b92-4415-be70-6fd15f8b8489-kube-api-access-dmkmc\") pod \"payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq\" (UID: \"e418f176-5b92-4415-be70-6fd15f8b8489\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"
Jul 24 16:04:44.846423 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.846374    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e418f176-5b92-4415-be70-6fd15f8b8489-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq\" (UID: \"e418f176-5b92-4415-be70-6fd15f8b8489\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"
Jul 24 16:04:44.847072 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.847051    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2af0f0f3-ad1f-4334-a48d-776d4dd95ea5-plugins-config-volume\") pod \"payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv\" (UID: \"2af0f0f3-ad1f-4334-a48d-776d4dd95ea5\") " pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"
Jul 24 16:04:44.847148 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.847111    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e418f176-5b92-4415-be70-6fd15f8b8489-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq\" (UID: \"e418f176-5b92-4415-be70-6fd15f8b8489\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"
Jul 24 16:04:44.854520 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.854486    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmkmc\" (UniqueName: \"kubernetes.io/projected/e418f176-5b92-4415-be70-6fd15f8b8489-kube-api-access-dmkmc\") pod \"payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq\" (UID: \"e418f176-5b92-4415-be70-6fd15f8b8489\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"
Jul 24 16:04:44.854740 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.854553    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrzk7\" (UniqueName: \"kubernetes.io/projected/2af0f0f3-ad1f-4334-a48d-776d4dd95ea5-kube-api-access-qrzk7\") pod \"payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv\" (UID: \"2af0f0f3-ad1f-4334-a48d-776d4dd95ea5\") " pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"
Jul 24 16:04:44.995024 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:44.994983    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"
Jul 24 16:04:45.034526 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:45.034487    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"
Jul 24 16:04:45.204235 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:45.204019    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"]
Jul 24 16:04:45.207149 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:04:45.207120    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2af0f0f3_ad1f_4334_a48d_776d4dd95ea5.slice/crio-481fbbe523ff1ac61dbb1bc0b9b42927a9c84122dd6d3f6e234b14c8d3d90f1b WatchSource:0}: Error finding container 481fbbe523ff1ac61dbb1bc0b9b42927a9c84122dd6d3f6e234b14c8d3d90f1b: Status 404 returned error can't find the container with id 481fbbe523ff1ac61dbb1bc0b9b42927a9c84122dd6d3f6e234b14c8d3d90f1b
Jul 24 16:04:45.367810 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:45.367770    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"]
Jul 24 16:04:45.370948 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:04:45.370918    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode418f176_5b92_4415_be70_6fd15f8b8489.slice/crio-bf120d22fba884cd62adf7b976894ad808bb0ce022dc3f0163e0dde18ae15ea1 WatchSource:0}: Error finding container bf120d22fba884cd62adf7b976894ad808bb0ce022dc3f0163e0dde18ae15ea1: Status 404 returned error can't find the container with id bf120d22fba884cd62adf7b976894ad808bb0ce022dc3f0163e0dde18ae15ea1
Jul 24 16:04:45.931784 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:45.931743    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq" event={"ID":"e418f176-5b92-4415-be70-6fd15f8b8489","Type":"ContainerStarted","Data":"85f79b8f5daf494f9c5a5706be602fe7bf06212a370352bb10e5be0f924c93e0"}
Jul 24 16:04:45.932289 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:45.931824    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"
Jul 24 16:04:45.932289 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:45.931854    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq" event={"ID":"e418f176-5b92-4415-be70-6fd15f8b8489","Type":"ContainerStarted","Data":"bf120d22fba884cd62adf7b976894ad808bb0ce022dc3f0163e0dde18ae15ea1"}
Jul 24 16:04:45.933200 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:45.933171    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv" event={"ID":"2af0f0f3-ad1f-4334-a48d-776d4dd95ea5","Type":"ContainerStarted","Data":"73c088af3f742cae073dfde7a20e0da4c079f320e8bbe157a2c3df7203f28e20"}
Jul 24 16:04:45.933326 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:45.933216    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"
Jul 24 16:04:45.933326 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:45.933232    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv" event={"ID":"2af0f0f3-ad1f-4334-a48d-776d4dd95ea5","Type":"ContainerStarted","Data":"481fbbe523ff1ac61dbb1bc0b9b42927a9c84122dd6d3f6e234b14c8d3d90f1b"}
Jul 24 16:04:45.949753 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:45.949654    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq" podStartSLOduration=1.949631463 podStartE2EDuration="1.949631463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:04:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:04:45.94710749 +0000 UTC m=+1714.357818107" watchObservedRunningTime="2026-07-24 16:04:45.949631463 +0000 UTC m=+1714.360342070"
Jul 24 16:04:45.967417 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:45.967362    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv" podStartSLOduration=1.9673436309999999 podStartE2EDuration="1.967343631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:04:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:04:45.96603039 +0000 UTC m=+1714.376741008" watchObservedRunningTime="2026-07-24 16:04:45.967343631 +0000 UTC m=+1714.378054248"
Jul 24 16:04:56.948021 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:56.947985    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"
Jul 24 16:04:56.948432 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:04:56.948079    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"
Jul 24 16:05:19.446244 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.446204    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"]
Jul 24 16:05:19.446919 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.446481    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv" podUID="2af0f0f3-ad1f-4334-a48d-776d4dd95ea5" containerName="payload-processing" containerID="cri-o://73c088af3f742cae073dfde7a20e0da4c079f320e8bbe157a2c3df7203f28e20" gracePeriod=30
Jul 24 16:05:19.452983 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.452937    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"]
Jul 24 16:05:19.453233 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.453184    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq" podUID="e418f176-5b92-4415-be70-6fd15f8b8489" containerName="payload-pre-processing" containerID="cri-o://85f79b8f5daf494f9c5a5706be602fe7bf06212a370352bb10e5be0f924c93e0" gracePeriod=30
Jul 24 16:05:19.740757 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.740732    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"
Jul 24 16:05:19.744409 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.744379    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"
Jul 24 16:05:19.863693 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.863620    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e418f176-5b92-4415-be70-6fd15f8b8489-plugins-config-volume\") pod \"e418f176-5b92-4415-be70-6fd15f8b8489\" (UID: \"e418f176-5b92-4415-be70-6fd15f8b8489\") "
Jul 24 16:05:19.863880 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.863714    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qrzk7\" (UniqueName: \"kubernetes.io/projected/2af0f0f3-ad1f-4334-a48d-776d4dd95ea5-kube-api-access-qrzk7\") pod \"2af0f0f3-ad1f-4334-a48d-776d4dd95ea5\" (UID: \"2af0f0f3-ad1f-4334-a48d-776d4dd95ea5\") "
Jul 24 16:05:19.863880 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.863760    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2af0f0f3-ad1f-4334-a48d-776d4dd95ea5-plugins-config-volume\") pod \"2af0f0f3-ad1f-4334-a48d-776d4dd95ea5\" (UID: \"2af0f0f3-ad1f-4334-a48d-776d4dd95ea5\") "
Jul 24 16:05:19.863880 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.863780    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmkmc\" (UniqueName: \"kubernetes.io/projected/e418f176-5b92-4415-be70-6fd15f8b8489-kube-api-access-dmkmc\") pod \"e418f176-5b92-4415-be70-6fd15f8b8489\" (UID: \"e418f176-5b92-4415-be70-6fd15f8b8489\") "
Jul 24 16:05:19.864056 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.864037    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e418f176-5b92-4415-be70-6fd15f8b8489-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e418f176-5b92-4415-be70-6fd15f8b8489" (UID: "e418f176-5b92-4415-be70-6fd15f8b8489"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:05:19.864162 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.864136    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2af0f0f3-ad1f-4334-a48d-776d4dd95ea5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2af0f0f3-ad1f-4334-a48d-776d4dd95ea5" (UID: "2af0f0f3-ad1f-4334-a48d-776d4dd95ea5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:05:19.866197 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.866172    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2af0f0f3-ad1f-4334-a48d-776d4dd95ea5-kube-api-access-qrzk7" (OuterVolumeSpecName: "kube-api-access-qrzk7") pod "2af0f0f3-ad1f-4334-a48d-776d4dd95ea5" (UID: "2af0f0f3-ad1f-4334-a48d-776d4dd95ea5"). InnerVolumeSpecName "kube-api-access-qrzk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:05:19.866294 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.866201    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e418f176-5b92-4415-be70-6fd15f8b8489-kube-api-access-dmkmc" (OuterVolumeSpecName: "kube-api-access-dmkmc") pod "e418f176-5b92-4415-be70-6fd15f8b8489" (UID: "e418f176-5b92-4415-be70-6fd15f8b8489"). InnerVolumeSpecName "kube-api-access-dmkmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:05:19.965126 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.965040    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmkmc\" (UniqueName: \"kubernetes.io/projected/e418f176-5b92-4415-be70-6fd15f8b8489-kube-api-access-dmkmc\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:05:19.965126 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.965075    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e418f176-5b92-4415-be70-6fd15f8b8489-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:05:19.965126 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.965085    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrzk7\" (UniqueName: \"kubernetes.io/projected/2af0f0f3-ad1f-4334-a48d-776d4dd95ea5-kube-api-access-qrzk7\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:05:19.965126 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:19.965093    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2af0f0f3-ad1f-4334-a48d-776d4dd95ea5-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:05:20.102981 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.102950    2561 generic.go:356] "Generic (PLEG): container finished" podID="e418f176-5b92-4415-be70-6fd15f8b8489" containerID="85f79b8f5daf494f9c5a5706be602fe7bf06212a370352bb10e5be0f924c93e0" exitCode=0
Jul 24 16:05:20.103180 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.103071    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"
Jul 24 16:05:20.104422 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.104394    2561 generic.go:356] "Generic (PLEG): container finished" podID="2af0f0f3-ad1f-4334-a48d-776d4dd95ea5" containerID="73c088af3f742cae073dfde7a20e0da4c079f320e8bbe157a2c3df7203f28e20" exitCode=0
Jul 24 16:05:20.104586 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.104476    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"
Jul 24 16:05:20.108762 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.108722    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq" event={"ID":"e418f176-5b92-4415-be70-6fd15f8b8489","Type":"ContainerDied","Data":"85f79b8f5daf494f9c5a5706be602fe7bf06212a370352bb10e5be0f924c93e0"}
Jul 24 16:05:20.108924 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.108786    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq" event={"ID":"e418f176-5b92-4415-be70-6fd15f8b8489","Type":"ContainerDied","Data":"bf120d22fba884cd62adf7b976894ad808bb0ce022dc3f0163e0dde18ae15ea1"}
Jul 24 16:05:20.108924 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.108810    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv" event={"ID":"2af0f0f3-ad1f-4334-a48d-776d4dd95ea5","Type":"ContainerDied","Data":"73c088af3f742cae073dfde7a20e0da4c079f320e8bbe157a2c3df7203f28e20"}
Jul 24 16:05:20.108924 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.108834    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv" event={"ID":"2af0f0f3-ad1f-4334-a48d-776d4dd95ea5","Type":"ContainerDied","Data":"481fbbe523ff1ac61dbb1bc0b9b42927a9c84122dd6d3f6e234b14c8d3d90f1b"}
Jul 24 16:05:20.108924 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.108862    2561 scope.go:117] "RemoveContainer" containerID="85f79b8f5daf494f9c5a5706be602fe7bf06212a370352bb10e5be0f924c93e0"
Jul 24 16:05:20.127177 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.127146    2561 scope.go:117] "RemoveContainer" containerID="85f79b8f5daf494f9c5a5706be602fe7bf06212a370352bb10e5be0f924c93e0"
Jul 24 16:05:20.127559 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:05:20.127536    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85f79b8f5daf494f9c5a5706be602fe7bf06212a370352bb10e5be0f924c93e0\": container with ID starting with 85f79b8f5daf494f9c5a5706be602fe7bf06212a370352bb10e5be0f924c93e0 not found: ID does not exist" containerID="85f79b8f5daf494f9c5a5706be602fe7bf06212a370352bb10e5be0f924c93e0"
Jul 24 16:05:20.127650 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.127573    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85f79b8f5daf494f9c5a5706be602fe7bf06212a370352bb10e5be0f924c93e0"} err="failed to get container status \"85f79b8f5daf494f9c5a5706be602fe7bf06212a370352bb10e5be0f924c93e0\": rpc error: code = NotFound desc = could not find container \"85f79b8f5daf494f9c5a5706be602fe7bf06212a370352bb10e5be0f924c93e0\": container with ID starting with 85f79b8f5daf494f9c5a5706be602fe7bf06212a370352bb10e5be0f924c93e0 not found: ID does not exist"
Jul 24 16:05:20.127650 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.127600    2561 scope.go:117] "RemoveContainer" containerID="73c088af3f742cae073dfde7a20e0da4c079f320e8bbe157a2c3df7203f28e20"
Jul 24 16:05:20.135560 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.135520    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"]
Jul 24 16:05:20.139398 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.139366    2561 scope.go:117] "RemoveContainer" containerID="73c088af3f742cae073dfde7a20e0da4c079f320e8bbe157a2c3df7203f28e20"
Jul 24 16:05:20.139635 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.139611    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-977c6dda-7575c9cb58-vd8gv"]
Jul 24 16:05:20.139796 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:05:20.139767    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73c088af3f742cae073dfde7a20e0da4c079f320e8bbe157a2c3df7203f28e20\": container with ID starting with 73c088af3f742cae073dfde7a20e0da4c079f320e8bbe157a2c3df7203f28e20 not found: ID does not exist" containerID="73c088af3f742cae073dfde7a20e0da4c079f320e8bbe157a2c3df7203f28e20"
Jul 24 16:05:20.139862 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.139793    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73c088af3f742cae073dfde7a20e0da4c079f320e8bbe157a2c3df7203f28e20"} err="failed to get container status \"73c088af3f742cae073dfde7a20e0da4c079f320e8bbe157a2c3df7203f28e20\": rpc error: code = NotFound desc = could not find container \"73c088af3f742cae073dfde7a20e0da4c079f320e8bbe157a2c3df7203f28e20\": container with ID starting with 73c088af3f742cae073dfde7a20e0da4c079f320e8bbe157a2c3df7203f28e20 not found: ID does not exist"
Jul 24 16:05:20.151754 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.151718    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"]
Jul 24 16:05:20.156274 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:20.156239    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-977c6dda-7d47699f6-xrswq"]
Jul 24 16:05:22.107149 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:22.107115    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2af0f0f3-ad1f-4334-a48d-776d4dd95ea5" path="/var/lib/kubelet/pods/2af0f0f3-ad1f-4334-a48d-776d4dd95ea5/volumes"
Jul 24 16:05:22.107545 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:05:22.107461    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e418f176-5b92-4415-be70-6fd15f8b8489" path="/var/lib/kubelet/pods/e418f176-5b92-4415-be70-6fd15f8b8489/volumes"
Jul 24 16:07:06.237203 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.237161    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"]
Jul 24 16:07:06.237867 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.237525    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2af0f0f3-ad1f-4334-a48d-776d4dd95ea5" containerName="payload-processing"
Jul 24 16:07:06.237867 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.237538    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2af0f0f3-ad1f-4334-a48d-776d4dd95ea5" containerName="payload-processing"
Jul 24 16:07:06.237867 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.237562    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e418f176-5b92-4415-be70-6fd15f8b8489" containerName="payload-pre-processing"
Jul 24 16:07:06.237867 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.237567    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e418f176-5b92-4415-be70-6fd15f8b8489" containerName="payload-pre-processing"
Jul 24 16:07:06.237867 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.237639    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2af0f0f3-ad1f-4334-a48d-776d4dd95ea5" containerName="payload-processing"
Jul 24 16:07:06.237867 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.237649    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e418f176-5b92-4415-be70-6fd15f8b8489" containerName="payload-pre-processing"
Jul 24 16:07:06.240645 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.240620    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"
Jul 24 16:07:06.243800 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.243772    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-850ff15f\""
Jul 24 16:07:06.259568 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.259531    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"]
Jul 24 16:07:06.323369 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.323323    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"]
Jul 24 16:07:06.326753 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.326728    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"
Jul 24 16:07:06.329803 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.329782    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-850ff15f-dockercfg-9rrkf\""
Jul 24 16:07:06.344921 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.344889    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"]
Jul 24 16:07:06.354524 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.354487    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f84322f0-1631-422e-beef-0402b56942da-plugins-config-volume\") pod \"payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz\" (UID: \"f84322f0-1631-422e-beef-0402b56942da\") " pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"
Jul 24 16:07:06.354760 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.354534    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6p4b\" (UniqueName: \"kubernetes.io/projected/f84322f0-1631-422e-beef-0402b56942da-kube-api-access-x6p4b\") pod \"payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz\" (UID: \"f84322f0-1631-422e-beef-0402b56942da\") " pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"
Jul 24 16:07:06.354760 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.354572    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7lsd\" (UniqueName: \"kubernetes.io/projected/d7def994-eb00-4538-a58c-22ff73387975-kube-api-access-r7lsd\") pod \"payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf\" (UID: \"d7def994-eb00-4538-a58c-22ff73387975\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"
Jul 24 16:07:06.354760 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.354708    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7def994-eb00-4538-a58c-22ff73387975-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf\" (UID: \"d7def994-eb00-4538-a58c-22ff73387975\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"
Jul 24 16:07:06.455610 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.455565    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7lsd\" (UniqueName: \"kubernetes.io/projected/d7def994-eb00-4538-a58c-22ff73387975-kube-api-access-r7lsd\") pod \"payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf\" (UID: \"d7def994-eb00-4538-a58c-22ff73387975\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"
Jul 24 16:07:06.455854 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.455653    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7def994-eb00-4538-a58c-22ff73387975-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf\" (UID: \"d7def994-eb00-4538-a58c-22ff73387975\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"
Jul 24 16:07:06.455854 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.455834    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f84322f0-1631-422e-beef-0402b56942da-plugins-config-volume\") pod \"payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz\" (UID: \"f84322f0-1631-422e-beef-0402b56942da\") " pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"
Jul 24 16:07:06.455980 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.455903    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6p4b\" (UniqueName: \"kubernetes.io/projected/f84322f0-1631-422e-beef-0402b56942da-kube-api-access-x6p4b\") pod \"payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz\" (UID: \"f84322f0-1631-422e-beef-0402b56942da\") " pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"
Jul 24 16:07:06.456597 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.456566    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f84322f0-1631-422e-beef-0402b56942da-plugins-config-volume\") pod \"payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz\" (UID: \"f84322f0-1631-422e-beef-0402b56942da\") " pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"
Jul 24 16:07:06.456762 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.456566    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7def994-eb00-4538-a58c-22ff73387975-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf\" (UID: \"d7def994-eb00-4538-a58c-22ff73387975\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"
Jul 24 16:07:06.465825 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.465796    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7lsd\" (UniqueName: \"kubernetes.io/projected/d7def994-eb00-4538-a58c-22ff73387975-kube-api-access-r7lsd\") pod \"payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf\" (UID: \"d7def994-eb00-4538-a58c-22ff73387975\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"
Jul 24 16:07:06.465991 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.465969    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6p4b\" (UniqueName: \"kubernetes.io/projected/f84322f0-1631-422e-beef-0402b56942da-kube-api-access-x6p4b\") pod \"payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz\" (UID: \"f84322f0-1631-422e-beef-0402b56942da\") " pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"
Jul 24 16:07:06.551951 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.551853    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"
Jul 24 16:07:06.638163 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.638051    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"
Jul 24 16:07:06.713538 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.713349    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"]
Jul 24 16:07:06.717807 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:07:06.717629    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7def994_eb00_4538_a58c_22ff73387975.slice/crio-d40fd8621c010eec8c55b98e88f00b3273ddfe53be26c6ea70da5372cd67f651 WatchSource:0}: Error finding container d40fd8621c010eec8c55b98e88f00b3273ddfe53be26c6ea70da5372cd67f651: Status 404 returned error can't find the container with id d40fd8621c010eec8c55b98e88f00b3273ddfe53be26c6ea70da5372cd67f651
Jul 24 16:07:06.810116 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:06.810086    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"]
Jul 24 16:07:06.811960 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:07:06.811931    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf84322f0_1631_422e_beef_0402b56942da.slice/crio-0b1f65c8d36b23d69686c548deb69d1ea81d94b8a9a607d7546b650cc3075607 WatchSource:0}: Error finding container 0b1f65c8d36b23d69686c548deb69d1ea81d94b8a9a607d7546b650cc3075607: Status 404 returned error can't find the container with id 0b1f65c8d36b23d69686c548deb69d1ea81d94b8a9a607d7546b650cc3075607
Jul 24 16:07:07.647132 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:07.647093    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf" event={"ID":"d7def994-eb00-4538-a58c-22ff73387975","Type":"ContainerStarted","Data":"64be26499c2bbe83ee61c8716d565e73826656490c02923f6e1c4d113693837b"}
Jul 24 16:07:07.647630 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:07.647161    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf" event={"ID":"d7def994-eb00-4538-a58c-22ff73387975","Type":"ContainerStarted","Data":"d40fd8621c010eec8c55b98e88f00b3273ddfe53be26c6ea70da5372cd67f651"}
Jul 24 16:07:07.647630 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:07.647193    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"
Jul 24 16:07:07.648966 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:07.648935    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz" event={"ID":"f84322f0-1631-422e-beef-0402b56942da","Type":"ContainerStarted","Data":"ba236223742aa4a6f6eb628fc61375ee37cb2ec93ee0c137602faed91cd46383"}
Jul 24 16:07:07.649086 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:07.648996    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz" event={"ID":"f84322f0-1631-422e-beef-0402b56942da","Type":"ContainerStarted","Data":"0b1f65c8d36b23d69686c548deb69d1ea81d94b8a9a607d7546b650cc3075607"}
Jul 24 16:07:07.649086 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:07.649027    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"
Jul 24 16:07:07.666985 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:07.666926    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf" podStartSLOduration=1.6669099 podStartE2EDuration="1.6669099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:07:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:07:07.664520489 +0000 UTC m=+1856.075231106" watchObservedRunningTime="2026-07-24 16:07:07.6669099 +0000 UTC m=+1856.077620569"
Jul 24 16:07:07.682294 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:07.682233    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz" podStartSLOduration=1.682215488 podStartE2EDuration="1.682215488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:07:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:07:07.680499833 +0000 UTC m=+1856.091210477" watchObservedRunningTime="2026-07-24 16:07:07.682215488 +0000 UTC m=+1856.092926139"
Jul 24 16:07:18.657046 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:18.657009    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"
Jul 24 16:07:18.657436 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:18.657093    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"
Jul 24 16:07:27.786846 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:27.786803    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"]
Jul 24 16:07:27.787289 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:27.787096    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz" podUID="f84322f0-1631-422e-beef-0402b56942da" containerName="payload-processing" containerID="cri-o://ba236223742aa4a6f6eb628fc61375ee37cb2ec93ee0c137602faed91cd46383" gracePeriod=30
Jul 24 16:07:27.799316 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:27.799281    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"]
Jul 24 16:07:27.799547 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:27.799524    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf" podUID="d7def994-eb00-4538-a58c-22ff73387975" containerName="payload-pre-processing" containerID="cri-o://64be26499c2bbe83ee61c8716d565e73826656490c02923f6e1c4d113693837b" gracePeriod=30
Jul 24 16:07:28.079330 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.079301    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"
Jul 24 16:07:28.082703 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.082678    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"
Jul 24 16:07:28.160809 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.160765    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r7lsd\" (UniqueName: \"kubernetes.io/projected/d7def994-eb00-4538-a58c-22ff73387975-kube-api-access-r7lsd\") pod \"d7def994-eb00-4538-a58c-22ff73387975\" (UID: \"d7def994-eb00-4538-a58c-22ff73387975\") "
Jul 24 16:07:28.160809 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.160815    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7def994-eb00-4538-a58c-22ff73387975-plugins-config-volume\") pod \"d7def994-eb00-4538-a58c-22ff73387975\" (UID: \"d7def994-eb00-4538-a58c-22ff73387975\") "
Jul 24 16:07:28.161043 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.160853    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6p4b\" (UniqueName: \"kubernetes.io/projected/f84322f0-1631-422e-beef-0402b56942da-kube-api-access-x6p4b\") pod \"f84322f0-1631-422e-beef-0402b56942da\" (UID: \"f84322f0-1631-422e-beef-0402b56942da\") "
Jul 24 16:07:28.161043 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.160880    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f84322f0-1631-422e-beef-0402b56942da-plugins-config-volume\") pod \"f84322f0-1631-422e-beef-0402b56942da\" (UID: \"f84322f0-1631-422e-beef-0402b56942da\") "
Jul 24 16:07:28.161279 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.161241    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7def994-eb00-4538-a58c-22ff73387975-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d7def994-eb00-4538-a58c-22ff73387975" (UID: "d7def994-eb00-4538-a58c-22ff73387975"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:07:28.161394 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.161317    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f84322f0-1631-422e-beef-0402b56942da-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f84322f0-1631-422e-beef-0402b56942da" (UID: "f84322f0-1631-422e-beef-0402b56942da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:07:28.163277 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.163249    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7def994-eb00-4538-a58c-22ff73387975-kube-api-access-r7lsd" (OuterVolumeSpecName: "kube-api-access-r7lsd") pod "d7def994-eb00-4538-a58c-22ff73387975" (UID: "d7def994-eb00-4538-a58c-22ff73387975"). InnerVolumeSpecName "kube-api-access-r7lsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:07:28.163372 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.163297    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f84322f0-1631-422e-beef-0402b56942da-kube-api-access-x6p4b" (OuterVolumeSpecName: "kube-api-access-x6p4b") pod "f84322f0-1631-422e-beef-0402b56942da" (UID: "f84322f0-1631-422e-beef-0402b56942da"). InnerVolumeSpecName "kube-api-access-x6p4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:07:28.262368 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.262321    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f84322f0-1631-422e-beef-0402b56942da-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:07:28.262368 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.262358    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7lsd\" (UniqueName: \"kubernetes.io/projected/d7def994-eb00-4538-a58c-22ff73387975-kube-api-access-r7lsd\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:07:28.262368 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.262370    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7def994-eb00-4538-a58c-22ff73387975-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:07:28.262368 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.262381    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6p4b\" (UniqueName: \"kubernetes.io/projected/f84322f0-1631-422e-beef-0402b56942da-kube-api-access-x6p4b\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:07:28.756119 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.756088    2561 generic.go:356] "Generic (PLEG): container finished" podID="d7def994-eb00-4538-a58c-22ff73387975" containerID="64be26499c2bbe83ee61c8716d565e73826656490c02923f6e1c4d113693837b" exitCode=0
Jul 24 16:07:28.756324 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.756137    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"
Jul 24 16:07:28.756324 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.756180    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf" event={"ID":"d7def994-eb00-4538-a58c-22ff73387975","Type":"ContainerDied","Data":"64be26499c2bbe83ee61c8716d565e73826656490c02923f6e1c4d113693837b"}
Jul 24 16:07:28.756324 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.756232    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf" event={"ID":"d7def994-eb00-4538-a58c-22ff73387975","Type":"ContainerDied","Data":"d40fd8621c010eec8c55b98e88f00b3273ddfe53be26c6ea70da5372cd67f651"}
Jul 24 16:07:28.756324 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.756261    2561 scope.go:117] "RemoveContainer" containerID="64be26499c2bbe83ee61c8716d565e73826656490c02923f6e1c4d113693837b"
Jul 24 16:07:28.757618 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.757594    2561 generic.go:356] "Generic (PLEG): container finished" podID="f84322f0-1631-422e-beef-0402b56942da" containerID="ba236223742aa4a6f6eb628fc61375ee37cb2ec93ee0c137602faed91cd46383" exitCode=0
Jul 24 16:07:28.757744 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.757676    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"
Jul 24 16:07:28.757744 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.757678    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz" event={"ID":"f84322f0-1631-422e-beef-0402b56942da","Type":"ContainerDied","Data":"ba236223742aa4a6f6eb628fc61375ee37cb2ec93ee0c137602faed91cd46383"}
Jul 24 16:07:28.757874 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.757743    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz" event={"ID":"f84322f0-1631-422e-beef-0402b56942da","Type":"ContainerDied","Data":"0b1f65c8d36b23d69686c548deb69d1ea81d94b8a9a607d7546b650cc3075607"}
Jul 24 16:07:28.766920 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.766849    2561 scope.go:117] "RemoveContainer" containerID="64be26499c2bbe83ee61c8716d565e73826656490c02923f6e1c4d113693837b"
Jul 24 16:07:28.767231 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:07:28.767202    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64be26499c2bbe83ee61c8716d565e73826656490c02923f6e1c4d113693837b\": container with ID starting with 64be26499c2bbe83ee61c8716d565e73826656490c02923f6e1c4d113693837b not found: ID does not exist" containerID="64be26499c2bbe83ee61c8716d565e73826656490c02923f6e1c4d113693837b"
Jul 24 16:07:28.767335 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.767240    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64be26499c2bbe83ee61c8716d565e73826656490c02923f6e1c4d113693837b"} err="failed to get container status \"64be26499c2bbe83ee61c8716d565e73826656490c02923f6e1c4d113693837b\": rpc error: code = NotFound desc = could not find container \"64be26499c2bbe83ee61c8716d565e73826656490c02923f6e1c4d113693837b\": container with ID starting with 64be26499c2bbe83ee61c8716d565e73826656490c02923f6e1c4d113693837b not found: ID does not exist"
Jul 24 16:07:28.767335 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.767266    2561 scope.go:117] "RemoveContainer" containerID="ba236223742aa4a6f6eb628fc61375ee37cb2ec93ee0c137602faed91cd46383"
Jul 24 16:07:28.776539 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.776518    2561 scope.go:117] "RemoveContainer" containerID="ba236223742aa4a6f6eb628fc61375ee37cb2ec93ee0c137602faed91cd46383"
Jul 24 16:07:28.776927 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:07:28.776902    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba236223742aa4a6f6eb628fc61375ee37cb2ec93ee0c137602faed91cd46383\": container with ID starting with ba236223742aa4a6f6eb628fc61375ee37cb2ec93ee0c137602faed91cd46383 not found: ID does not exist" containerID="ba236223742aa4a6f6eb628fc61375ee37cb2ec93ee0c137602faed91cd46383"
Jul 24 16:07:28.777016 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.776936    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba236223742aa4a6f6eb628fc61375ee37cb2ec93ee0c137602faed91cd46383"} err="failed to get container status \"ba236223742aa4a6f6eb628fc61375ee37cb2ec93ee0c137602faed91cd46383\": rpc error: code = NotFound desc = could not find container \"ba236223742aa4a6f6eb628fc61375ee37cb2ec93ee0c137602faed91cd46383\": container with ID starting with ba236223742aa4a6f6eb628fc61375ee37cb2ec93ee0c137602faed91cd46383 not found: ID does not exist"
Jul 24 16:07:28.779599 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.779571    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"]
Jul 24 16:07:28.786997 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.786959    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-850ff15f-86594cb8f6-j2qgf"]
Jul 24 16:07:28.798408 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.798378    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"]
Jul 24 16:07:28.800631 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:28.800602    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-850ff15f-649b9d75c8-jqklz"]
Jul 24 16:07:30.106518 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:30.106482    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7def994-eb00-4538-a58c-22ff73387975" path="/var/lib/kubelet/pods/d7def994-eb00-4538-a58c-22ff73387975/volumes"
Jul 24 16:07:30.106933 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:07:30.106860    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f84322f0-1631-422e-beef-0402b56942da" path="/var/lib/kubelet/pods/f84322f0-1631-422e-beef-0402b56942da/volumes"
Jul 24 16:08:03.571644 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.571603    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"]
Jul 24 16:08:03.572129 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.572015    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7def994-eb00-4538-a58c-22ff73387975" containerName="payload-pre-processing"
Jul 24 16:08:03.572129 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.572028    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7def994-eb00-4538-a58c-22ff73387975" containerName="payload-pre-processing"
Jul 24 16:08:03.572129 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.572052    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f84322f0-1631-422e-beef-0402b56942da" containerName="payload-processing"
Jul 24 16:08:03.572129 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.572059    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f84322f0-1631-422e-beef-0402b56942da" containerName="payload-processing"
Jul 24 16:08:03.572129 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.572126    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7def994-eb00-4538-a58c-22ff73387975" containerName="payload-pre-processing"
Jul 24 16:08:03.572300 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.572133    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f84322f0-1631-422e-beef-0402b56942da" containerName="payload-processing"
Jul 24 16:08:03.574843 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.574826    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"
Jul 24 16:08:03.577448 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.577427    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-5b382fb6-dockercfg-km4w8\""
Jul 24 16:08:03.577938 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.577919    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-5b382fb6\""
Jul 24 16:08:03.590996 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.590964    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"]
Jul 24 16:08:03.625643 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.625604    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"]
Jul 24 16:08:03.629026 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.629006    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"
Jul 24 16:08:03.648161 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.648127    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"]
Jul 24 16:08:03.670068 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.670016    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2kh9\" (UniqueName: \"kubernetes.io/projected/36880849-3a32-4e98-ae65-8fe933850cc5-kube-api-access-s2kh9\") pod \"payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg\" (UID: \"36880849-3a32-4e98-ae65-8fe933850cc5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"
Jul 24 16:08:03.670271 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.670095    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqq8v\" (UniqueName: \"kubernetes.io/projected/c2bf943a-65ac-4264-93b9-73a97e803f53-kube-api-access-pqq8v\") pod \"payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp\" (UID: \"c2bf943a-65ac-4264-93b9-73a97e803f53\") " pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"
Jul 24 16:08:03.670271 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.670193    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2bf943a-65ac-4264-93b9-73a97e803f53-plugins-config-volume\") pod \"payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp\" (UID: \"c2bf943a-65ac-4264-93b9-73a97e803f53\") " pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"
Jul 24 16:08:03.670271 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.670234    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36880849-3a32-4e98-ae65-8fe933850cc5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg\" (UID: \"36880849-3a32-4e98-ae65-8fe933850cc5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"
Jul 24 16:08:03.771298 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.771249    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2bf943a-65ac-4264-93b9-73a97e803f53-plugins-config-volume\") pod \"payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp\" (UID: \"c2bf943a-65ac-4264-93b9-73a97e803f53\") " pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"
Jul 24 16:08:03.771298 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.771303    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36880849-3a32-4e98-ae65-8fe933850cc5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg\" (UID: \"36880849-3a32-4e98-ae65-8fe933850cc5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"
Jul 24 16:08:03.771587 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.771335    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2kh9\" (UniqueName: \"kubernetes.io/projected/36880849-3a32-4e98-ae65-8fe933850cc5-kube-api-access-s2kh9\") pod \"payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg\" (UID: \"36880849-3a32-4e98-ae65-8fe933850cc5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"
Jul 24 16:08:03.771587 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.771410    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqq8v\" (UniqueName: \"kubernetes.io/projected/c2bf943a-65ac-4264-93b9-73a97e803f53-kube-api-access-pqq8v\") pod \"payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp\" (UID: \"c2bf943a-65ac-4264-93b9-73a97e803f53\") " pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"
Jul 24 16:08:03.772074 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.772054    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36880849-3a32-4e98-ae65-8fe933850cc5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg\" (UID: \"36880849-3a32-4e98-ae65-8fe933850cc5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"
Jul 24 16:08:03.772139 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.772123    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2bf943a-65ac-4264-93b9-73a97e803f53-plugins-config-volume\") pod \"payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp\" (UID: \"c2bf943a-65ac-4264-93b9-73a97e803f53\") " pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"
Jul 24 16:08:03.780966 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.780934    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqq8v\" (UniqueName: \"kubernetes.io/projected/c2bf943a-65ac-4264-93b9-73a97e803f53-kube-api-access-pqq8v\") pod \"payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp\" (UID: \"c2bf943a-65ac-4264-93b9-73a97e803f53\") " pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"
Jul 24 16:08:03.781123 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.780934    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2kh9\" (UniqueName: \"kubernetes.io/projected/36880849-3a32-4e98-ae65-8fe933850cc5-kube-api-access-s2kh9\") pod \"payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg\" (UID: \"36880849-3a32-4e98-ae65-8fe933850cc5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"
Jul 24 16:08:03.884921 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.884821    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"
Jul 24 16:08:03.939738 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:03.939546    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"
Jul 24 16:08:04.066501 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:04.066461    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"]
Jul 24 16:08:04.070125 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:08:04.070091    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2bf943a_65ac_4264_93b9_73a97e803f53.slice/crio-0ccb37b2dc431883d07be8d0fcdaa3d20df1792e2b1e8e0c1fff23bf508548df WatchSource:0}: Error finding container 0ccb37b2dc431883d07be8d0fcdaa3d20df1792e2b1e8e0c1fff23bf508548df: Status 404 returned error can't find the container with id 0ccb37b2dc431883d07be8d0fcdaa3d20df1792e2b1e8e0c1fff23bf508548df
Jul 24 16:08:04.118116 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:04.118086    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"]
Jul 24 16:08:04.120331 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:08:04.120302    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36880849_3a32_4e98_ae65_8fe933850cc5.slice/crio-9ff8ac696bbced4aba2a763747ac3d6d370d53adb2f3f719dd78c3c4aa3895f9 WatchSource:0}: Error finding container 9ff8ac696bbced4aba2a763747ac3d6d370d53adb2f3f719dd78c3c4aa3895f9: Status 404 returned error can't find the container with id 9ff8ac696bbced4aba2a763747ac3d6d370d53adb2f3f719dd78c3c4aa3895f9
Jul 24 16:08:04.931495 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:04.931452    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg" event={"ID":"36880849-3a32-4e98-ae65-8fe933850cc5","Type":"ContainerStarted","Data":"9ad33b5cf9a6eafa0516354f28846d1d68fd991a8637ee3fad7958ae18ccf255"}
Jul 24 16:08:04.931959 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:04.931512    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg" event={"ID":"36880849-3a32-4e98-ae65-8fe933850cc5","Type":"ContainerStarted","Data":"9ff8ac696bbced4aba2a763747ac3d6d370d53adb2f3f719dd78c3c4aa3895f9"}
Jul 24 16:08:04.931959 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:04.931578    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"
Jul 24 16:08:04.933247 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:04.933221    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp" event={"ID":"c2bf943a-65ac-4264-93b9-73a97e803f53","Type":"ContainerStarted","Data":"33ce8b3345192d2e56f3f410fdba27bb69c1d8b1b01f43fb4c4cae02f8996fc3"}
Jul 24 16:08:04.933361 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:04.933271    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"
Jul 24 16:08:04.933361 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:04.933287    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp" event={"ID":"c2bf943a-65ac-4264-93b9-73a97e803f53","Type":"ContainerStarted","Data":"0ccb37b2dc431883d07be8d0fcdaa3d20df1792e2b1e8e0c1fff23bf508548df"}
Jul 24 16:08:04.950080 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:04.950010    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg" podStartSLOduration=1.9499882579999999 podStartE2EDuration="1.949988258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:08:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:08:04.948479405 +0000 UTC m=+1913.359190060" watchObservedRunningTime="2026-07-24 16:08:04.949988258 +0000 UTC m=+1913.360698854"
Jul 24 16:08:04.966534 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:04.966474    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp" podStartSLOduration=1.9664568089999999 podStartE2EDuration="1.966456809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:08:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:08:04.964035194 +0000 UTC m=+1913.374745812" watchObservedRunningTime="2026-07-24 16:08:04.966456809 +0000 UTC m=+1913.377167426"
Jul 24 16:08:15.940955 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:15.940919    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"
Jul 24 16:08:15.941480 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:15.941004    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"
Jul 24 16:08:52.138767 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.137052    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"]
Jul 24 16:08:52.138767 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.137322    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp" podUID="c2bf943a-65ac-4264-93b9-73a97e803f53" containerName="payload-processing" containerID="cri-o://33ce8b3345192d2e56f3f410fdba27bb69c1d8b1b01f43fb4c4cae02f8996fc3" gracePeriod=30
Jul 24 16:08:52.144651 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.144603    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"]
Jul 24 16:08:52.145461 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.145086    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg" podUID="36880849-3a32-4e98-ae65-8fe933850cc5" containerName="payload-pre-processing" containerID="cri-o://9ad33b5cf9a6eafa0516354f28846d1d68fd991a8637ee3fad7958ae18ccf255" gracePeriod=30
Jul 24 16:08:52.403372 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.403279    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"
Jul 24 16:08:52.430946 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.430911    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"
Jul 24 16:08:52.481564 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.481528    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqq8v\" (UniqueName: \"kubernetes.io/projected/c2bf943a-65ac-4264-93b9-73a97e803f53-kube-api-access-pqq8v\") pod \"c2bf943a-65ac-4264-93b9-73a97e803f53\" (UID: \"c2bf943a-65ac-4264-93b9-73a97e803f53\") "
Jul 24 16:08:52.481769 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.481593    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2bf943a-65ac-4264-93b9-73a97e803f53-plugins-config-volume\") pod \"c2bf943a-65ac-4264-93b9-73a97e803f53\" (UID: \"c2bf943a-65ac-4264-93b9-73a97e803f53\") "
Jul 24 16:08:52.481769 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.481612    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36880849-3a32-4e98-ae65-8fe933850cc5-plugins-config-volume\") pod \"36880849-3a32-4e98-ae65-8fe933850cc5\" (UID: \"36880849-3a32-4e98-ae65-8fe933850cc5\") "
Jul 24 16:08:52.481769 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.481647    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2kh9\" (UniqueName: \"kubernetes.io/projected/36880849-3a32-4e98-ae65-8fe933850cc5-kube-api-access-s2kh9\") pod \"36880849-3a32-4e98-ae65-8fe933850cc5\" (UID: \"36880849-3a32-4e98-ae65-8fe933850cc5\") "
Jul 24 16:08:52.482034 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.482010    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2bf943a-65ac-4264-93b9-73a97e803f53-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c2bf943a-65ac-4264-93b9-73a97e803f53" (UID: "c2bf943a-65ac-4264-93b9-73a97e803f53"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:08:52.482085 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.482049    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36880849-3a32-4e98-ae65-8fe933850cc5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "36880849-3a32-4e98-ae65-8fe933850cc5" (UID: "36880849-3a32-4e98-ae65-8fe933850cc5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:08:52.484073 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.484045    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2bf943a-65ac-4264-93b9-73a97e803f53-kube-api-access-pqq8v" (OuterVolumeSpecName: "kube-api-access-pqq8v") pod "c2bf943a-65ac-4264-93b9-73a97e803f53" (UID: "c2bf943a-65ac-4264-93b9-73a97e803f53"). InnerVolumeSpecName "kube-api-access-pqq8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:08:52.484196 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.484070    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36880849-3a32-4e98-ae65-8fe933850cc5-kube-api-access-s2kh9" (OuterVolumeSpecName: "kube-api-access-s2kh9") pod "36880849-3a32-4e98-ae65-8fe933850cc5" (UID: "36880849-3a32-4e98-ae65-8fe933850cc5"). InnerVolumeSpecName "kube-api-access-s2kh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:08:52.582579 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.582537    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2kh9\" (UniqueName: \"kubernetes.io/projected/36880849-3a32-4e98-ae65-8fe933850cc5-kube-api-access-s2kh9\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:08:52.582579 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.582572    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqq8v\" (UniqueName: \"kubernetes.io/projected/c2bf943a-65ac-4264-93b9-73a97e803f53-kube-api-access-pqq8v\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:08:52.582579 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.582581    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c2bf943a-65ac-4264-93b9-73a97e803f53-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:08:52.582579 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:52.582592    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36880849-3a32-4e98-ae65-8fe933850cc5-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:08:53.171872 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.171838    2561 generic.go:356] "Generic (PLEG): container finished" podID="36880849-3a32-4e98-ae65-8fe933850cc5" containerID="9ad33b5cf9a6eafa0516354f28846d1d68fd991a8637ee3fad7958ae18ccf255" exitCode=0
Jul 24 16:08:53.172342 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.171890    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"
Jul 24 16:08:53.172342 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.171912    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg" event={"ID":"36880849-3a32-4e98-ae65-8fe933850cc5","Type":"ContainerDied","Data":"9ad33b5cf9a6eafa0516354f28846d1d68fd991a8637ee3fad7958ae18ccf255"}
Jul 24 16:08:53.172342 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.171957    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg" event={"ID":"36880849-3a32-4e98-ae65-8fe933850cc5","Type":"ContainerDied","Data":"9ff8ac696bbced4aba2a763747ac3d6d370d53adb2f3f719dd78c3c4aa3895f9"}
Jul 24 16:08:53.172342 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.171983    2561 scope.go:117] "RemoveContainer" containerID="9ad33b5cf9a6eafa0516354f28846d1d68fd991a8637ee3fad7958ae18ccf255"
Jul 24 16:08:53.173314 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.173226    2561 generic.go:356] "Generic (PLEG): container finished" podID="c2bf943a-65ac-4264-93b9-73a97e803f53" containerID="33ce8b3345192d2e56f3f410fdba27bb69c1d8b1b01f43fb4c4cae02f8996fc3" exitCode=0
Jul 24 16:08:53.173314 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.173274    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"
Jul 24 16:08:53.173314 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.173265    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp" event={"ID":"c2bf943a-65ac-4264-93b9-73a97e803f53","Type":"ContainerDied","Data":"33ce8b3345192d2e56f3f410fdba27bb69c1d8b1b01f43fb4c4cae02f8996fc3"}
Jul 24 16:08:53.173498 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.173335    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp" event={"ID":"c2bf943a-65ac-4264-93b9-73a97e803f53","Type":"ContainerDied","Data":"0ccb37b2dc431883d07be8d0fcdaa3d20df1792e2b1e8e0c1fff23bf508548df"}
Jul 24 16:08:53.181066 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.181044    2561 scope.go:117] "RemoveContainer" containerID="9ad33b5cf9a6eafa0516354f28846d1d68fd991a8637ee3fad7958ae18ccf255"
Jul 24 16:08:53.181528 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:08:53.181420    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ad33b5cf9a6eafa0516354f28846d1d68fd991a8637ee3fad7958ae18ccf255\": container with ID starting with 9ad33b5cf9a6eafa0516354f28846d1d68fd991a8637ee3fad7958ae18ccf255 not found: ID does not exist" containerID="9ad33b5cf9a6eafa0516354f28846d1d68fd991a8637ee3fad7958ae18ccf255"
Jul 24 16:08:53.181528 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.181448    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ad33b5cf9a6eafa0516354f28846d1d68fd991a8637ee3fad7958ae18ccf255"} err="failed to get container status \"9ad33b5cf9a6eafa0516354f28846d1d68fd991a8637ee3fad7958ae18ccf255\": rpc error: code = NotFound desc = could not find container \"9ad33b5cf9a6eafa0516354f28846d1d68fd991a8637ee3fad7958ae18ccf255\": container with ID starting with 9ad33b5cf9a6eafa0516354f28846d1d68fd991a8637ee3fad7958ae18ccf255 not found: ID does not exist"
Jul 24 16:08:53.181528 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.181465    2561 scope.go:117] "RemoveContainer" containerID="33ce8b3345192d2e56f3f410fdba27bb69c1d8b1b01f43fb4c4cae02f8996fc3"
Jul 24 16:08:53.190512 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.190492    2561 scope.go:117] "RemoveContainer" containerID="33ce8b3345192d2e56f3f410fdba27bb69c1d8b1b01f43fb4c4cae02f8996fc3"
Jul 24 16:08:53.190853 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:08:53.190833    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33ce8b3345192d2e56f3f410fdba27bb69c1d8b1b01f43fb4c4cae02f8996fc3\": container with ID starting with 33ce8b3345192d2e56f3f410fdba27bb69c1d8b1b01f43fb4c4cae02f8996fc3 not found: ID does not exist" containerID="33ce8b3345192d2e56f3f410fdba27bb69c1d8b1b01f43fb4c4cae02f8996fc3"
Jul 24 16:08:53.190949 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.190864    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33ce8b3345192d2e56f3f410fdba27bb69c1d8b1b01f43fb4c4cae02f8996fc3"} err="failed to get container status \"33ce8b3345192d2e56f3f410fdba27bb69c1d8b1b01f43fb4c4cae02f8996fc3\": rpc error: code = NotFound desc = could not find container \"33ce8b3345192d2e56f3f410fdba27bb69c1d8b1b01f43fb4c4cae02f8996fc3\": container with ID starting with 33ce8b3345192d2e56f3f410fdba27bb69c1d8b1b01f43fb4c4cae02f8996fc3 not found: ID does not exist"
Jul 24 16:08:53.197033 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.197007    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"]
Jul 24 16:08:53.202050 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.202026    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5b382fb6-64cf48c6d6-rxngp"]
Jul 24 16:08:53.211822 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.211795    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"]
Jul 24 16:08:53.217752 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:53.217725    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5b382fb6-85676c5c5b-q29sg"]
Jul 24 16:08:54.106339 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:54.106304    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36880849-3a32-4e98-ae65-8fe933850cc5" path="/var/lib/kubelet/pods/36880849-3a32-4e98-ae65-8fe933850cc5/volumes"
Jul 24 16:08:54.106684 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:08:54.106648    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2bf943a-65ac-4264-93b9-73a97e803f53" path="/var/lib/kubelet/pods/c2bf943a-65ac-4264-93b9-73a97e803f53/volumes"
Jul 24 16:11:03.280822 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.280780    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"]
Jul 24 16:11:03.281261 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.281143    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2bf943a-65ac-4264-93b9-73a97e803f53" containerName="payload-processing"
Jul 24 16:11:03.281261 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.281155    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2bf943a-65ac-4264-93b9-73a97e803f53" containerName="payload-processing"
Jul 24 16:11:03.281261 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.281167    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36880849-3a32-4e98-ae65-8fe933850cc5" containerName="payload-pre-processing"
Jul 24 16:11:03.281261 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.281174    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="36880849-3a32-4e98-ae65-8fe933850cc5" containerName="payload-pre-processing"
Jul 24 16:11:03.281261 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.281223    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="36880849-3a32-4e98-ae65-8fe933850cc5" containerName="payload-pre-processing"
Jul 24 16:11:03.281261 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.281234    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2bf943a-65ac-4264-93b9-73a97e803f53" containerName="payload-processing"
Jul 24 16:11:03.284288 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.284269    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"
Jul 24 16:11:03.286829 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.286808    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-e03b82\""
Jul 24 16:11:03.295581 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.295555    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"]
Jul 24 16:11:03.322985 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.322948    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"]
Jul 24 16:11:03.326390 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.326371    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"
Jul 24 16:11:03.328834 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.328811    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-e03b82-dockercfg-n54dh\""
Jul 24 16:11:03.335920 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.335889    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"]
Jul 24 16:11:03.423004 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.422972    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84209967-3ca5-4bb1-b943-d830c8e055e7-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n\" (UID: \"84209967-3ca5-4bb1-b943-d830c8e055e7\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"
Jul 24 16:11:03.423004 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.423010    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f63aded2-88e6-4e39-82ae-93939f1082ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms\" (UID: \"f63aded2-88e6-4e39-82ae-93939f1082ca\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"
Jul 24 16:11:03.423250 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.423033    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxbjv\" (UniqueName: \"kubernetes.io/projected/f63aded2-88e6-4e39-82ae-93939f1082ca-kube-api-access-nxbjv\") pod \"payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms\" (UID: \"f63aded2-88e6-4e39-82ae-93939f1082ca\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"
Jul 24 16:11:03.423250 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.423172    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdvxh\" (UniqueName: \"kubernetes.io/projected/84209967-3ca5-4bb1-b943-d830c8e055e7-kube-api-access-fdvxh\") pod \"payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n\" (UID: \"84209967-3ca5-4bb1-b943-d830c8e055e7\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"
Jul 24 16:11:03.523638 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.523599    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdvxh\" (UniqueName: \"kubernetes.io/projected/84209967-3ca5-4bb1-b943-d830c8e055e7-kube-api-access-fdvxh\") pod \"payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n\" (UID: \"84209967-3ca5-4bb1-b943-d830c8e055e7\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"
Jul 24 16:11:03.523901 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.523690    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84209967-3ca5-4bb1-b943-d830c8e055e7-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n\" (UID: \"84209967-3ca5-4bb1-b943-d830c8e055e7\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"
Jul 24 16:11:03.523901 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.523725    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f63aded2-88e6-4e39-82ae-93939f1082ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms\" (UID: \"f63aded2-88e6-4e39-82ae-93939f1082ca\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"
Jul 24 16:11:03.523901 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.523848    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxbjv\" (UniqueName: \"kubernetes.io/projected/f63aded2-88e6-4e39-82ae-93939f1082ca-kube-api-access-nxbjv\") pod \"payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms\" (UID: \"f63aded2-88e6-4e39-82ae-93939f1082ca\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"
Jul 24 16:11:03.524451 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.524431    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f63aded2-88e6-4e39-82ae-93939f1082ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms\" (UID: \"f63aded2-88e6-4e39-82ae-93939f1082ca\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"
Jul 24 16:11:03.524520 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.524450    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84209967-3ca5-4bb1-b943-d830c8e055e7-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n\" (UID: \"84209967-3ca5-4bb1-b943-d830c8e055e7\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"
Jul 24 16:11:03.533612 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.533532    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdvxh\" (UniqueName: \"kubernetes.io/projected/84209967-3ca5-4bb1-b943-d830c8e055e7-kube-api-access-fdvxh\") pod \"payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n\" (UID: \"84209967-3ca5-4bb1-b943-d830c8e055e7\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"
Jul 24 16:11:03.533612 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.533555    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxbjv\" (UniqueName: \"kubernetes.io/projected/f63aded2-88e6-4e39-82ae-93939f1082ca-kube-api-access-nxbjv\") pod \"payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms\" (UID: \"f63aded2-88e6-4e39-82ae-93939f1082ca\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"
Jul 24 16:11:03.595577 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.595521    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"
Jul 24 16:11:03.638642 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.638296    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"
Jul 24 16:11:03.760040 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.759791    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"]
Jul 24 16:11:03.760355 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:11:03.760316    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf63aded2_88e6_4e39_82ae_93939f1082ca.slice/crio-4f1a7cfb615c76869a4b6fc49f4b0204e12ce5aff6a04509904fc8e7d7a45e75 WatchSource:0}: Error finding container 4f1a7cfb615c76869a4b6fc49f4b0204e12ce5aff6a04509904fc8e7d7a45e75: Status 404 returned error can't find the container with id 4f1a7cfb615c76869a4b6fc49f4b0204e12ce5aff6a04509904fc8e7d7a45e75
Jul 24 16:11:03.789108 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.789031    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms" event={"ID":"f63aded2-88e6-4e39-82ae-93939f1082ca","Type":"ContainerStarted","Data":"4f1a7cfb615c76869a4b6fc49f4b0204e12ce5aff6a04509904fc8e7d7a45e75"}
Jul 24 16:11:03.814006 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:03.813979    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"]
Jul 24 16:11:03.816779 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:11:03.816756    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84209967_3ca5_4bb1_b943_d830c8e055e7.slice/crio-e56147715185be24affbcdd8b0fd84c2ea7eb3fb255b875d5f35f5a77011d0e4 WatchSource:0}: Error finding container e56147715185be24affbcdd8b0fd84c2ea7eb3fb255b875d5f35f5a77011d0e4: Status 404 returned error can't find the container with id e56147715185be24affbcdd8b0fd84c2ea7eb3fb255b875d5f35f5a77011d0e4
Jul 24 16:11:04.794724 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:04.794650    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms" event={"ID":"f63aded2-88e6-4e39-82ae-93939f1082ca","Type":"ContainerStarted","Data":"bfcc47946a1d78f8c0b3f78aa3585f8bbdb2c49aaf087b6faea0cbc42e314cba"}
Jul 24 16:11:04.795190 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:04.794799    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"
Jul 24 16:11:04.796211 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:04.796183    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n" event={"ID":"84209967-3ca5-4bb1-b943-d830c8e055e7","Type":"ContainerStarted","Data":"3812674b15d6fd13a288cb8200f62a57f90612608628b57406155f8b1d1c6f0a"}
Jul 24 16:11:04.796316 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:04.796235    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"
Jul 24 16:11:04.796316 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:04.796253    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n" event={"ID":"84209967-3ca5-4bb1-b943-d830c8e055e7","Type":"ContainerStarted","Data":"e56147715185be24affbcdd8b0fd84c2ea7eb3fb255b875d5f35f5a77011d0e4"}
Jul 24 16:11:04.811735 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:04.811684    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms" podStartSLOduration=1.8116461209999999 podStartE2EDuration="1.811646121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:11:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:11:04.809998321 +0000 UTC m=+2093.220708938" watchObservedRunningTime="2026-07-24 16:11:04.811646121 +0000 UTC m=+2093.222356738"
Jul 24 16:11:04.829061 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:04.828995    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n" podStartSLOduration=1.828979537 podStartE2EDuration="1.828979537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:11:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:11:04.827054738 +0000 UTC m=+2093.237765356" watchObservedRunningTime="2026-07-24 16:11:04.828979537 +0000 UTC m=+2093.239690153"
Jul 24 16:11:11.545630 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.545591    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"]
Jul 24 16:11:11.549861 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.549826    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"
Jul 24 16:11:11.552357 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.552320    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-aedd03-dockercfg-t8x7g\""
Jul 24 16:11:11.552490 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.552374    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-aedd03\""
Jul 24 16:11:11.560408 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.560373    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"]
Jul 24 16:11:11.567212 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.567180    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"]
Jul 24 16:11:11.570652 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.570628    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"
Jul 24 16:11:11.584744 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.584706    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"]
Jul 24 16:11:11.697599 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.697561    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vg24\" (UniqueName: \"kubernetes.io/projected/8315a6fe-d0e4-4780-a999-a535a974f783-kube-api-access-7vg24\") pod \"payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74\" (UID: \"8315a6fe-d0e4-4780-a999-a535a974f783\") " pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"
Jul 24 16:11:11.697880 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.697618    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ddff3f0-b95e-44c9-a453-7de15ee35ff1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp\" (UID: \"6ddff3f0-b95e-44c9-a453-7de15ee35ff1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"
Jul 24 16:11:11.697880 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.697643    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvrw7\" (UniqueName: \"kubernetes.io/projected/6ddff3f0-b95e-44c9-a453-7de15ee35ff1-kube-api-access-cvrw7\") pod \"payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp\" (UID: \"6ddff3f0-b95e-44c9-a453-7de15ee35ff1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"
Jul 24 16:11:11.697880 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.697710    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8315a6fe-d0e4-4780-a999-a535a974f783-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74\" (UID: \"8315a6fe-d0e4-4780-a999-a535a974f783\") " pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"
Jul 24 16:11:11.799081 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.798984    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ddff3f0-b95e-44c9-a453-7de15ee35ff1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp\" (UID: \"6ddff3f0-b95e-44c9-a453-7de15ee35ff1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"
Jul 24 16:11:11.799081 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.799030    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvrw7\" (UniqueName: \"kubernetes.io/projected/6ddff3f0-b95e-44c9-a453-7de15ee35ff1-kube-api-access-cvrw7\") pod \"payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp\" (UID: \"6ddff3f0-b95e-44c9-a453-7de15ee35ff1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"
Jul 24 16:11:11.799081 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.799062    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8315a6fe-d0e4-4780-a999-a535a974f783-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74\" (UID: \"8315a6fe-d0e4-4780-a999-a535a974f783\") " pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"
Jul 24 16:11:11.799327 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.799132    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vg24\" (UniqueName: \"kubernetes.io/projected/8315a6fe-d0e4-4780-a999-a535a974f783-kube-api-access-7vg24\") pod \"payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74\" (UID: \"8315a6fe-d0e4-4780-a999-a535a974f783\") " pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"
Jul 24 16:11:11.799840 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.799820    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8315a6fe-d0e4-4780-a999-a535a974f783-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74\" (UID: \"8315a6fe-d0e4-4780-a999-a535a974f783\") " pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"
Jul 24 16:11:11.799884 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.799832    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ddff3f0-b95e-44c9-a453-7de15ee35ff1-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp\" (UID: \"6ddff3f0-b95e-44c9-a453-7de15ee35ff1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"
Jul 24 16:11:11.807393 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.807364    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvrw7\" (UniqueName: \"kubernetes.io/projected/6ddff3f0-b95e-44c9-a453-7de15ee35ff1-kube-api-access-cvrw7\") pod \"payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp\" (UID: \"6ddff3f0-b95e-44c9-a453-7de15ee35ff1\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"
Jul 24 16:11:11.807535 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.807364    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vg24\" (UniqueName: \"kubernetes.io/projected/8315a6fe-d0e4-4780-a999-a535a974f783-kube-api-access-7vg24\") pod \"payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74\" (UID: \"8315a6fe-d0e4-4780-a999-a535a974f783\") " pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"
Jul 24 16:11:11.863165 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.863125    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"
Jul 24 16:11:11.881874 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:11.881830    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"
Jul 24 16:11:12.024196 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:12.024069    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"]
Jul 24 16:11:12.028144 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:11:12.028113    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8315a6fe_d0e4_4780_a999_a535a974f783.slice/crio-049e9719dfe887199cc16cac227da6a16e45bed81f7fa6f45844b3a970965714 WatchSource:0}: Error finding container 049e9719dfe887199cc16cac227da6a16e45bed81f7fa6f45844b3a970965714: Status 404 returned error can't find the container with id 049e9719dfe887199cc16cac227da6a16e45bed81f7fa6f45844b3a970965714
Jul 24 16:11:12.061832 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:12.061800    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"]
Jul 24 16:11:12.065923 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:11:12.065894    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ddff3f0_b95e_44c9_a453_7de15ee35ff1.slice/crio-87fc0e33f555270bcfe2742946ca8038a9eebdac361ae9d64dd968558a5b9935 WatchSource:0}: Error finding container 87fc0e33f555270bcfe2742946ca8038a9eebdac361ae9d64dd968558a5b9935: Status 404 returned error can't find the container with id 87fc0e33f555270bcfe2742946ca8038a9eebdac361ae9d64dd968558a5b9935
Jul 24 16:11:12.842892 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:12.842853    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74" event={"ID":"8315a6fe-d0e4-4780-a999-a535a974f783","Type":"ContainerStarted","Data":"edf66ff1a9322ad18887ce3f362e8f7b6e637fa9b2354e53fb45dc51b352615f"}
Jul 24 16:11:12.843495 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:12.843471    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"
Jul 24 16:11:12.843618 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:12.843508    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74" event={"ID":"8315a6fe-d0e4-4780-a999-a535a974f783","Type":"ContainerStarted","Data":"049e9719dfe887199cc16cac227da6a16e45bed81f7fa6f45844b3a970965714"}
Jul 24 16:11:12.845120 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:12.845092    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp" event={"ID":"6ddff3f0-b95e-44c9-a453-7de15ee35ff1","Type":"ContainerStarted","Data":"d9a3d1265ab470b591901171bf00d221e5f99b08a5b386cbcde80353b5ef0739"}
Jul 24 16:11:12.845250 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:12.845147    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp" event={"ID":"6ddff3f0-b95e-44c9-a453-7de15ee35ff1","Type":"ContainerStarted","Data":"87fc0e33f555270bcfe2742946ca8038a9eebdac361ae9d64dd968558a5b9935"}
Jul 24 16:11:12.845250 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:12.845200    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"
Jul 24 16:11:12.923103 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:12.923041    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp" podStartSLOduration=1.923023701 podStartE2EDuration="1.923023701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:11:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:11:12.921281042 +0000 UTC m=+2101.331991661" watchObservedRunningTime="2026-07-24 16:11:12.923023701 +0000 UTC m=+2101.333734318"
Jul 24 16:11:12.924432 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:12.924385    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74" podStartSLOduration=1.9243710859999998 podStartE2EDuration="1.924371086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:11:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:11:12.889596935 +0000 UTC m=+2101.300307551" watchObservedRunningTime="2026-07-24 16:11:12.924371086 +0000 UTC m=+2101.335081702"
Jul 24 16:11:15.808541 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:15.808492    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms" podUID="f63aded2-88e6-4e39-82ae-93939f1082ca" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.58:9004: connect: connection refused"
Jul 24 16:11:15.809004 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:15.808990    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"
Jul 24 16:11:22.795274 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.795227    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"]
Jul 24 16:11:22.799307 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.799278    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:22.802124 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.802092    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e03b82\"/\"kube-root-ca.crt\""
Jul 24 16:11:22.803036 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.802995    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e03b82\"/\"test-model-e03b82-kserve-self-signed-certs\""
Jul 24 16:11:22.803036 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.803005    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e03b82\"/\"openshift-service-ca.crt\""
Jul 24 16:11:22.803036 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.803005    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e03b82\"/\"default-dockercfg-pd2ch\""
Jul 24 16:11:22.809632 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.809600    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"]
Jul 24 16:11:22.896030 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.895995    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmgvc\" (UniqueName: \"kubernetes.io/projected/4e75db08-517e-4e5f-8e62-a2c102db8bbd-kube-api-access-gmgvc\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:22.896232 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.896042    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-dshm\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:22.896232 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.896099    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e75db08-517e-4e5f-8e62-a2c102db8bbd-tls-certs\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:22.896232 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.896130    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-home\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:22.896232 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.896165    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-model-cache\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:22.896232 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.896185    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-tmp-dir\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:22.996717 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.996638    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmgvc\" (UniqueName: \"kubernetes.io/projected/4e75db08-517e-4e5f-8e62-a2c102db8bbd-kube-api-access-gmgvc\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:22.996717 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.996728    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-dshm\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:22.997006 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.996756    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e75db08-517e-4e5f-8e62-a2c102db8bbd-tls-certs\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:22.997006 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.996773    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-home\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:22.997006 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.996802    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-model-cache\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:22.997006 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.996820    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-tmp-dir\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:22.997280 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.997252    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-home\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:22.997280 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.997268    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-tmp-dir\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:22.997479 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.997311    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-model-cache\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:22.999280 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.999254    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-dshm\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:22.999498 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:22.999481    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e75db08-517e-4e5f-8e62-a2c102db8bbd-tls-certs\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:23.006348 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:23.006317    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmgvc\" (UniqueName: \"kubernetes.io/projected/4e75db08-517e-4e5f-8e62-a2c102db8bbd-kube-api-access-gmgvc\") pod \"test-model-e03b82-kserve-6bfc46599c-l5jkx\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") " pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:23.110804 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:23.110705    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:23.265043 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:23.264882    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"]
Jul 24 16:11:23.267946 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:11:23.267914    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e75db08_517e_4e5f_8e62_a2c102db8bbd.slice/crio-8f98003d17a44d5e990ac34bcb00ce30f63096c4d269cd272132dd19780aad9d WatchSource:0}: Error finding container 8f98003d17a44d5e990ac34bcb00ce30f63096c4d269cd272132dd19780aad9d: Status 404 returned error can't find the container with id 8f98003d17a44d5e990ac34bcb00ce30f63096c4d269cd272132dd19780aad9d
Jul 24 16:11:23.852880 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:23.852845    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"
Jul 24 16:11:23.853300 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:23.853281    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"
Jul 24 16:11:23.902782 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:23.902743    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx" event={"ID":"4e75db08-517e-4e5f-8e62-a2c102db8bbd","Type":"ContainerStarted","Data":"efd369c34e8d71329909e16460d2d25f7b73436dbbe0f2a088a2d1181496a733"}
Jul 24 16:11:23.902935 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:23.902791    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx" event={"ID":"4e75db08-517e-4e5f-8e62-a2c102db8bbd","Type":"ContainerStarted","Data":"8f98003d17a44d5e990ac34bcb00ce30f63096c4d269cd272132dd19780aad9d"}
Jul 24 16:11:23.922364 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:23.922303    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx" podStartSLOduration=1.922280196 podStartE2EDuration="1.922280196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:11:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:11:23.920581316 +0000 UTC m=+2112.331291933" watchObservedRunningTime="2026-07-24 16:11:23.922280196 +0000 UTC m=+2112.332990814"
Jul 24 16:11:25.807606 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:25.807573    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"
Jul 24 16:11:33.111766 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:33.111724    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:33.112269 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:33.111779    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:33.125175 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:33.125143    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:11:33.963956 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:11:33.963925    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:12:40.221509 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:40.221472    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"]
Jul 24 16:12:52.123250 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.123210    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"]
Jul 24 16:12:52.123890 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.123462    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n" podUID="84209967-3ca5-4bb1-b943-d830c8e055e7" containerName="payload-processing" containerID="cri-o://3812674b15d6fd13a288cb8200f62a57f90612608628b57406155f8b1d1c6f0a" gracePeriod=30
Jul 24 16:12:52.129255 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.129202    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"]
Jul 24 16:12:52.129504 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.129452    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms" podUID="f63aded2-88e6-4e39-82ae-93939f1082ca" containerName="payload-pre-processing" containerID="cri-o://bfcc47946a1d78f8c0b3f78aa3585f8bbdb2c49aaf087b6faea0cbc42e314cba" gracePeriod=30
Jul 24 16:12:52.343045 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.343008    2561 generic.go:356] "Generic (PLEG): container finished" podID="f63aded2-88e6-4e39-82ae-93939f1082ca" containerID="bfcc47946a1d78f8c0b3f78aa3585f8bbdb2c49aaf087b6faea0cbc42e314cba" exitCode=0
Jul 24 16:12:52.343310 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.343178    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms" event={"ID":"f63aded2-88e6-4e39-82ae-93939f1082ca","Type":"ContainerDied","Data":"bfcc47946a1d78f8c0b3f78aa3585f8bbdb2c49aaf087b6faea0cbc42e314cba"}
Jul 24 16:12:52.349431 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.349334    2561 generic.go:356] "Generic (PLEG): container finished" podID="84209967-3ca5-4bb1-b943-d830c8e055e7" containerID="3812674b15d6fd13a288cb8200f62a57f90612608628b57406155f8b1d1c6f0a" exitCode=0
Jul 24 16:12:52.349590 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.349428    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n" event={"ID":"84209967-3ca5-4bb1-b943-d830c8e055e7","Type":"ContainerDied","Data":"3812674b15d6fd13a288cb8200f62a57f90612608628b57406155f8b1d1c6f0a"}
Jul 24 16:12:52.418234 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.418199    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"
Jul 24 16:12:52.421382 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.421356    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"
Jul 24 16:12:52.511992 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.511958    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdvxh\" (UniqueName: \"kubernetes.io/projected/84209967-3ca5-4bb1-b943-d830c8e055e7-kube-api-access-fdvxh\") pod \"84209967-3ca5-4bb1-b943-d830c8e055e7\" (UID: \"84209967-3ca5-4bb1-b943-d830c8e055e7\") "
Jul 24 16:12:52.512185 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.512016    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84209967-3ca5-4bb1-b943-d830c8e055e7-plugins-config-volume\") pod \"84209967-3ca5-4bb1-b943-d830c8e055e7\" (UID: \"84209967-3ca5-4bb1-b943-d830c8e055e7\") "
Jul 24 16:12:52.512185 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.512037    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxbjv\" (UniqueName: \"kubernetes.io/projected/f63aded2-88e6-4e39-82ae-93939f1082ca-kube-api-access-nxbjv\") pod \"f63aded2-88e6-4e39-82ae-93939f1082ca\" (UID: \"f63aded2-88e6-4e39-82ae-93939f1082ca\") "
Jul 24 16:12:52.512185 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.512121    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f63aded2-88e6-4e39-82ae-93939f1082ca-plugins-config-volume\") pod \"f63aded2-88e6-4e39-82ae-93939f1082ca\" (UID: \"f63aded2-88e6-4e39-82ae-93939f1082ca\") "
Jul 24 16:12:52.512465 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.512438    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84209967-3ca5-4bb1-b943-d830c8e055e7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "84209967-3ca5-4bb1-b943-d830c8e055e7" (UID: "84209967-3ca5-4bb1-b943-d830c8e055e7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:12:52.512611 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.512578    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f63aded2-88e6-4e39-82ae-93939f1082ca-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f63aded2-88e6-4e39-82ae-93939f1082ca" (UID: "f63aded2-88e6-4e39-82ae-93939f1082ca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:12:52.514647 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.514622    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84209967-3ca5-4bb1-b943-d830c8e055e7-kube-api-access-fdvxh" (OuterVolumeSpecName: "kube-api-access-fdvxh") pod "84209967-3ca5-4bb1-b943-d830c8e055e7" (UID: "84209967-3ca5-4bb1-b943-d830c8e055e7"). InnerVolumeSpecName "kube-api-access-fdvxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:12:52.514796 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.514775    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f63aded2-88e6-4e39-82ae-93939f1082ca-kube-api-access-nxbjv" (OuterVolumeSpecName: "kube-api-access-nxbjv") pod "f63aded2-88e6-4e39-82ae-93939f1082ca" (UID: "f63aded2-88e6-4e39-82ae-93939f1082ca"). InnerVolumeSpecName "kube-api-access-nxbjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:12:52.613292 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.613253    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdvxh\" (UniqueName: \"kubernetes.io/projected/84209967-3ca5-4bb1-b943-d830c8e055e7-kube-api-access-fdvxh\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:12:52.613292 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.613286    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84209967-3ca5-4bb1-b943-d830c8e055e7-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:12:52.613292 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.613295    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxbjv\" (UniqueName: \"kubernetes.io/projected/f63aded2-88e6-4e39-82ae-93939f1082ca-kube-api-access-nxbjv\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:12:52.613292 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:52.613305    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f63aded2-88e6-4e39-82ae-93939f1082ca-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:12:53.356407 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:53.356369    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"
Jul 24 16:12:53.356407 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:53.356391    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms" event={"ID":"f63aded2-88e6-4e39-82ae-93939f1082ca","Type":"ContainerDied","Data":"4f1a7cfb615c76869a4b6fc49f4b0204e12ce5aff6a04509904fc8e7d7a45e75"}
Jul 24 16:12:53.356935 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:53.356455    2561 scope.go:117] "RemoveContainer" containerID="bfcc47946a1d78f8c0b3f78aa3585f8bbdb2c49aaf087b6faea0cbc42e314cba"
Jul 24 16:12:53.357859 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:53.357792    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n" event={"ID":"84209967-3ca5-4bb1-b943-d830c8e055e7","Type":"ContainerDied","Data":"e56147715185be24affbcdd8b0fd84c2ea7eb3fb255b875d5f35f5a77011d0e4"}
Jul 24 16:12:53.357859 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:53.357814    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"
Jul 24 16:12:53.365938 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:53.365916    2561 scope.go:117] "RemoveContainer" containerID="3812674b15d6fd13a288cb8200f62a57f90612608628b57406155f8b1d1c6f0a"
Jul 24 16:12:53.381775 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:53.381743    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"]
Jul 24 16:12:53.386139 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:53.386109    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e03b82-c76d5b9bd-9tqms"]
Jul 24 16:12:53.397396 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:53.397310    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"]
Jul 24 16:12:53.405781 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:53.405748    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e03b82-6f97599bbb-t8d2n"]
Jul 24 16:12:54.107421 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:54.107383    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84209967-3ca5-4bb1-b943-d830c8e055e7" path="/var/lib/kubelet/pods/84209967-3ca5-4bb1-b943-d830c8e055e7/volumes"
Jul 24 16:12:54.107815 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:54.107800    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f63aded2-88e6-4e39-82ae-93939f1082ca" path="/var/lib/kubelet/pods/f63aded2-88e6-4e39-82ae-93939f1082ca/volumes"
Jul 24 16:12:55.243516 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.243472    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx" podUID="4e75db08-517e-4e5f-8e62-a2c102db8bbd" containerName="main" containerID="cri-o://efd369c34e8d71329909e16460d2d25f7b73436dbbe0f2a088a2d1181496a733" gracePeriod=45
Jul 24 16:12:55.371366 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.371336    2561 generic.go:356] "Generic (PLEG): container finished" podID="4e75db08-517e-4e5f-8e62-a2c102db8bbd" containerID="efd369c34e8d71329909e16460d2d25f7b73436dbbe0f2a088a2d1181496a733" exitCode=0
Jul 24 16:12:55.371554 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.371420    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx" event={"ID":"4e75db08-517e-4e5f-8e62-a2c102db8bbd","Type":"ContainerDied","Data":"efd369c34e8d71329909e16460d2d25f7b73436dbbe0f2a088a2d1181496a733"}
Jul 24 16:12:55.504348 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.504273    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:12:55.539421 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.539391    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-dshm\") pod \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") "
Jul 24 16:12:55.539593 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.539469    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-model-cache\") pod \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") "
Jul 24 16:12:55.539593 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.539514    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e75db08-517e-4e5f-8e62-a2c102db8bbd-tls-certs\") pod \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") "
Jul 24 16:12:55.539593 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.539535    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-home\") pod \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") "
Jul 24 16:12:55.539801 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.539637    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-tmp-dir\") pod \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") "
Jul 24 16:12:55.539801 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.539688    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmgvc\" (UniqueName: \"kubernetes.io/projected/4e75db08-517e-4e5f-8e62-a2c102db8bbd-kube-api-access-gmgvc\") pod \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\" (UID: \"4e75db08-517e-4e5f-8e62-a2c102db8bbd\") "
Jul 24 16:12:55.539905 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.539824    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-model-cache" (OuterVolumeSpecName: "model-cache") pod "4e75db08-517e-4e5f-8e62-a2c102db8bbd" (UID: "4e75db08-517e-4e5f-8e62-a2c102db8bbd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:12:55.540009 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.539958    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-model-cache\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:12:55.540112 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.540088    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4e75db08-517e-4e5f-8e62-a2c102db8bbd" (UID: "4e75db08-517e-4e5f-8e62-a2c102db8bbd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:12:55.540188 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.540154    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-home" (OuterVolumeSpecName: "home") pod "4e75db08-517e-4e5f-8e62-a2c102db8bbd" (UID: "4e75db08-517e-4e5f-8e62-a2c102db8bbd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:12:55.542023 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.541994    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e75db08-517e-4e5f-8e62-a2c102db8bbd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e75db08-517e-4e5f-8e62-a2c102db8bbd" (UID: "4e75db08-517e-4e5f-8e62-a2c102db8bbd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:12:55.542023 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.542000    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e75db08-517e-4e5f-8e62-a2c102db8bbd-kube-api-access-gmgvc" (OuterVolumeSpecName: "kube-api-access-gmgvc") pod "4e75db08-517e-4e5f-8e62-a2c102db8bbd" (UID: "4e75db08-517e-4e5f-8e62-a2c102db8bbd"). InnerVolumeSpecName "kube-api-access-gmgvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:12:55.542199 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.542184    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-dshm" (OuterVolumeSpecName: "dshm") pod "4e75db08-517e-4e5f-8e62-a2c102db8bbd" (UID: "4e75db08-517e-4e5f-8e62-a2c102db8bbd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:12:55.641168 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.641123    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-tmp-dir\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:12:55.641168 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.641159    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmgvc\" (UniqueName: \"kubernetes.io/projected/4e75db08-517e-4e5f-8e62-a2c102db8bbd-kube-api-access-gmgvc\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:12:55.641168 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.641169    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-dshm\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:12:55.641168 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.641178    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e75db08-517e-4e5f-8e62-a2c102db8bbd-tls-certs\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:12:55.641507 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:55.641186    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e75db08-517e-4e5f-8e62-a2c102db8bbd-home\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:12:56.377561 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:56.377464    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx" event={"ID":"4e75db08-517e-4e5f-8e62-a2c102db8bbd","Type":"ContainerDied","Data":"8f98003d17a44d5e990ac34bcb00ce30f63096c4d269cd272132dd19780aad9d"}
Jul 24 16:12:56.377561 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:56.377489    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"
Jul 24 16:12:56.377561 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:56.377515    2561 scope.go:117] "RemoveContainer" containerID="efd369c34e8d71329909e16460d2d25f7b73436dbbe0f2a088a2d1181496a733"
Jul 24 16:12:56.396107 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:56.396066    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"]
Jul 24 16:12:56.399085 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:56.399053    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-e03b82/test-model-e03b82-kserve-6bfc46599c-l5jkx"]
Jul 24 16:12:58.106329 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:12:58.106287    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e75db08-517e-4e5f-8e62-a2c102db8bbd" path="/var/lib/kubelet/pods/4e75db08-517e-4e5f-8e62-a2c102db8bbd/volumes"
Jul 24 16:13:14.485054 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:14.485015    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"]
Jul 24 16:13:14.485527 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:14.485296    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74" podUID="8315a6fe-d0e4-4780-a999-a535a974f783" containerName="payload-processing" containerID="cri-o://edf66ff1a9322ad18887ce3f362e8f7b6e637fa9b2354e53fb45dc51b352615f" gracePeriod=30
Jul 24 16:13:14.491869 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:14.491836    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"]
Jul 24 16:13:14.492100 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:14.492076    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp" podUID="6ddff3f0-b95e-44c9-a453-7de15ee35ff1" containerName="payload-pre-processing" containerID="cri-o://d9a3d1265ab470b591901171bf00d221e5f99b08a5b386cbcde80353b5ef0739" gracePeriod=30
Jul 24 16:13:14.806154 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:14.806127    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"
Jul 24 16:13:14.809786 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:14.809753    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"
Jul 24 16:13:14.919870 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:14.919834    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ddff3f0-b95e-44c9-a453-7de15ee35ff1-plugins-config-volume\") pod \"6ddff3f0-b95e-44c9-a453-7de15ee35ff1\" (UID: \"6ddff3f0-b95e-44c9-a453-7de15ee35ff1\") "
Jul 24 16:13:14.920083 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:14.919887    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8315a6fe-d0e4-4780-a999-a535a974f783-plugins-config-volume\") pod \"8315a6fe-d0e4-4780-a999-a535a974f783\" (UID: \"8315a6fe-d0e4-4780-a999-a535a974f783\") "
Jul 24 16:13:14.920083 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:14.919961    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvrw7\" (UniqueName: \"kubernetes.io/projected/6ddff3f0-b95e-44c9-a453-7de15ee35ff1-kube-api-access-cvrw7\") pod \"6ddff3f0-b95e-44c9-a453-7de15ee35ff1\" (UID: \"6ddff3f0-b95e-44c9-a453-7de15ee35ff1\") "
Jul 24 16:13:14.920083 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:14.920007    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vg24\" (UniqueName: \"kubernetes.io/projected/8315a6fe-d0e4-4780-a999-a535a974f783-kube-api-access-7vg24\") pod \"8315a6fe-d0e4-4780-a999-a535a974f783\" (UID: \"8315a6fe-d0e4-4780-a999-a535a974f783\") "
Jul 24 16:13:14.920254 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:14.920232    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ddff3f0-b95e-44c9-a453-7de15ee35ff1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6ddff3f0-b95e-44c9-a453-7de15ee35ff1" (UID: "6ddff3f0-b95e-44c9-a453-7de15ee35ff1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:13:14.920323 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:14.920299    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8315a6fe-d0e4-4780-a999-a535a974f783-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8315a6fe-d0e4-4780-a999-a535a974f783" (UID: "8315a6fe-d0e4-4780-a999-a535a974f783"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:13:14.922482 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:14.922445    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8315a6fe-d0e4-4780-a999-a535a974f783-kube-api-access-7vg24" (OuterVolumeSpecName: "kube-api-access-7vg24") pod "8315a6fe-d0e4-4780-a999-a535a974f783" (UID: "8315a6fe-d0e4-4780-a999-a535a974f783"). InnerVolumeSpecName "kube-api-access-7vg24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:13:14.922482 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:14.922456    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ddff3f0-b95e-44c9-a453-7de15ee35ff1-kube-api-access-cvrw7" (OuterVolumeSpecName: "kube-api-access-cvrw7") pod "6ddff3f0-b95e-44c9-a453-7de15ee35ff1" (UID: "6ddff3f0-b95e-44c9-a453-7de15ee35ff1"). InnerVolumeSpecName "kube-api-access-cvrw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:13:15.021236 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.021186    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ddff3f0-b95e-44c9-a453-7de15ee35ff1-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:13:15.021236 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.021227    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8315a6fe-d0e4-4780-a999-a535a974f783-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:13:15.021236 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.021242    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvrw7\" (UniqueName: \"kubernetes.io/projected/6ddff3f0-b95e-44c9-a453-7de15ee35ff1-kube-api-access-cvrw7\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:13:15.021236 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.021251    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vg24\" (UniqueName: \"kubernetes.io/projected/8315a6fe-d0e4-4780-a999-a535a974f783-kube-api-access-7vg24\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:13:15.469744 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.469706    2561 generic.go:356] "Generic (PLEG): container finished" podID="8315a6fe-d0e4-4780-a999-a535a974f783" containerID="edf66ff1a9322ad18887ce3f362e8f7b6e637fa9b2354e53fb45dc51b352615f" exitCode=0
Jul 24 16:13:15.469968 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.469754    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"
Jul 24 16:13:15.469968 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.469797    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74" event={"ID":"8315a6fe-d0e4-4780-a999-a535a974f783","Type":"ContainerDied","Data":"edf66ff1a9322ad18887ce3f362e8f7b6e637fa9b2354e53fb45dc51b352615f"}
Jul 24 16:13:15.469968 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.469865    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74" event={"ID":"8315a6fe-d0e4-4780-a999-a535a974f783","Type":"ContainerDied","Data":"049e9719dfe887199cc16cac227da6a16e45bed81f7fa6f45844b3a970965714"}
Jul 24 16:13:15.469968 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.469893    2561 scope.go:117] "RemoveContainer" containerID="edf66ff1a9322ad18887ce3f362e8f7b6e637fa9b2354e53fb45dc51b352615f"
Jul 24 16:13:15.471375 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.471348    2561 generic.go:356] "Generic (PLEG): container finished" podID="6ddff3f0-b95e-44c9-a453-7de15ee35ff1" containerID="d9a3d1265ab470b591901171bf00d221e5f99b08a5b386cbcde80353b5ef0739" exitCode=0
Jul 24 16:13:15.471511 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.471385    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp" event={"ID":"6ddff3f0-b95e-44c9-a453-7de15ee35ff1","Type":"ContainerDied","Data":"d9a3d1265ab470b591901171bf00d221e5f99b08a5b386cbcde80353b5ef0739"}
Jul 24 16:13:15.471511 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.471414    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"
Jul 24 16:13:15.471511 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.471452    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp" event={"ID":"6ddff3f0-b95e-44c9-a453-7de15ee35ff1","Type":"ContainerDied","Data":"87fc0e33f555270bcfe2742946ca8038a9eebdac361ae9d64dd968558a5b9935"}
Jul 24 16:13:15.479469 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.479385    2561 scope.go:117] "RemoveContainer" containerID="edf66ff1a9322ad18887ce3f362e8f7b6e637fa9b2354e53fb45dc51b352615f"
Jul 24 16:13:15.479913 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:13:15.479888    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edf66ff1a9322ad18887ce3f362e8f7b6e637fa9b2354e53fb45dc51b352615f\": container with ID starting with edf66ff1a9322ad18887ce3f362e8f7b6e637fa9b2354e53fb45dc51b352615f not found: ID does not exist" containerID="edf66ff1a9322ad18887ce3f362e8f7b6e637fa9b2354e53fb45dc51b352615f"
Jul 24 16:13:15.479988 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.479925    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edf66ff1a9322ad18887ce3f362e8f7b6e637fa9b2354e53fb45dc51b352615f"} err="failed to get container status \"edf66ff1a9322ad18887ce3f362e8f7b6e637fa9b2354e53fb45dc51b352615f\": rpc error: code = NotFound desc = could not find container \"edf66ff1a9322ad18887ce3f362e8f7b6e637fa9b2354e53fb45dc51b352615f\": container with ID starting with edf66ff1a9322ad18887ce3f362e8f7b6e637fa9b2354e53fb45dc51b352615f not found: ID does not exist"
Jul 24 16:13:15.479988 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.479945    2561 scope.go:117] "RemoveContainer" containerID="d9a3d1265ab470b591901171bf00d221e5f99b08a5b386cbcde80353b5ef0739"
Jul 24 16:13:15.490016 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.489719    2561 scope.go:117] "RemoveContainer" containerID="d9a3d1265ab470b591901171bf00d221e5f99b08a5b386cbcde80353b5ef0739"
Jul 24 16:13:15.490299 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:13:15.490133    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9a3d1265ab470b591901171bf00d221e5f99b08a5b386cbcde80353b5ef0739\": container with ID starting with d9a3d1265ab470b591901171bf00d221e5f99b08a5b386cbcde80353b5ef0739 not found: ID does not exist" containerID="d9a3d1265ab470b591901171bf00d221e5f99b08a5b386cbcde80353b5ef0739"
Jul 24 16:13:15.490299 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.490181    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9a3d1265ab470b591901171bf00d221e5f99b08a5b386cbcde80353b5ef0739"} err="failed to get container status \"d9a3d1265ab470b591901171bf00d221e5f99b08a5b386cbcde80353b5ef0739\": rpc error: code = NotFound desc = could not find container \"d9a3d1265ab470b591901171bf00d221e5f99b08a5b386cbcde80353b5ef0739\": container with ID starting with d9a3d1265ab470b591901171bf00d221e5f99b08a5b386cbcde80353b5ef0739 not found: ID does not exist"
Jul 24 16:13:15.498112 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.498049    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"]
Jul 24 16:13:15.500472 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.500444    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-aedd03-fc8fc6d8-ksvzp"]
Jul 24 16:13:15.510694 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.510642    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"]
Jul 24 16:13:15.512976 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:15.512946    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-aedd03-9c9b4bbf4-gwn74"]
Jul 24 16:13:16.106887 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:16.106851    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ddff3f0-b95e-44c9-a453-7de15ee35ff1" path="/var/lib/kubelet/pods/6ddff3f0-b95e-44c9-a453-7de15ee35ff1/volumes"
Jul 24 16:13:16.107217 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:16.107204    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8315a6fe-d0e4-4780-a999-a535a974f783" path="/var/lib/kubelet/pods/8315a6fe-d0e4-4780-a999-a535a974f783/volumes"
Jul 24 16:13:32.898290 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.898199    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"]
Jul 24 16:13:32.898824 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.898561    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f63aded2-88e6-4e39-82ae-93939f1082ca" containerName="payload-pre-processing"
Jul 24 16:13:32.898824 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.898572    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f63aded2-88e6-4e39-82ae-93939f1082ca" containerName="payload-pre-processing"
Jul 24 16:13:32.898824 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.898582    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ddff3f0-b95e-44c9-a453-7de15ee35ff1" containerName="payload-pre-processing"
Jul 24 16:13:32.898824 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.898587    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ddff3f0-b95e-44c9-a453-7de15ee35ff1" containerName="payload-pre-processing"
Jul 24 16:13:32.898824 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.898597    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e75db08-517e-4e5f-8e62-a2c102db8bbd" containerName="main"
Jul 24 16:13:32.898824 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.898603    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e75db08-517e-4e5f-8e62-a2c102db8bbd" containerName="main"
Jul 24 16:13:32.898824 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.898610    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8315a6fe-d0e4-4780-a999-a535a974f783" containerName="payload-processing"
Jul 24 16:13:32.898824 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.898616    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8315a6fe-d0e4-4780-a999-a535a974f783" containerName="payload-processing"
Jul 24 16:13:32.898824 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.898625    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84209967-3ca5-4bb1-b943-d830c8e055e7" containerName="payload-processing"
Jul 24 16:13:32.898824 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.898630    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="84209967-3ca5-4bb1-b943-d830c8e055e7" containerName="payload-processing"
Jul 24 16:13:32.898824 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.898706    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f63aded2-88e6-4e39-82ae-93939f1082ca" containerName="payload-pre-processing"
Jul 24 16:13:32.898824 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.898712    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ddff3f0-b95e-44c9-a453-7de15ee35ff1" containerName="payload-pre-processing"
Jul 24 16:13:32.898824 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.898719    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="84209967-3ca5-4bb1-b943-d830c8e055e7" containerName="payload-processing"
Jul 24 16:13:32.898824 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.898726    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e75db08-517e-4e5f-8e62-a2c102db8bbd" containerName="main"
Jul 24 16:13:32.898824 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.898732    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8315a6fe-d0e4-4780-a999-a535a974f783" containerName="payload-processing"
Jul 24 16:13:32.902063 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.902045    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"
Jul 24 16:13:32.913678 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.913636    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-57ceec\""
Jul 24 16:13:32.913814 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.913676    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-57ceec-dockercfg-grkjh\""
Jul 24 16:13:32.931510 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.931470    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"]
Jul 24 16:13:32.947002 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.946966    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"]
Jul 24 16:13:32.950430 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.950411    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"
Jul 24 16:13:32.965998 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.965959    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"]
Jul 24 16:13:32.983515 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.983481    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv6fz\" (UniqueName: \"kubernetes.io/projected/a019a21f-68d0-4de3-9579-4c318969649e-kube-api-access-xv6fz\") pod \"payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f\" (UID: \"a019a21f-68d0-4de3-9579-4c318969649e\") " pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"
Jul 24 16:13:32.983721 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:32.983544    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a019a21f-68d0-4de3-9579-4c318969649e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f\" (UID: \"a019a21f-68d0-4de3-9579-4c318969649e\") " pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"
Jul 24 16:13:33.084597 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.084556    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050f6845-2879-4cb2-8eb4-f00de454fbbf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd\" (UID: \"050f6845-2879-4cb2-8eb4-f00de454fbbf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"
Jul 24 16:13:33.084798 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.084730    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv6fz\" (UniqueName: \"kubernetes.io/projected/a019a21f-68d0-4de3-9579-4c318969649e-kube-api-access-xv6fz\") pod \"payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f\" (UID: \"a019a21f-68d0-4de3-9579-4c318969649e\") " pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"
Jul 24 16:13:33.084798 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.084784    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv7kb\" (UniqueName: \"kubernetes.io/projected/050f6845-2879-4cb2-8eb4-f00de454fbbf-kube-api-access-tv7kb\") pod \"payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd\" (UID: \"050f6845-2879-4cb2-8eb4-f00de454fbbf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"
Jul 24 16:13:33.084896 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.084837    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a019a21f-68d0-4de3-9579-4c318969649e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f\" (UID: \"a019a21f-68d0-4de3-9579-4c318969649e\") " pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"
Jul 24 16:13:33.085737 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.085713    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a019a21f-68d0-4de3-9579-4c318969649e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f\" (UID: \"a019a21f-68d0-4de3-9579-4c318969649e\") " pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"
Jul 24 16:13:33.097623 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.097583    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv6fz\" (UniqueName: \"kubernetes.io/projected/a019a21f-68d0-4de3-9579-4c318969649e-kube-api-access-xv6fz\") pod \"payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f\" (UID: \"a019a21f-68d0-4de3-9579-4c318969649e\") " pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"
Jul 24 16:13:33.185369 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.185256    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050f6845-2879-4cb2-8eb4-f00de454fbbf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd\" (UID: \"050f6845-2879-4cb2-8eb4-f00de454fbbf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"
Jul 24 16:13:33.185553 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.185368    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv7kb\" (UniqueName: \"kubernetes.io/projected/050f6845-2879-4cb2-8eb4-f00de454fbbf-kube-api-access-tv7kb\") pod \"payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd\" (UID: \"050f6845-2879-4cb2-8eb4-f00de454fbbf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"
Jul 24 16:13:33.186066 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.186045    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050f6845-2879-4cb2-8eb4-f00de454fbbf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd\" (UID: \"050f6845-2879-4cb2-8eb4-f00de454fbbf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"
Jul 24 16:13:33.195823 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.195795    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv7kb\" (UniqueName: \"kubernetes.io/projected/050f6845-2879-4cb2-8eb4-f00de454fbbf-kube-api-access-tv7kb\") pod \"payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd\" (UID: \"050f6845-2879-4cb2-8eb4-f00de454fbbf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"
Jul 24 16:13:33.211760 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.211722    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"
Jul 24 16:13:33.260316 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.260273    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"
Jul 24 16:13:33.397253 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.397116    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"]
Jul 24 16:13:33.402385 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:13:33.402334    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda019a21f_68d0_4de3_9579_4c318969649e.slice/crio-e917a8e98f7bbd89f17e26c6a7da7726df9078bcad41ca2487990ea50f1e3103 WatchSource:0}: Error finding container e917a8e98f7bbd89f17e26c6a7da7726df9078bcad41ca2487990ea50f1e3103: Status 404 returned error can't find the container with id e917a8e98f7bbd89f17e26c6a7da7726df9078bcad41ca2487990ea50f1e3103
Jul 24 16:13:33.443567 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.443534    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"]
Jul 24 16:13:33.447593 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:13:33.447551    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod050f6845_2879_4cb2_8eb4_f00de454fbbf.slice/crio-a3909157d1001b4dcc1913e83cd4b8e06ae765b1f002d6bb3ea295dcb377702b WatchSource:0}: Error finding container a3909157d1001b4dcc1913e83cd4b8e06ae765b1f002d6bb3ea295dcb377702b: Status 404 returned error can't find the container with id a3909157d1001b4dcc1913e83cd4b8e06ae765b1f002d6bb3ea295dcb377702b
Jul 24 16:13:33.564195 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.564159    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f" event={"ID":"a019a21f-68d0-4de3-9579-4c318969649e","Type":"ContainerStarted","Data":"a7e665a6612c1264f0dc7aee0d0c5a8ad211b593e5f1d6328ad26e21d1a9b5cc"}
Jul 24 16:13:33.564406 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.564237    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f" event={"ID":"a019a21f-68d0-4de3-9579-4c318969649e","Type":"ContainerStarted","Data":"e917a8e98f7bbd89f17e26c6a7da7726df9078bcad41ca2487990ea50f1e3103"}
Jul 24 16:13:33.564483 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.564405    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"
Jul 24 16:13:33.566077 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.566048    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd" event={"ID":"050f6845-2879-4cb2-8eb4-f00de454fbbf","Type":"ContainerStarted","Data":"1d2d70efd5c9628e097a326029168fe5984d4208a6dd823bff1cda8e8998ec91"}
Jul 24 16:13:33.566216 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.566095    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd" event={"ID":"050f6845-2879-4cb2-8eb4-f00de454fbbf","Type":"ContainerStarted","Data":"a3909157d1001b4dcc1913e83cd4b8e06ae765b1f002d6bb3ea295dcb377702b"}
Jul 24 16:13:33.566216 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.566130    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"
Jul 24 16:13:33.610075 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.610009    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f" podStartSLOduration=1.609993577 podStartE2EDuration="1.609993577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:13:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:13:33.609377523 +0000 UTC m=+2242.020088139" watchObservedRunningTime="2026-07-24 16:13:33.609993577 +0000 UTC m=+2242.020704194"
Jul 24 16:13:33.665027 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:33.664964    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd" podStartSLOduration=1.664948004 podStartE2EDuration="1.664948004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:13:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:13:33.664400634 +0000 UTC m=+2242.075111252" watchObservedRunningTime="2026-07-24 16:13:33.664948004 +0000 UTC m=+2242.075658623"
Jul 24 16:13:41.485542 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.485501    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"]
Jul 24 16:13:41.489262 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.489242    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"
Jul 24 16:13:41.492976 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.492950    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-afdeb1\""
Jul 24 16:13:41.504044 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.504010    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"]
Jul 24 16:13:41.521907 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.521871    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"]
Jul 24 16:13:41.525271 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.525249    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"
Jul 24 16:13:41.530074 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.530044    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-afdeb1-dockercfg-lcnfp\""
Jul 24 16:13:41.543869 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.543835    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"]
Jul 24 16:13:41.663093 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.663038    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a36a0f1e-bde9-4165-8756-fcfac4ae9883-plugins-config-volume\") pod \"payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np\" (UID: \"a36a0f1e-bde9-4165-8756-fcfac4ae9883\") " pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"
Jul 24 16:13:41.663093 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.663098    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec4059b8-b098-4d69-8f3f-f13dda27107e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd\" (UID: \"ec4059b8-b098-4d69-8f3f-f13dda27107e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"
Jul 24 16:13:41.663340 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.663138    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd5j4\" (UniqueName: \"kubernetes.io/projected/a36a0f1e-bde9-4165-8756-fcfac4ae9883-kube-api-access-dd5j4\") pod \"payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np\" (UID: \"a36a0f1e-bde9-4165-8756-fcfac4ae9883\") " pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"
Jul 24 16:13:41.663340 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.663166    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfksf\" (UniqueName: \"kubernetes.io/projected/ec4059b8-b098-4d69-8f3f-f13dda27107e-kube-api-access-mfksf\") pod \"payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd\" (UID: \"ec4059b8-b098-4d69-8f3f-f13dda27107e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"
Jul 24 16:13:41.764035 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.763937    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfksf\" (UniqueName: \"kubernetes.io/projected/ec4059b8-b098-4d69-8f3f-f13dda27107e-kube-api-access-mfksf\") pod \"payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd\" (UID: \"ec4059b8-b098-4d69-8f3f-f13dda27107e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"
Jul 24 16:13:41.764035 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.764036    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a36a0f1e-bde9-4165-8756-fcfac4ae9883-plugins-config-volume\") pod \"payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np\" (UID: \"a36a0f1e-bde9-4165-8756-fcfac4ae9883\") " pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"
Jul 24 16:13:41.764230 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.764055    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec4059b8-b098-4d69-8f3f-f13dda27107e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd\" (UID: \"ec4059b8-b098-4d69-8f3f-f13dda27107e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"
Jul 24 16:13:41.764230 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.764078    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd5j4\" (UniqueName: \"kubernetes.io/projected/a36a0f1e-bde9-4165-8756-fcfac4ae9883-kube-api-access-dd5j4\") pod \"payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np\" (UID: \"a36a0f1e-bde9-4165-8756-fcfac4ae9883\") " pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"
Jul 24 16:13:41.764816 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.764797    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a36a0f1e-bde9-4165-8756-fcfac4ae9883-plugins-config-volume\") pod \"payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np\" (UID: \"a36a0f1e-bde9-4165-8756-fcfac4ae9883\") " pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"
Jul 24 16:13:41.764865 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.764825    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec4059b8-b098-4d69-8f3f-f13dda27107e-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd\" (UID: \"ec4059b8-b098-4d69-8f3f-f13dda27107e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"
Jul 24 16:13:41.789855 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.789823    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfksf\" (UniqueName: \"kubernetes.io/projected/ec4059b8-b098-4d69-8f3f-f13dda27107e-kube-api-access-mfksf\") pod \"payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd\" (UID: \"ec4059b8-b098-4d69-8f3f-f13dda27107e\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"
Jul 24 16:13:41.791383 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.791360    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd5j4\" (UniqueName: \"kubernetes.io/projected/a36a0f1e-bde9-4165-8756-fcfac4ae9883-kube-api-access-dd5j4\") pod \"payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np\" (UID: \"a36a0f1e-bde9-4165-8756-fcfac4ae9883\") " pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"
Jul 24 16:13:41.800236 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.800200    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"
Jul 24 16:13:41.836479 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.836438    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"
Jul 24 16:13:41.989943 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:41.989912    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"]
Jul 24 16:13:41.992852 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:13:41.992823    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda36a0f1e_bde9_4165_8756_fcfac4ae9883.slice/crio-5adfffca956b5137da28981023fdbf81525cb9e2c78312b48974733c771c536d WatchSource:0}: Error finding container 5adfffca956b5137da28981023fdbf81525cb9e2c78312b48974733c771c536d: Status 404 returned error can't find the container with id 5adfffca956b5137da28981023fdbf81525cb9e2c78312b48974733c771c536d
Jul 24 16:13:42.009264 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:42.009233    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"]
Jul 24 16:13:42.013581 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:13:42.013152    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec4059b8_b098_4d69_8f3f_f13dda27107e.slice/crio-8ab3c55886ab78300201f26cfd8a63da231ed68d9c72d957a0fc604cb855d332 WatchSource:0}: Error finding container 8ab3c55886ab78300201f26cfd8a63da231ed68d9c72d957a0fc604cb855d332: Status 404 returned error can't find the container with id 8ab3c55886ab78300201f26cfd8a63da231ed68d9c72d957a0fc604cb855d332
Jul 24 16:13:42.614308 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:42.614269    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd" event={"ID":"ec4059b8-b098-4d69-8f3f-f13dda27107e","Type":"ContainerStarted","Data":"2e4cbaf61fd08b97640eb3846f0667cd4be628e80aa1bc2b7bb9fd4f5d7c60bb"}
Jul 24 16:13:42.614816 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:42.614324    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd" event={"ID":"ec4059b8-b098-4d69-8f3f-f13dda27107e","Type":"ContainerStarted","Data":"8ab3c55886ab78300201f26cfd8a63da231ed68d9c72d957a0fc604cb855d332"}
Jul 24 16:13:42.614816 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:42.614369    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"
Jul 24 16:13:42.615838 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:42.615809    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np" event={"ID":"a36a0f1e-bde9-4165-8756-fcfac4ae9883","Type":"ContainerStarted","Data":"a4772b3ed015e325d3979bcae89f3b1ed5da7e33f9bc25a792726fe9d9cfef1f"}
Jul 24 16:13:42.615952 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:42.615866    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np" event={"ID":"a36a0f1e-bde9-4165-8756-fcfac4ae9883","Type":"ContainerStarted","Data":"5adfffca956b5137da28981023fdbf81525cb9e2c78312b48974733c771c536d"}
Jul 24 16:13:42.615952 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:42.615907    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"
Jul 24 16:13:42.635093 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:42.635030    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd" podStartSLOduration=1.635008225 podStartE2EDuration="1.635008225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:13:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:13:42.63360964 +0000 UTC m=+2251.044320269" watchObservedRunningTime="2026-07-24 16:13:42.635008225 +0000 UTC m=+2251.045718846"
Jul 24 16:13:42.657899 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:42.657827    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np" podStartSLOduration=1.657807617 podStartE2EDuration="1.657807617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:13:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:13:42.656758758 +0000 UTC m=+2251.067469376" watchObservedRunningTime="2026-07-24 16:13:42.657807617 +0000 UTC m=+2251.068518214"
Jul 24 16:13:44.572596 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:44.572562    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"
Jul 24 16:13:44.573095 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:44.572638    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"
Jul 24 16:13:53.624143 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:53.624112    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"
Jul 24 16:13:53.624577 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:13:53.624186    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"
Jul 24 16:14:08.944702 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:08.944642    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"]
Jul 24 16:14:08.945329 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:08.944925    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f" podUID="a019a21f-68d0-4de3-9579-4c318969649e" containerName="payload-processing" containerID="cri-o://a7e665a6612c1264f0dc7aee0d0c5a8ad211b593e5f1d6328ad26e21d1a9b5cc" gracePeriod=30
Jul 24 16:14:08.953085 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:08.953051    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"]
Jul 24 16:14:08.953718 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:08.953318    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd" podUID="050f6845-2879-4cb2-8eb4-f00de454fbbf" containerName="payload-pre-processing" containerID="cri-o://1d2d70efd5c9628e097a326029168fe5984d4208a6dd823bff1cda8e8998ec91" gracePeriod=30
Jul 24 16:14:09.245855 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.245825    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"
Jul 24 16:14:09.249130 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.249104    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"
Jul 24 16:14:09.301215 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.301170    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a019a21f-68d0-4de3-9579-4c318969649e-plugins-config-volume\") pod \"a019a21f-68d0-4de3-9579-4c318969649e\" (UID: \"a019a21f-68d0-4de3-9579-4c318969649e\") "
Jul 24 16:14:09.301215 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.301206    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050f6845-2879-4cb2-8eb4-f00de454fbbf-plugins-config-volume\") pod \"050f6845-2879-4cb2-8eb4-f00de454fbbf\" (UID: \"050f6845-2879-4cb2-8eb4-f00de454fbbf\") "
Jul 24 16:14:09.301495 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.301275    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xv6fz\" (UniqueName: \"kubernetes.io/projected/a019a21f-68d0-4de3-9579-4c318969649e-kube-api-access-xv6fz\") pod \"a019a21f-68d0-4de3-9579-4c318969649e\" (UID: \"a019a21f-68d0-4de3-9579-4c318969649e\") "
Jul 24 16:14:09.301495 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.301302    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tv7kb\" (UniqueName: \"kubernetes.io/projected/050f6845-2879-4cb2-8eb4-f00de454fbbf-kube-api-access-tv7kb\") pod \"050f6845-2879-4cb2-8eb4-f00de454fbbf\" (UID: \"050f6845-2879-4cb2-8eb4-f00de454fbbf\") "
Jul 24 16:14:09.301650 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.301622    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a019a21f-68d0-4de3-9579-4c318969649e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a019a21f-68d0-4de3-9579-4c318969649e" (UID: "a019a21f-68d0-4de3-9579-4c318969649e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:14:09.301752 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.301700    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/050f6845-2879-4cb2-8eb4-f00de454fbbf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "050f6845-2879-4cb2-8eb4-f00de454fbbf" (UID: "050f6845-2879-4cb2-8eb4-f00de454fbbf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:14:09.303622 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.303588    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a019a21f-68d0-4de3-9579-4c318969649e-kube-api-access-xv6fz" (OuterVolumeSpecName: "kube-api-access-xv6fz") pod "a019a21f-68d0-4de3-9579-4c318969649e" (UID: "a019a21f-68d0-4de3-9579-4c318969649e"). InnerVolumeSpecName "kube-api-access-xv6fz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:14:09.303760 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.303700    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/050f6845-2879-4cb2-8eb4-f00de454fbbf-kube-api-access-tv7kb" (OuterVolumeSpecName: "kube-api-access-tv7kb") pod "050f6845-2879-4cb2-8eb4-f00de454fbbf" (UID: "050f6845-2879-4cb2-8eb4-f00de454fbbf"). InnerVolumeSpecName "kube-api-access-tv7kb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:14:09.402588 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.402539    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xv6fz\" (UniqueName: \"kubernetes.io/projected/a019a21f-68d0-4de3-9579-4c318969649e-kube-api-access-xv6fz\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:14:09.402588 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.402578    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tv7kb\" (UniqueName: \"kubernetes.io/projected/050f6845-2879-4cb2-8eb4-f00de454fbbf-kube-api-access-tv7kb\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:14:09.402588 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.402590    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a019a21f-68d0-4de3-9579-4c318969649e-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:14:09.402588 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.402598    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050f6845-2879-4cb2-8eb4-f00de454fbbf-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:14:09.748308 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.748268    2561 generic.go:356] "Generic (PLEG): container finished" podID="050f6845-2879-4cb2-8eb4-f00de454fbbf" containerID="1d2d70efd5c9628e097a326029168fe5984d4208a6dd823bff1cda8e8998ec91" exitCode=0
Jul 24 16:14:09.748503 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.748320    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"
Jul 24 16:14:09.748503 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.748352    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd" event={"ID":"050f6845-2879-4cb2-8eb4-f00de454fbbf","Type":"ContainerDied","Data":"1d2d70efd5c9628e097a326029168fe5984d4208a6dd823bff1cda8e8998ec91"}
Jul 24 16:14:09.748503 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.748413    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd" event={"ID":"050f6845-2879-4cb2-8eb4-f00de454fbbf","Type":"ContainerDied","Data":"a3909157d1001b4dcc1913e83cd4b8e06ae765b1f002d6bb3ea295dcb377702b"}
Jul 24 16:14:09.748503 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.748442    2561 scope.go:117] "RemoveContainer" containerID="1d2d70efd5c9628e097a326029168fe5984d4208a6dd823bff1cda8e8998ec91"
Jul 24 16:14:09.749803 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.749772    2561 generic.go:356] "Generic (PLEG): container finished" podID="a019a21f-68d0-4de3-9579-4c318969649e" containerID="a7e665a6612c1264f0dc7aee0d0c5a8ad211b593e5f1d6328ad26e21d1a9b5cc" exitCode=0
Jul 24 16:14:09.749919 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.749834    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"
Jul 24 16:14:09.749919 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.749836    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f" event={"ID":"a019a21f-68d0-4de3-9579-4c318969649e","Type":"ContainerDied","Data":"a7e665a6612c1264f0dc7aee0d0c5a8ad211b593e5f1d6328ad26e21d1a9b5cc"}
Jul 24 16:14:09.749919 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.749870    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f" event={"ID":"a019a21f-68d0-4de3-9579-4c318969649e","Type":"ContainerDied","Data":"e917a8e98f7bbd89f17e26c6a7da7726df9078bcad41ca2487990ea50f1e3103"}
Jul 24 16:14:09.758870 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.758843    2561 scope.go:117] "RemoveContainer" containerID="1d2d70efd5c9628e097a326029168fe5984d4208a6dd823bff1cda8e8998ec91"
Jul 24 16:14:09.759221 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:14:09.759191    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d2d70efd5c9628e097a326029168fe5984d4208a6dd823bff1cda8e8998ec91\": container with ID starting with 1d2d70efd5c9628e097a326029168fe5984d4208a6dd823bff1cda8e8998ec91 not found: ID does not exist" containerID="1d2d70efd5c9628e097a326029168fe5984d4208a6dd823bff1cda8e8998ec91"
Jul 24 16:14:09.759286 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.759237    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d2d70efd5c9628e097a326029168fe5984d4208a6dd823bff1cda8e8998ec91"} err="failed to get container status \"1d2d70efd5c9628e097a326029168fe5984d4208a6dd823bff1cda8e8998ec91\": rpc error: code = NotFound desc = could not find container \"1d2d70efd5c9628e097a326029168fe5984d4208a6dd823bff1cda8e8998ec91\": container with ID starting with 1d2d70efd5c9628e097a326029168fe5984d4208a6dd823bff1cda8e8998ec91 not found: ID does not exist"
Jul 24 16:14:09.759286 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.759260    2561 scope.go:117] "RemoveContainer" containerID="a7e665a6612c1264f0dc7aee0d0c5a8ad211b593e5f1d6328ad26e21d1a9b5cc"
Jul 24 16:14:09.768210 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.768190    2561 scope.go:117] "RemoveContainer" containerID="a7e665a6612c1264f0dc7aee0d0c5a8ad211b593e5f1d6328ad26e21d1a9b5cc"
Jul 24 16:14:09.768525 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:14:09.768506    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7e665a6612c1264f0dc7aee0d0c5a8ad211b593e5f1d6328ad26e21d1a9b5cc\": container with ID starting with a7e665a6612c1264f0dc7aee0d0c5a8ad211b593e5f1d6328ad26e21d1a9b5cc not found: ID does not exist" containerID="a7e665a6612c1264f0dc7aee0d0c5a8ad211b593e5f1d6328ad26e21d1a9b5cc"
Jul 24 16:14:09.768579 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.768534    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7e665a6612c1264f0dc7aee0d0c5a8ad211b593e5f1d6328ad26e21d1a9b5cc"} err="failed to get container status \"a7e665a6612c1264f0dc7aee0d0c5a8ad211b593e5f1d6328ad26e21d1a9b5cc\": rpc error: code = NotFound desc = could not find container \"a7e665a6612c1264f0dc7aee0d0c5a8ad211b593e5f1d6328ad26e21d1a9b5cc\": container with ID starting with a7e665a6612c1264f0dc7aee0d0c5a8ad211b593e5f1d6328ad26e21d1a9b5cc not found: ID does not exist"
Jul 24 16:14:09.774929 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.774899    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"]
Jul 24 16:14:09.778534 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.778506    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-57ceec-5fbf764d8-g9wdd"]
Jul 24 16:14:09.789297 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.789257    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"]
Jul 24 16:14:09.793797 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:09.793757    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-57ceec-84948c77cb-mrw9f"]
Jul 24 16:14:10.106242 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:10.106146    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="050f6845-2879-4cb2-8eb4-f00de454fbbf" path="/var/lib/kubelet/pods/050f6845-2879-4cb2-8eb4-f00de454fbbf/volumes"
Jul 24 16:14:10.106762 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:10.106713    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a019a21f-68d0-4de3-9579-4c318969649e" path="/var/lib/kubelet/pods/a019a21f-68d0-4de3-9579-4c318969649e/volumes"
Jul 24 16:14:36.244449 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.244409    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"]
Jul 24 16:14:36.245117 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.244674    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np" podUID="a36a0f1e-bde9-4165-8756-fcfac4ae9883" containerName="payload-processing" containerID="cri-o://a4772b3ed015e325d3979bcae89f3b1ed5da7e33f9bc25a792726fe9d9cfef1f" gracePeriod=30
Jul 24 16:14:36.255540 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.255428    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"]
Jul 24 16:14:36.255807 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.255775    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd" podUID="ec4059b8-b098-4d69-8f3f-f13dda27107e" containerName="payload-pre-processing" containerID="cri-o://2e4cbaf61fd08b97640eb3846f0667cd4be628e80aa1bc2b7bb9fd4f5d7c60bb" gracePeriod=30
Jul 24 16:14:36.560571 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.560545    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"
Jul 24 16:14:36.563909 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.563880    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"
Jul 24 16:14:36.650340 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.650291    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec4059b8-b098-4d69-8f3f-f13dda27107e-plugins-config-volume\") pod \"ec4059b8-b098-4d69-8f3f-f13dda27107e\" (UID: \"ec4059b8-b098-4d69-8f3f-f13dda27107e\") "
Jul 24 16:14:36.650539 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.650373    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a36a0f1e-bde9-4165-8756-fcfac4ae9883-plugins-config-volume\") pod \"a36a0f1e-bde9-4165-8756-fcfac4ae9883\" (UID: \"a36a0f1e-bde9-4165-8756-fcfac4ae9883\") "
Jul 24 16:14:36.650539 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.650437    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd5j4\" (UniqueName: \"kubernetes.io/projected/a36a0f1e-bde9-4165-8756-fcfac4ae9883-kube-api-access-dd5j4\") pod \"a36a0f1e-bde9-4165-8756-fcfac4ae9883\" (UID: \"a36a0f1e-bde9-4165-8756-fcfac4ae9883\") "
Jul 24 16:14:36.650539 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.650460    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfksf\" (UniqueName: \"kubernetes.io/projected/ec4059b8-b098-4d69-8f3f-f13dda27107e-kube-api-access-mfksf\") pod \"ec4059b8-b098-4d69-8f3f-f13dda27107e\" (UID: \"ec4059b8-b098-4d69-8f3f-f13dda27107e\") "
Jul 24 16:14:36.650835 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.650805    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec4059b8-b098-4d69-8f3f-f13dda27107e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ec4059b8-b098-4d69-8f3f-f13dda27107e" (UID: "ec4059b8-b098-4d69-8f3f-f13dda27107e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:14:36.650903 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.650807    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a36a0f1e-bde9-4165-8756-fcfac4ae9883-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a36a0f1e-bde9-4165-8756-fcfac4ae9883" (UID: "a36a0f1e-bde9-4165-8756-fcfac4ae9883"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:14:36.653001 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.652966    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a36a0f1e-bde9-4165-8756-fcfac4ae9883-kube-api-access-dd5j4" (OuterVolumeSpecName: "kube-api-access-dd5j4") pod "a36a0f1e-bde9-4165-8756-fcfac4ae9883" (UID: "a36a0f1e-bde9-4165-8756-fcfac4ae9883"). InnerVolumeSpecName "kube-api-access-dd5j4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:14:36.653001 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.652978    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec4059b8-b098-4d69-8f3f-f13dda27107e-kube-api-access-mfksf" (OuterVolumeSpecName: "kube-api-access-mfksf") pod "ec4059b8-b098-4d69-8f3f-f13dda27107e" (UID: "ec4059b8-b098-4d69-8f3f-f13dda27107e"). InnerVolumeSpecName "kube-api-access-mfksf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:14:36.751908 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.751867    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd5j4\" (UniqueName: \"kubernetes.io/projected/a36a0f1e-bde9-4165-8756-fcfac4ae9883-kube-api-access-dd5j4\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:14:36.751908 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.751901    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfksf\" (UniqueName: \"kubernetes.io/projected/ec4059b8-b098-4d69-8f3f-f13dda27107e-kube-api-access-mfksf\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:14:36.751908 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.751910    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec4059b8-b098-4d69-8f3f-f13dda27107e-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:14:36.751908 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.751921    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a36a0f1e-bde9-4165-8756-fcfac4ae9883-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:14:36.888153 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.888047    2561 generic.go:356] "Generic (PLEG): container finished" podID="a36a0f1e-bde9-4165-8756-fcfac4ae9883" containerID="a4772b3ed015e325d3979bcae89f3b1ed5da7e33f9bc25a792726fe9d9cfef1f" exitCode=0
Jul 24 16:14:36.888153 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.888104    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"
Jul 24 16:14:36.888153 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.888129    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np" event={"ID":"a36a0f1e-bde9-4165-8756-fcfac4ae9883","Type":"ContainerDied","Data":"a4772b3ed015e325d3979bcae89f3b1ed5da7e33f9bc25a792726fe9d9cfef1f"}
Jul 24 16:14:36.888448 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.888181    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np" event={"ID":"a36a0f1e-bde9-4165-8756-fcfac4ae9883","Type":"ContainerDied","Data":"5adfffca956b5137da28981023fdbf81525cb9e2c78312b48974733c771c536d"}
Jul 24 16:14:36.888448 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.888204    2561 scope.go:117] "RemoveContainer" containerID="a4772b3ed015e325d3979bcae89f3b1ed5da7e33f9bc25a792726fe9d9cfef1f"
Jul 24 16:14:36.889697 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.889673    2561 generic.go:356] "Generic (PLEG): container finished" podID="ec4059b8-b098-4d69-8f3f-f13dda27107e" containerID="2e4cbaf61fd08b97640eb3846f0667cd4be628e80aa1bc2b7bb9fd4f5d7c60bb" exitCode=0
Jul 24 16:14:36.889821 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.889709    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd" event={"ID":"ec4059b8-b098-4d69-8f3f-f13dda27107e","Type":"ContainerDied","Data":"2e4cbaf61fd08b97640eb3846f0667cd4be628e80aa1bc2b7bb9fd4f5d7c60bb"}
Jul 24 16:14:36.889821 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.889774    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd" event={"ID":"ec4059b8-b098-4d69-8f3f-f13dda27107e","Type":"ContainerDied","Data":"8ab3c55886ab78300201f26cfd8a63da231ed68d9c72d957a0fc604cb855d332"}
Jul 24 16:14:36.889821 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.889727    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"
Jul 24 16:14:36.898019 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.897880    2561 scope.go:117] "RemoveContainer" containerID="a4772b3ed015e325d3979bcae89f3b1ed5da7e33f9bc25a792726fe9d9cfef1f"
Jul 24 16:14:36.898344 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:14:36.898311    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4772b3ed015e325d3979bcae89f3b1ed5da7e33f9bc25a792726fe9d9cfef1f\": container with ID starting with a4772b3ed015e325d3979bcae89f3b1ed5da7e33f9bc25a792726fe9d9cfef1f not found: ID does not exist" containerID="a4772b3ed015e325d3979bcae89f3b1ed5da7e33f9bc25a792726fe9d9cfef1f"
Jul 24 16:14:36.898466 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.898357    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4772b3ed015e325d3979bcae89f3b1ed5da7e33f9bc25a792726fe9d9cfef1f"} err="failed to get container status \"a4772b3ed015e325d3979bcae89f3b1ed5da7e33f9bc25a792726fe9d9cfef1f\": rpc error: code = NotFound desc = could not find container \"a4772b3ed015e325d3979bcae89f3b1ed5da7e33f9bc25a792726fe9d9cfef1f\": container with ID starting with a4772b3ed015e325d3979bcae89f3b1ed5da7e33f9bc25a792726fe9d9cfef1f not found: ID does not exist"
Jul 24 16:14:36.898466 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.898382    2561 scope.go:117] "RemoveContainer" containerID="2e4cbaf61fd08b97640eb3846f0667cd4be628e80aa1bc2b7bb9fd4f5d7c60bb"
Jul 24 16:14:36.908067 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.908044    2561 scope.go:117] "RemoveContainer" containerID="2e4cbaf61fd08b97640eb3846f0667cd4be628e80aa1bc2b7bb9fd4f5d7c60bb"
Jul 24 16:14:36.908386 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:14:36.908361    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e4cbaf61fd08b97640eb3846f0667cd4be628e80aa1bc2b7bb9fd4f5d7c60bb\": container with ID starting with 2e4cbaf61fd08b97640eb3846f0667cd4be628e80aa1bc2b7bb9fd4f5d7c60bb not found: ID does not exist" containerID="2e4cbaf61fd08b97640eb3846f0667cd4be628e80aa1bc2b7bb9fd4f5d7c60bb"
Jul 24 16:14:36.908457 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.908395    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e4cbaf61fd08b97640eb3846f0667cd4be628e80aa1bc2b7bb9fd4f5d7c60bb"} err="failed to get container status \"2e4cbaf61fd08b97640eb3846f0667cd4be628e80aa1bc2b7bb9fd4f5d7c60bb\": rpc error: code = NotFound desc = could not find container \"2e4cbaf61fd08b97640eb3846f0667cd4be628e80aa1bc2b7bb9fd4f5d7c60bb\": container with ID starting with 2e4cbaf61fd08b97640eb3846f0667cd4be628e80aa1bc2b7bb9fd4f5d7c60bb not found: ID does not exist"
Jul 24 16:14:36.915908 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.915878    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"]
Jul 24 16:14:36.922727 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.922697    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-afdeb1-5cbb9bbbfb-v7zqd"]
Jul 24 16:14:36.933526 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.933498    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"]
Jul 24 16:14:36.941734 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:36.941171    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-afdeb1-74cdd8469b-cw9np"]
Jul 24 16:14:38.106472 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:38.106435    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a36a0f1e-bde9-4165-8756-fcfac4ae9883" path="/var/lib/kubelet/pods/a36a0f1e-bde9-4165-8756-fcfac4ae9883/volumes"
Jul 24 16:14:38.106909 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:38.106812    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec4059b8-b098-4d69-8f3f-f13dda27107e" path="/var/lib/kubelet/pods/ec4059b8-b098-4d69-8f3f-f13dda27107e/volumes"
Jul 24 16:14:52.630169 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.630127    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"]
Jul 24 16:14:52.630654 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.630481    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a019a21f-68d0-4de3-9579-4c318969649e" containerName="payload-processing"
Jul 24 16:14:52.630654 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.630492    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a019a21f-68d0-4de3-9579-4c318969649e" containerName="payload-processing"
Jul 24 16:14:52.630654 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.630509    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="050f6845-2879-4cb2-8eb4-f00de454fbbf" containerName="payload-pre-processing"
Jul 24 16:14:52.630654 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.630516    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="050f6845-2879-4cb2-8eb4-f00de454fbbf" containerName="payload-pre-processing"
Jul 24 16:14:52.630654 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.630524    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec4059b8-b098-4d69-8f3f-f13dda27107e" containerName="payload-pre-processing"
Jul 24 16:14:52.630654 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.630530    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec4059b8-b098-4d69-8f3f-f13dda27107e" containerName="payload-pre-processing"
Jul 24 16:14:52.630654 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.630541    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a36a0f1e-bde9-4165-8756-fcfac4ae9883" containerName="payload-processing"
Jul 24 16:14:52.630654 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.630546    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a36a0f1e-bde9-4165-8756-fcfac4ae9883" containerName="payload-processing"
Jul 24 16:14:52.630654 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.630597    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a36a0f1e-bde9-4165-8756-fcfac4ae9883" containerName="payload-processing"
Jul 24 16:14:52.630654 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.630607    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a019a21f-68d0-4de3-9579-4c318969649e" containerName="payload-processing"
Jul 24 16:14:52.630654 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.630614    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="050f6845-2879-4cb2-8eb4-f00de454fbbf" containerName="payload-pre-processing"
Jul 24 16:14:52.630654 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.630621    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec4059b8-b098-4d69-8f3f-f13dda27107e" containerName="payload-pre-processing"
Jul 24 16:14:52.633643 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.633620    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"
Jul 24 16:14:52.636778 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.636748    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-fafc28\""
Jul 24 16:14:52.645301 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.645267    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"]
Jul 24 16:14:52.679369 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.679329    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"]
Jul 24 16:14:52.682743 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.682716    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"
Jul 24 16:14:52.685042 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.685009    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-fafc28-dockercfg-xsmgj\""
Jul 24 16:14:52.693005 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.692976    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"]
Jul 24 16:14:52.693333 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.693310    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtwjl\" (UniqueName: \"kubernetes.io/projected/a89ac429-adc7-43ad-a61a-77f2674a73d1-kube-api-access-gtwjl\") pod \"payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk\" (UID: \"a89ac429-adc7-43ad-a61a-77f2674a73d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"
Jul 24 16:14:52.693428 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.693373    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a89ac429-adc7-43ad-a61a-77f2674a73d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk\" (UID: \"a89ac429-adc7-43ad-a61a-77f2674a73d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"
Jul 24 16:14:52.794144 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.794100    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94e6b954-4694-4b82-933d-54651f0456c9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp\" (UID: \"94e6b954-4694-4b82-933d-54651f0456c9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"
Jul 24 16:14:52.794309 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.794150    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a89ac429-adc7-43ad-a61a-77f2674a73d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk\" (UID: \"a89ac429-adc7-43ad-a61a-77f2674a73d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"
Jul 24 16:14:52.794376 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.794303    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swbsn\" (UniqueName: \"kubernetes.io/projected/94e6b954-4694-4b82-933d-54651f0456c9-kube-api-access-swbsn\") pod \"payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp\" (UID: \"94e6b954-4694-4b82-933d-54651f0456c9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"
Jul 24 16:14:52.794467 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.794444    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtwjl\" (UniqueName: \"kubernetes.io/projected/a89ac429-adc7-43ad-a61a-77f2674a73d1-kube-api-access-gtwjl\") pod \"payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk\" (UID: \"a89ac429-adc7-43ad-a61a-77f2674a73d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"
Jul 24 16:14:52.794980 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.794961    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a89ac429-adc7-43ad-a61a-77f2674a73d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk\" (UID: \"a89ac429-adc7-43ad-a61a-77f2674a73d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"
Jul 24 16:14:52.804735 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.804703    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtwjl\" (UniqueName: \"kubernetes.io/projected/a89ac429-adc7-43ad-a61a-77f2674a73d1-kube-api-access-gtwjl\") pod \"payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk\" (UID: \"a89ac429-adc7-43ad-a61a-77f2674a73d1\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"
Jul 24 16:14:52.895959 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.895850    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swbsn\" (UniqueName: \"kubernetes.io/projected/94e6b954-4694-4b82-933d-54651f0456c9-kube-api-access-swbsn\") pod \"payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp\" (UID: \"94e6b954-4694-4b82-933d-54651f0456c9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"
Jul 24 16:14:52.896155 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.895977    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94e6b954-4694-4b82-933d-54651f0456c9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp\" (UID: \"94e6b954-4694-4b82-933d-54651f0456c9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"
Jul 24 16:14:52.896716 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.896686    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94e6b954-4694-4b82-933d-54651f0456c9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp\" (UID: \"94e6b954-4694-4b82-933d-54651f0456c9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"
Jul 24 16:14:52.904644 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.904609    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swbsn\" (UniqueName: \"kubernetes.io/projected/94e6b954-4694-4b82-933d-54651f0456c9-kube-api-access-swbsn\") pod \"payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp\" (UID: \"94e6b954-4694-4b82-933d-54651f0456c9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"
Jul 24 16:14:52.948284 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.948239    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"
Jul 24 16:14:52.994794 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:52.994754    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"
Jul 24 16:14:53.107178 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:53.107145    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"]
Jul 24 16:14:53.110938 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:14:53.110897    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda89ac429_adc7_43ad_a61a_77f2674a73d1.slice/crio-90cb9927c6918f165c96ae78a36e967c6bf5cee9f3f0b39f9ad248db23790fa0 WatchSource:0}: Error finding container 90cb9927c6918f165c96ae78a36e967c6bf5cee9f3f0b39f9ad248db23790fa0: Status 404 returned error can't find the container with id 90cb9927c6918f165c96ae78a36e967c6bf5cee9f3f0b39f9ad248db23790fa0
Jul 24 16:14:53.176008 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:53.175974    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"]
Jul 24 16:14:53.178959 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:14:53.178913    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94e6b954_4694_4b82_933d_54651f0456c9.slice/crio-d18f59761455ef59d0c0adf509e932e9f8ce63a9453f914debcb824e0872fd4f WatchSource:0}: Error finding container d18f59761455ef59d0c0adf509e932e9f8ce63a9453f914debcb824e0872fd4f: Status 404 returned error can't find the container with id d18f59761455ef59d0c0adf509e932e9f8ce63a9453f914debcb824e0872fd4f
Jul 24 16:14:53.979257 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:53.979215    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk" event={"ID":"a89ac429-adc7-43ad-a61a-77f2674a73d1","Type":"ContainerStarted","Data":"435344ffc1cf2e5a5a659fbb7917c27661c805b4e3ad964e7fbbfed37cf79688"}
Jul 24 16:14:53.979758 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:53.979295    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"
Jul 24 16:14:53.979758 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:53.979324    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk" event={"ID":"a89ac429-adc7-43ad-a61a-77f2674a73d1","Type":"ContainerStarted","Data":"90cb9927c6918f165c96ae78a36e967c6bf5cee9f3f0b39f9ad248db23790fa0"}
Jul 24 16:14:53.981027 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:53.981001    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp" event={"ID":"94e6b954-4694-4b82-933d-54651f0456c9","Type":"ContainerStarted","Data":"77c0326e81dc5c8e0dfd8e8fc76d96c29423189f894a301ec6d08d4048547320"}
Jul 24 16:14:53.981150 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:53.981043    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp" event={"ID":"94e6b954-4694-4b82-933d-54651f0456c9","Type":"ContainerStarted","Data":"d18f59761455ef59d0c0adf509e932e9f8ce63a9453f914debcb824e0872fd4f"}
Jul 24 16:14:53.981150 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:53.981066    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"
Jul 24 16:14:53.996171 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:53.996113    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk" podStartSLOduration=1.996091152 podStartE2EDuration="1.996091152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:14:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:14:53.994957287 +0000 UTC m=+2322.405667908" watchObservedRunningTime="2026-07-24 16:14:53.996091152 +0000 UTC m=+2322.406801769"
Jul 24 16:14:54.011695 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:14:54.011604    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp" podStartSLOduration=2.01158562 podStartE2EDuration="2.01158562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:14:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:14:54.010229931 +0000 UTC m=+2322.420940549" watchObservedRunningTime="2026-07-24 16:14:54.01158562 +0000 UTC m=+2322.422296236"
Jul 24 16:15:00.993522 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:00.993476    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"]
Jul 24 16:15:00.997317 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:00.997292    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"
Jul 24 16:15:01.000675 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.000634    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-c6c91e-dockercfg-kjpdh\""
Jul 24 16:15:01.000840 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.000760    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-c6c91e\""
Jul 24 16:15:01.008288 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.008254    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"]
Jul 24 16:15:01.030832 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.030786    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"]
Jul 24 16:15:01.034313 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.034282    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"
Jul 24 16:15:01.043678 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.043630    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"]
Jul 24 16:15:01.068944 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.068893    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1025b2-acfb-48e0-bdce-c80127a09ec1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b\" (UID: \"eb1025b2-acfb-48e0-bdce-c80127a09ec1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"
Jul 24 16:15:01.069175 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.068978    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7ppk\" (UniqueName: \"kubernetes.io/projected/eb1025b2-acfb-48e0-bdce-c80127a09ec1-kube-api-access-b7ppk\") pod \"payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b\" (UID: \"eb1025b2-acfb-48e0-bdce-c80127a09ec1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"
Jul 24 16:15:01.170264 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.170220    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f9bee4d-8b84-4ddd-8740-1db054920dbe-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66\" (UID: \"3f9bee4d-8b84-4ddd-8740-1db054920dbe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"
Jul 24 16:15:01.170491 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.170278    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1025b2-acfb-48e0-bdce-c80127a09ec1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b\" (UID: \"eb1025b2-acfb-48e0-bdce-c80127a09ec1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"
Jul 24 16:15:01.170491 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.170321    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7ppk\" (UniqueName: \"kubernetes.io/projected/eb1025b2-acfb-48e0-bdce-c80127a09ec1-kube-api-access-b7ppk\") pod \"payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b\" (UID: \"eb1025b2-acfb-48e0-bdce-c80127a09ec1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"
Jul 24 16:15:01.170491 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.170354    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bswhb\" (UniqueName: \"kubernetes.io/projected/3f9bee4d-8b84-4ddd-8740-1db054920dbe-kube-api-access-bswhb\") pod \"payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66\" (UID: \"3f9bee4d-8b84-4ddd-8740-1db054920dbe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"
Jul 24 16:15:01.171145 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.171118    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1025b2-acfb-48e0-bdce-c80127a09ec1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b\" (UID: \"eb1025b2-acfb-48e0-bdce-c80127a09ec1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"
Jul 24 16:15:01.178924 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.178890    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7ppk\" (UniqueName: \"kubernetes.io/projected/eb1025b2-acfb-48e0-bdce-c80127a09ec1-kube-api-access-b7ppk\") pod \"payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b\" (UID: \"eb1025b2-acfb-48e0-bdce-c80127a09ec1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"
Jul 24 16:15:01.271419 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.271327    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bswhb\" (UniqueName: \"kubernetes.io/projected/3f9bee4d-8b84-4ddd-8740-1db054920dbe-kube-api-access-bswhb\") pod \"payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66\" (UID: \"3f9bee4d-8b84-4ddd-8740-1db054920dbe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"
Jul 24 16:15:01.271419 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.271386    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f9bee4d-8b84-4ddd-8740-1db054920dbe-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66\" (UID: \"3f9bee4d-8b84-4ddd-8740-1db054920dbe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"
Jul 24 16:15:01.272226 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.272201    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f9bee4d-8b84-4ddd-8740-1db054920dbe-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66\" (UID: \"3f9bee4d-8b84-4ddd-8740-1db054920dbe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"
Jul 24 16:15:01.279821 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.279792    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bswhb\" (UniqueName: \"kubernetes.io/projected/3f9bee4d-8b84-4ddd-8740-1db054920dbe-kube-api-access-bswhb\") pod \"payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66\" (UID: \"3f9bee4d-8b84-4ddd-8740-1db054920dbe\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"
Jul 24 16:15:01.309801 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.309758    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"
Jul 24 16:15:01.346712 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.346623    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"
Jul 24 16:15:01.488491 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.488445    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"]
Jul 24 16:15:01.490975 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:15:01.490946    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb1025b2_acfb_48e0_bdce_c80127a09ec1.slice/crio-658b43759e57f5c9f85740d15d3965d631a37bedf490972ee4923a2f8b56f70f WatchSource:0}: Error finding container 658b43759e57f5c9f85740d15d3965d631a37bedf490972ee4923a2f8b56f70f: Status 404 returned error can't find the container with id 658b43759e57f5c9f85740d15d3965d631a37bedf490972ee4923a2f8b56f70f
Jul 24 16:15:01.518498 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:01.518465    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"]
Jul 24 16:15:01.523929 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:15:01.523891    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f9bee4d_8b84_4ddd_8740_1db054920dbe.slice/crio-5b59ed1defb5fb960e98d653edd2b5522dcfe006e63479f7cb5ef0d3aa2d9c07 WatchSource:0}: Error finding container 5b59ed1defb5fb960e98d653edd2b5522dcfe006e63479f7cb5ef0d3aa2d9c07: Status 404 returned error can't find the container with id 5b59ed1defb5fb960e98d653edd2b5522dcfe006e63479f7cb5ef0d3aa2d9c07
Jul 24 16:15:02.023004 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:02.022966    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b" event={"ID":"eb1025b2-acfb-48e0-bdce-c80127a09ec1","Type":"ContainerStarted","Data":"d52af8ea55dce8d7bc9e73e7dd8965695ee187e9682d355f6124d2d794f07d4a"}
Jul 24 16:15:02.023539 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:02.023067    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"
Jul 24 16:15:02.023539 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:02.023086    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b" event={"ID":"eb1025b2-acfb-48e0-bdce-c80127a09ec1","Type":"ContainerStarted","Data":"658b43759e57f5c9f85740d15d3965d631a37bedf490972ee4923a2f8b56f70f"}
Jul 24 16:15:02.024597 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:02.024571    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66" event={"ID":"3f9bee4d-8b84-4ddd-8740-1db054920dbe","Type":"ContainerStarted","Data":"d31690e4e4930e27c4851170f8e895b7699f2786926b90a9f3a7f3485a3c3dbe"}
Jul 24 16:15:02.024760 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:02.024614    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66" event={"ID":"3f9bee4d-8b84-4ddd-8740-1db054920dbe","Type":"ContainerStarted","Data":"5b59ed1defb5fb960e98d653edd2b5522dcfe006e63479f7cb5ef0d3aa2d9c07"}
Jul 24 16:15:02.024760 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:02.024683    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"
Jul 24 16:15:02.040323 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:02.040262    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b" podStartSLOduration=2.040237659 podStartE2EDuration="2.040237659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:15:02.040030536 +0000 UTC m=+2330.450741155" watchObservedRunningTime="2026-07-24 16:15:02.040237659 +0000 UTC m=+2330.450948276"
Jul 24 16:15:02.068408 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:02.068308    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66" podStartSLOduration=1.068284513 podStartE2EDuration="1.068284513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:15:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:15:02.068021452 +0000 UTC m=+2330.478732106" watchObservedRunningTime="2026-07-24 16:15:02.068284513 +0000 UTC m=+2330.478995133"
Jul 24 16:15:04.987917 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:04.987826    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"
Jul 24 16:15:04.987917 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:04.987897    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"
Jul 24 16:15:13.031652 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:13.031618    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"
Jul 24 16:15:13.032168 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:13.031709    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"
Jul 24 16:15:53.301095 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.301057    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"]
Jul 24 16:15:53.304885 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.304866    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.308882 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.308854    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c6c91e\"/\"default-dockercfg-nr5hm\""
Jul 24 16:15:53.309766 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.309736    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c6c91e\"/\"auth-test-model-c6c91e-kserve-self-signed-certs\""
Jul 24 16:15:53.309986 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.309736    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c6c91e\"/\"kube-root-ca.crt\""
Jul 24 16:15:53.309986 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.309736    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c6c91e\"/\"openshift-service-ca.crt\""
Jul 24 16:15:53.319541 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.319510    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"]
Jul 24 16:15:53.441839 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.441795    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-home\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.441839 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.441835    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-tls-certs\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.442117 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.441882    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-dshm\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.442117 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.442021    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpwv2\" (UniqueName: \"kubernetes.io/projected/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-kube-api-access-wpwv2\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.442117 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.442061    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-tmp-dir\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.442242 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.442127    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-model-cache\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.543007 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.542949    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-dshm\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.543225 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.543041    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpwv2\" (UniqueName: \"kubernetes.io/projected/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-kube-api-access-wpwv2\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.543225 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.543062    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-tmp-dir\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.543225 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.543087    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-model-cache\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.543225 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.543137    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-home\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.543225 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.543159    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-tls-certs\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.543588 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.543561    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-tmp-dir\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.543742 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.543619    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-model-cache\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.543742 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.543725    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-home\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.545763 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.545734    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-dshm\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.546052 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.546029    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-tls-certs\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.553305 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.553223    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpwv2\" (UniqueName: \"kubernetes.io/projected/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-kube-api-access-wpwv2\") pod \"auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") " pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.615033 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.614999    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:15:53.766797 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:53.766764    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"]
Jul 24 16:15:53.770262 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:15:53.770227    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d1b1cc7_3ca3_4b16_b18c_ae2b06ca1581.slice/crio-ce26d5f0cc99221c1af46530d0382de5ab69534dd877ff70874b11bda7d94538 WatchSource:0}: Error finding container ce26d5f0cc99221c1af46530d0382de5ab69534dd877ff70874b11bda7d94538: Status 404 returned error can't find the container with id ce26d5f0cc99221c1af46530d0382de5ab69534dd877ff70874b11bda7d94538
Jul 24 16:15:54.300274 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:54.300236    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv" event={"ID":"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581","Type":"ContainerStarted","Data":"28eae8a080f376eadf5fbd7cae9c0ebbe92e0ab0f29a0259800fdfdb3c6946d6"}
Jul 24 16:15:54.300274 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:54.300280    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv" event={"ID":"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581","Type":"ContainerStarted","Data":"ce26d5f0cc99221c1af46530d0382de5ab69534dd877ff70874b11bda7d94538"}
Jul 24 16:15:54.319616 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:15:54.319558    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv" podStartSLOduration=1.319541227 podStartE2EDuration="1.319541227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:15:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:15:54.318165862 +0000 UTC m=+2382.728876479" watchObservedRunningTime="2026-07-24 16:15:54.319541227 +0000 UTC m=+2382.730251878"
Jul 24 16:16:03.615961 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:16:03.615906    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:16:03.615961 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:16:03.615973    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:16:03.629118 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:16:03.629085    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:16:04.362198 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:16:04.362170    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:18:05.487375 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:05.487271    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"]
Jul 24 16:18:09.015647 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.015607    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"]
Jul 24 16:18:09.019198 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.019177    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.023119 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.023093    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fafc28\"/\"sub-test-model-fafc28-kserve-self-signed-certs\""
Jul 24 16:18:09.023119 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.023116    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fafc28\"/\"kube-root-ca.crt\""
Jul 24 16:18:09.023323 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.023137    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fafc28\"/\"default-dockercfg-pzqn4\""
Jul 24 16:18:09.023323 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.023116    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fafc28\"/\"openshift-service-ca.crt\""
Jul 24 16:18:09.035599 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.035560    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"]
Jul 24 16:18:09.073900 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.073862    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-home\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.074084 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.073906    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-dshm\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.074084 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.073984    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-model-cache\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.074084 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.074009    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5df698da-0395-47f3-b6b1-7dfac17a18dd-tls-certs\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.074084 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.074046    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp9zd\" (UniqueName: \"kubernetes.io/projected/5df698da-0395-47f3-b6b1-7dfac17a18dd-kube-api-access-tp9zd\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.074233 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.074085    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-tmp-dir\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.174810 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.174767    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp9zd\" (UniqueName: \"kubernetes.io/projected/5df698da-0395-47f3-b6b1-7dfac17a18dd-kube-api-access-tp9zd\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.174810 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.174811    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-tmp-dir\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.175073 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.174866    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-home\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.175073 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.174998    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-dshm\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.175073 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.175080    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-model-cache\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.175261 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.175099    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5df698da-0395-47f3-b6b1-7dfac17a18dd-tls-certs\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.175320 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.175254    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-tmp-dir\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.175320 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.175300    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-home\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.175399 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.175373    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-model-cache\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.177493 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.177461    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-dshm\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.177700 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.177683    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5df698da-0395-47f3-b6b1-7dfac17a18dd-tls-certs\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.185813 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.185775    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp9zd\" (UniqueName: \"kubernetes.io/projected/5df698da-0395-47f3-b6b1-7dfac17a18dd-kube-api-access-tp9zd\") pod \"sub-test-model-fafc28-kserve-7957d69fcf-db7mk\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") " pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.329284 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.329188    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:09.492138 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.492107    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"]
Jul 24 16:18:09.495059 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:18:09.495030    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5df698da_0395_47f3_b6b1_7dfac17a18dd.slice/crio-82deccaaed705e43db00c2bedd30909b8927ed6e2f989e174be80bb328d3e970 WatchSource:0}: Error finding container 82deccaaed705e43db00c2bedd30909b8927ed6e2f989e174be80bb328d3e970: Status 404 returned error can't find the container with id 82deccaaed705e43db00c2bedd30909b8927ed6e2f989e174be80bb328d3e970
Jul 24 16:18:09.986240 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.986206    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk" event={"ID":"5df698da-0395-47f3-b6b1-7dfac17a18dd","Type":"ContainerStarted","Data":"948a5d16d8e0f4cc40c2ff67143553fa7f89746cc3da90735f05522f7254f6c5"}
Jul 24 16:18:09.986240 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:09.986246    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk" event={"ID":"5df698da-0395-47f3-b6b1-7dfac17a18dd","Type":"ContainerStarted","Data":"82deccaaed705e43db00c2bedd30909b8927ed6e2f989e174be80bb328d3e970"}
Jul 24 16:18:10.026036 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:10.025973    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk" podStartSLOduration=2.025954736 podStartE2EDuration="2.025954736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:18:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:18:10.025382515 +0000 UTC m=+2518.436093136" watchObservedRunningTime="2026-07-24 16:18:10.025954736 +0000 UTC m=+2518.436665408"
Jul 24 16:18:19.329628 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:19.329581    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:19.330226 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:19.329642    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:19.343019 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:19.342978    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:20.050240 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.050209    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:18:20.508113 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.508063    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv" podUID="9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581" containerName="main" containerID="cri-o://28eae8a080f376eadf5fbd7cae9c0ebbe92e0ab0f29a0259800fdfdb3c6946d6" gracePeriod=45
Jul 24 16:18:20.770032 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.769958    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:18:20.876905 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.876869    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-home\") pod \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") "
Jul 24 16:18:20.877102 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.876936    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-dshm\") pod \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") "
Jul 24 16:18:20.877102 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.876968    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-tls-certs\") pod \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") "
Jul 24 16:18:20.877102 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.876990    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-tmp-dir\") pod \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") "
Jul 24 16:18:20.877102 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.877029    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-model-cache\") pod \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") "
Jul 24 16:18:20.877102 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.877066    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpwv2\" (UniqueName: \"kubernetes.io/projected/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-kube-api-access-wpwv2\") pod \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\" (UID: \"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581\") "
Jul 24 16:18:20.877384 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.877191    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-home" (OuterVolumeSpecName: "home") pod "9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581" (UID: "9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:18:20.877384 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.877270    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581" (UID: "9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:18:20.877384 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.877361    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-tmp-dir\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:18:20.877384 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.877379    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-home\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:18:20.877532 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.877397    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-model-cache" (OuterVolumeSpecName: "model-cache") pod "9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581" (UID: "9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:18:20.879411 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.879390    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-dshm" (OuterVolumeSpecName: "dshm") pod "9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581" (UID: "9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:18:20.879498 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.879441    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581" (UID: "9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:18:20.879536 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.879514    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-kube-api-access-wpwv2" (OuterVolumeSpecName: "kube-api-access-wpwv2") pod "9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581" (UID: "9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581"). InnerVolumeSpecName "kube-api-access-wpwv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:18:20.978635 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.978595    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-model-cache\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:18:20.978635 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.978629    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpwv2\" (UniqueName: \"kubernetes.io/projected/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-kube-api-access-wpwv2\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:18:20.978635 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.978641    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-dshm\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:18:20.978893 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:20.978651    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581-tls-certs\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:18:21.045444 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:21.045359    2561 generic.go:356] "Generic (PLEG): container finished" podID="9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581" containerID="28eae8a080f376eadf5fbd7cae9c0ebbe92e0ab0f29a0259800fdfdb3c6946d6" exitCode=0
Jul 24 16:18:21.045444 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:21.045405    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"
Jul 24 16:18:21.045444 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:21.045436    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv" event={"ID":"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581","Type":"ContainerDied","Data":"28eae8a080f376eadf5fbd7cae9c0ebbe92e0ab0f29a0259800fdfdb3c6946d6"}
Jul 24 16:18:21.045774 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:21.045475    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv" event={"ID":"9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581","Type":"ContainerDied","Data":"ce26d5f0cc99221c1af46530d0382de5ab69534dd877ff70874b11bda7d94538"}
Jul 24 16:18:21.045774 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:21.045508    2561 scope.go:117] "RemoveContainer" containerID="28eae8a080f376eadf5fbd7cae9c0ebbe92e0ab0f29a0259800fdfdb3c6946d6"
Jul 24 16:18:21.055730 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:21.055710    2561 scope.go:117] "RemoveContainer" containerID="28eae8a080f376eadf5fbd7cae9c0ebbe92e0ab0f29a0259800fdfdb3c6946d6"
Jul 24 16:18:21.056001 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:18:21.055983    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28eae8a080f376eadf5fbd7cae9c0ebbe92e0ab0f29a0259800fdfdb3c6946d6\": container with ID starting with 28eae8a080f376eadf5fbd7cae9c0ebbe92e0ab0f29a0259800fdfdb3c6946d6 not found: ID does not exist" containerID="28eae8a080f376eadf5fbd7cae9c0ebbe92e0ab0f29a0259800fdfdb3c6946d6"
Jul 24 16:18:21.056081 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:21.056009    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28eae8a080f376eadf5fbd7cae9c0ebbe92e0ab0f29a0259800fdfdb3c6946d6"} err="failed to get container status \"28eae8a080f376eadf5fbd7cae9c0ebbe92e0ab0f29a0259800fdfdb3c6946d6\": rpc error: code = NotFound desc = could not find container \"28eae8a080f376eadf5fbd7cae9c0ebbe92e0ab0f29a0259800fdfdb3c6946d6\": container with ID starting with 28eae8a080f376eadf5fbd7cae9c0ebbe92e0ab0f29a0259800fdfdb3c6946d6 not found: ID does not exist"
Jul 24 16:18:21.073413 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:21.073380    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"]
Jul 24 16:18:21.080987 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:21.080964    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/auth-test-model-c6c91e-kserve-68c5f59bb7-w5mpv"]
Jul 24 16:18:22.106456 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:22.106419    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581" path="/var/lib/kubelet/pods/9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581/volumes"
Jul 24 16:18:31.960193 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:31.960153    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"]
Jul 24 16:18:31.960605 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:31.960535    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581" containerName="main"
Jul 24 16:18:31.960605 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:31.960547    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581" containerName="main"
Jul 24 16:18:31.960721 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:31.960607    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d1b1cc7-3ca3-4b16-b18c-ae2b06ca1581" containerName="main"
Jul 24 16:18:31.964769 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:31.964745    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:31.967996 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:31.967972    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c6c91e\"/\"sub-test-model-c6c91e-kserve-self-signed-certs\""
Jul 24 16:18:31.968133 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:31.968001    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c6c91e\"/\"kube-root-ca.crt\""
Jul 24 16:18:31.968133 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:31.968025    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c6c91e\"/\"default-dockercfg-nr5hm\""
Jul 24 16:18:31.968133 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:31.968001    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c6c91e\"/\"openshift-service-ca.crt\""
Jul 24 16:18:31.984197 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:31.984167    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"]
Jul 24 16:18:32.082878 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.082841    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-dshm\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.082878 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.082891    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-model-cache\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.083138 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.082925    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5fm9\" (UniqueName: \"kubernetes.io/projected/b23b97d9-fda3-4e94-b0f0-457c60f66502-kube-api-access-s5fm9\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.083138 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.082976    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-home\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.083138 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.083067    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b23b97d9-fda3-4e94-b0f0-457c60f66502-tls-certs\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.083138 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.083119    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-tmp-dir\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.184419 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.184376    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b23b97d9-fda3-4e94-b0f0-457c60f66502-tls-certs\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.184614 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.184426    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-tmp-dir\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.184685 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.184619    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-dshm\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.184750 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.184732    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-model-cache\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.184790 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.184771    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5fm9\" (UniqueName: \"kubernetes.io/projected/b23b97d9-fda3-4e94-b0f0-457c60f66502-kube-api-access-s5fm9\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.184839 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.184799    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-home\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.184883 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.184834    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-tmp-dir\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.185086 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.185064    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-model-cache\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.185189 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.185099    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-home\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.187185 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.187152    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-dshm\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.187393 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.187374    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b23b97d9-fda3-4e94-b0f0-457c60f66502-tls-certs\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.196215 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.196187    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5fm9\" (UniqueName: \"kubernetes.io/projected/b23b97d9-fda3-4e94-b0f0-457c60f66502-kube-api-access-s5fm9\") pod \"sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") " pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.275045 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.274949    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:32.451333 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:32.451295    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"]
Jul 24 16:18:32.454301 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:18:32.454276    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb23b97d9_fda3_4e94_b0f0_457c60f66502.slice/crio-d643754b5512275ebe19880727771b98f6b23b7d79078a5c4833945bc7bb9adc WatchSource:0}: Error finding container d643754b5512275ebe19880727771b98f6b23b7d79078a5c4833945bc7bb9adc: Status 404 returned error can't find the container with id d643754b5512275ebe19880727771b98f6b23b7d79078a5c4833945bc7bb9adc
Jul 24 16:18:33.107980 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:33.107939    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd" event={"ID":"b23b97d9-fda3-4e94-b0f0-457c60f66502","Type":"ContainerStarted","Data":"01d8aa1a79e817d271e473747d3baae05a86fcedd8518d0322c3134789af5d5a"}
Jul 24 16:18:33.108402 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:33.107995    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd" event={"ID":"b23b97d9-fda3-4e94-b0f0-457c60f66502","Type":"ContainerStarted","Data":"d643754b5512275ebe19880727771b98f6b23b7d79078a5c4833945bc7bb9adc"}
Jul 24 16:18:33.137904 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:33.137822    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd" podStartSLOduration=2.137798943 podStartE2EDuration="2.137798943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:18:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:18:33.136985449 +0000 UTC m=+2541.547696079" watchObservedRunningTime="2026-07-24 16:18:33.137798943 +0000 UTC m=+2541.548509565"
Jul 24 16:18:42.275770 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:42.275721    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:42.276227 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:42.275791    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:42.288832 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:42.288806    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:18:43.175422 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:18:43.175392    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:19:25.260755 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:25.260710    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"]
Jul 24 16:19:26.322477 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:26.322436    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"]
Jul 24 16:19:40.281354 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.281244    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk" podUID="5df698da-0395-47f3-b6b1-7dfac17a18dd" containerName="main" containerID="cri-o://948a5d16d8e0f4cc40c2ff67143553fa7f89746cc3da90735f05522f7254f6c5" gracePeriod=45
Jul 24 16:19:40.458426 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.458395    2561 generic.go:356] "Generic (PLEG): container finished" podID="5df698da-0395-47f3-b6b1-7dfac17a18dd" containerID="948a5d16d8e0f4cc40c2ff67143553fa7f89746cc3da90735f05522f7254f6c5" exitCode=0
Jul 24 16:19:40.458623 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.458475    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk" event={"ID":"5df698da-0395-47f3-b6b1-7dfac17a18dd","Type":"ContainerDied","Data":"948a5d16d8e0f4cc40c2ff67143553fa7f89746cc3da90735f05522f7254f6c5"}
Jul 24 16:19:40.559380 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.559350    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:19:40.706920 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.706885    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-model-cache\") pod \"5df698da-0395-47f3-b6b1-7dfac17a18dd\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") "
Jul 24 16:19:40.707108 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.706936    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-home\") pod \"5df698da-0395-47f3-b6b1-7dfac17a18dd\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") "
Jul 24 16:19:40.707108 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.706966    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tp9zd\" (UniqueName: \"kubernetes.io/projected/5df698da-0395-47f3-b6b1-7dfac17a18dd-kube-api-access-tp9zd\") pod \"5df698da-0395-47f3-b6b1-7dfac17a18dd\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") "
Jul 24 16:19:40.707108 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.706991    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-tmp-dir\") pod \"5df698da-0395-47f3-b6b1-7dfac17a18dd\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") "
Jul 24 16:19:40.707108 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.707059    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5df698da-0395-47f3-b6b1-7dfac17a18dd-tls-certs\") pod \"5df698da-0395-47f3-b6b1-7dfac17a18dd\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") "
Jul 24 16:19:40.707312 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.707130    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-dshm\") pod \"5df698da-0395-47f3-b6b1-7dfac17a18dd\" (UID: \"5df698da-0395-47f3-b6b1-7dfac17a18dd\") "
Jul 24 16:19:40.707312 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.707251    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-model-cache" (OuterVolumeSpecName: "model-cache") pod "5df698da-0395-47f3-b6b1-7dfac17a18dd" (UID: "5df698da-0395-47f3-b6b1-7dfac17a18dd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:19:40.707312 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.707269    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-home" (OuterVolumeSpecName: "home") pod "5df698da-0395-47f3-b6b1-7dfac17a18dd" (UID: "5df698da-0395-47f3-b6b1-7dfac17a18dd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:19:40.707481 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.707323    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5df698da-0395-47f3-b6b1-7dfac17a18dd" (UID: "5df698da-0395-47f3-b6b1-7dfac17a18dd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:19:40.707481 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.707455    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-home\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:19:40.707481 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.707472    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-tmp-dir\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:19:40.707481 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.707483    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-model-cache\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:19:40.709621 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.709595    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5df698da-0395-47f3-b6b1-7dfac17a18dd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5df698da-0395-47f3-b6b1-7dfac17a18dd" (UID: "5df698da-0395-47f3-b6b1-7dfac17a18dd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:19:40.709944 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.709912    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-dshm" (OuterVolumeSpecName: "dshm") pod "5df698da-0395-47f3-b6b1-7dfac17a18dd" (UID: "5df698da-0395-47f3-b6b1-7dfac17a18dd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:19:40.710042 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.709949    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5df698da-0395-47f3-b6b1-7dfac17a18dd-kube-api-access-tp9zd" (OuterVolumeSpecName: "kube-api-access-tp9zd") pod "5df698da-0395-47f3-b6b1-7dfac17a18dd" (UID: "5df698da-0395-47f3-b6b1-7dfac17a18dd"). InnerVolumeSpecName "kube-api-access-tp9zd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:19:40.808536 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.808444    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5df698da-0395-47f3-b6b1-7dfac17a18dd-tls-certs\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:19:40.808536 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.808479    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5df698da-0395-47f3-b6b1-7dfac17a18dd-dshm\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:19:40.808536 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:40.808489    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tp9zd\" (UniqueName: \"kubernetes.io/projected/5df698da-0395-47f3-b6b1-7dfac17a18dd-kube-api-access-tp9zd\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:19:41.345371 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.345327    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd" podUID="b23b97d9-fda3-4e94-b0f0-457c60f66502" containerName="main" containerID="cri-o://01d8aa1a79e817d271e473747d3baae05a86fcedd8518d0322c3134789af5d5a" gracePeriod=45
Jul 24 16:19:41.466066 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.466034    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"
Jul 24 16:19:41.466066 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.466061    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk" event={"ID":"5df698da-0395-47f3-b6b1-7dfac17a18dd","Type":"ContainerDied","Data":"82deccaaed705e43db00c2bedd30909b8927ed6e2f989e174be80bb328d3e970"}
Jul 24 16:19:41.466349 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.466124    2561 scope.go:117] "RemoveContainer" containerID="948a5d16d8e0f4cc40c2ff67143553fa7f89746cc3da90735f05522f7254f6c5"
Jul 24 16:19:41.467985 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.467957    2561 generic.go:356] "Generic (PLEG): container finished" podID="b23b97d9-fda3-4e94-b0f0-457c60f66502" containerID="01d8aa1a79e817d271e473747d3baae05a86fcedd8518d0322c3134789af5d5a" exitCode=0
Jul 24 16:19:41.468133 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.468013    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd" event={"ID":"b23b97d9-fda3-4e94-b0f0-457c60f66502","Type":"ContainerDied","Data":"01d8aa1a79e817d271e473747d3baae05a86fcedd8518d0322c3134789af5d5a"}
Jul 24 16:19:41.513189 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.513148    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"]
Jul 24 16:19:41.517985 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.517947    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-fafc28/sub-test-model-fafc28-kserve-7957d69fcf-db7mk"]
Jul 24 16:19:41.621301 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.621273    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:19:41.716917 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.716880    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b23b97d9-fda3-4e94-b0f0-457c60f66502-tls-certs\") pod \"b23b97d9-fda3-4e94-b0f0-457c60f66502\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") "
Jul 24 16:19:41.717122 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.716960    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5fm9\" (UniqueName: \"kubernetes.io/projected/b23b97d9-fda3-4e94-b0f0-457c60f66502-kube-api-access-s5fm9\") pod \"b23b97d9-fda3-4e94-b0f0-457c60f66502\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") "
Jul 24 16:19:41.717122 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.717023    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-tmp-dir\") pod \"b23b97d9-fda3-4e94-b0f0-457c60f66502\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") "
Jul 24 16:19:41.717122 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.717056    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-model-cache\") pod \"b23b97d9-fda3-4e94-b0f0-457c60f66502\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") "
Jul 24 16:19:41.717122 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.717074    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-home\") pod \"b23b97d9-fda3-4e94-b0f0-457c60f66502\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") "
Jul 24 16:19:41.717122 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.717119    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-dshm\") pod \"b23b97d9-fda3-4e94-b0f0-457c60f66502\" (UID: \"b23b97d9-fda3-4e94-b0f0-457c60f66502\") "
Jul 24 16:19:41.717410 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.717366    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-model-cache" (OuterVolumeSpecName: "model-cache") pod "b23b97d9-fda3-4e94-b0f0-457c60f66502" (UID: "b23b97d9-fda3-4e94-b0f0-457c60f66502"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:19:41.717410 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.717376    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b23b97d9-fda3-4e94-b0f0-457c60f66502" (UID: "b23b97d9-fda3-4e94-b0f0-457c60f66502"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:19:41.717410 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.717390    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-home" (OuterVolumeSpecName: "home") pod "b23b97d9-fda3-4e94-b0f0-457c60f66502" (UID: "b23b97d9-fda3-4e94-b0f0-457c60f66502"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:19:41.719383 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.719353    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b23b97d9-fda3-4e94-b0f0-457c60f66502-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b23b97d9-fda3-4e94-b0f0-457c60f66502" (UID: "b23b97d9-fda3-4e94-b0f0-457c60f66502"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:19:41.719383 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.719367    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b23b97d9-fda3-4e94-b0f0-457c60f66502-kube-api-access-s5fm9" (OuterVolumeSpecName: "kube-api-access-s5fm9") pod "b23b97d9-fda3-4e94-b0f0-457c60f66502" (UID: "b23b97d9-fda3-4e94-b0f0-457c60f66502"). InnerVolumeSpecName "kube-api-access-s5fm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:19:41.719586 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.719477    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-dshm" (OuterVolumeSpecName: "dshm") pod "b23b97d9-fda3-4e94-b0f0-457c60f66502" (UID: "b23b97d9-fda3-4e94-b0f0-457c60f66502"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:19:41.818640 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.818593    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-model-cache\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:19:41.818640 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.818631    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-home\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:19:41.818640 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.818642    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-dshm\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:19:41.818640 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.818652    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b23b97d9-fda3-4e94-b0f0-457c60f66502-tls-certs\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:19:41.818971 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.818701    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5fm9\" (UniqueName: \"kubernetes.io/projected/b23b97d9-fda3-4e94-b0f0-457c60f66502-kube-api-access-s5fm9\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:19:41.818971 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:41.818715    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b23b97d9-fda3-4e94-b0f0-457c60f66502-tmp-dir\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:19:42.108161 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:42.108123    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5df698da-0395-47f3-b6b1-7dfac17a18dd" path="/var/lib/kubelet/pods/5df698da-0395-47f3-b6b1-7dfac17a18dd/volumes"
Jul 24 16:19:42.475744 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:42.475708    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"
Jul 24 16:19:42.475744 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:42.475721    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd" event={"ID":"b23b97d9-fda3-4e94-b0f0-457c60f66502","Type":"ContainerDied","Data":"d643754b5512275ebe19880727771b98f6b23b7d79078a5c4833945bc7bb9adc"}
Jul 24 16:19:42.476343 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:42.475783    2561 scope.go:117] "RemoveContainer" containerID="01d8aa1a79e817d271e473747d3baae05a86fcedd8518d0322c3134789af5d5a"
Jul 24 16:19:42.507798 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:42.507757    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"]
Jul 24 16:19:42.521486 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:42.521444    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/sub-test-model-c6c91e-kserve-cbdd86b7c-79lhd"]
Jul 24 16:19:44.107253 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:19:44.107213    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b23b97d9-fda3-4e94-b0f0-457c60f66502" path="/var/lib/kubelet/pods/b23b97d9-fda3-4e94-b0f0-457c60f66502/volumes"
Jul 24 16:20:02.979252 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:02.979208    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"]
Jul 24 16:20:02.979768 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:02.979616    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b23b97d9-fda3-4e94-b0f0-457c60f66502" containerName="main"
Jul 24 16:20:02.979768 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:02.979627    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b23b97d9-fda3-4e94-b0f0-457c60f66502" containerName="main"
Jul 24 16:20:02.979768 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:02.979638    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5df698da-0395-47f3-b6b1-7dfac17a18dd" containerName="main"
Jul 24 16:20:02.979768 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:02.979643    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="5df698da-0395-47f3-b6b1-7dfac17a18dd" containerName="main"
Jul 24 16:20:02.979768 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:02.979714    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="5df698da-0395-47f3-b6b1-7dfac17a18dd" containerName="main"
Jul 24 16:20:02.979768 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:02.979725    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b23b97d9-fda3-4e94-b0f0-457c60f66502" containerName="main"
Jul 24 16:20:02.982819 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:02.982799    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:02.989646 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:02.989234    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c6c91e\"/\"kube-root-ca.crt\""
Jul 24 16:20:02.990709 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:02.990688    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c6c91e\"/\"openshift-service-ca.crt\""
Jul 24 16:20:02.998453 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:02.998425    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c6c91e\"/\"default-dockercfg-nr5hm\""
Jul 24 16:20:02.999144 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:02.999123    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c6c91e\"/\"rate-test-model-c6c91e-kserve-self-signed-certs\""
Jul 24 16:20:03.016394 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.016349    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-tmp-dir\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.016394 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.016390    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm7mn\" (UniqueName: \"kubernetes.io/projected/7e832d20-0edd-4703-a141-55b948c73667-kube-api-access-dm7mn\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.016622 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.016419    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-home\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.016622 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.016439    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-dshm\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.016622 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.016469    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-model-cache\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.016622 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.016533    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e832d20-0edd-4703-a141-55b948c73667-tls-certs\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.023363 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.023331    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"]
Jul 24 16:20:03.117861 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.117815    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e832d20-0edd-4703-a141-55b948c73667-tls-certs\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.118063 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.117871    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-tmp-dir\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.118063 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.117895    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm7mn\" (UniqueName: \"kubernetes.io/projected/7e832d20-0edd-4703-a141-55b948c73667-kube-api-access-dm7mn\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.118063 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.117935    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-home\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.118063 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.117953    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-dshm\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.118063 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.117969    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-model-cache\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.118335 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.118314    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-tmp-dir\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.118428 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.118413    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-home\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.118470 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.118452    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-model-cache\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.120495 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.120472    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-dshm\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.120675 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.120641    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e832d20-0edd-4703-a141-55b948c73667-tls-certs\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.128451 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.128419    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm7mn\" (UniqueName: \"kubernetes.io/projected/7e832d20-0edd-4703-a141-55b948c73667-kube-api-access-dm7mn\") pod \"rate-test-model-c6c91e-kserve-c77d66d47-bbl8c\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") " pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.292486 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.292384    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:03.461297 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.461268    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"]
Jul 24 16:20:03.463339 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:20:03.463310    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e832d20_0edd_4703_a141_55b948c73667.slice/crio-0890a8b52fde82968980335e5b1ba54fcd96ff9189ca1a424917de4907610e9d WatchSource:0}: Error finding container 0890a8b52fde82968980335e5b1ba54fcd96ff9189ca1a424917de4907610e9d: Status 404 returned error can't find the container with id 0890a8b52fde82968980335e5b1ba54fcd96ff9189ca1a424917de4907610e9d
Jul 24 16:20:03.582116 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.582017    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c" event={"ID":"7e832d20-0edd-4703-a141-55b948c73667","Type":"ContainerStarted","Data":"33ec0be45f626cabd9f4db2bd0594c015a99faf7c3732fa36d939686f8d52c79"}
Jul 24 16:20:03.582116 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.582061    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c" event={"ID":"7e832d20-0edd-4703-a141-55b948c73667","Type":"ContainerStarted","Data":"0890a8b52fde82968980335e5b1ba54fcd96ff9189ca1a424917de4907610e9d"}
Jul 24 16:20:03.611204 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:03.611135    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c" podStartSLOduration=1.611110705 podStartE2EDuration="1.611110705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:20:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:20:03.607931184 +0000 UTC m=+2632.018641801" watchObservedRunningTime="2026-07-24 16:20:03.611110705 +0000 UTC m=+2632.021821325"
Jul 24 16:20:13.293412 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:13.293370    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:13.294036 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:13.293430    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:13.306376 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:13.306344    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:13.655070 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:13.654985    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:20:53.820050 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:53.820007    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"]
Jul 24 16:20:54.863645 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:54.863604    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7484-140031903357616"]
Jul 24 16:20:54.867204 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:54.867184    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031903357616"
Jul 24 16:20:54.869840 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:54.869813    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-xzxkn\""
Jul 24 16:20:54.888254 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:54.888223    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7484-140031903357616"]
Jul 24 16:20:54.993872 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:54.993825    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmm8n\" (UniqueName: \"kubernetes.io/projected/80d1514c-43a5-4bed-a94f-e264899fd4ad-kube-api-access-rmm8n\") pod \"test-curl-7484-140031903357616\" (UID: \"80d1514c-43a5-4bed-a94f-e264899fd4ad\") " pod="opendatahub/test-curl-7484-140031903357616"
Jul 24 16:20:55.095389 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:55.095348    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmm8n\" (UniqueName: \"kubernetes.io/projected/80d1514c-43a5-4bed-a94f-e264899fd4ad-kube-api-access-rmm8n\") pod \"test-curl-7484-140031903357616\" (UID: \"80d1514c-43a5-4bed-a94f-e264899fd4ad\") " pod="opendatahub/test-curl-7484-140031903357616"
Jul 24 16:20:55.104812 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:55.104783    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmm8n\" (UniqueName: \"kubernetes.io/projected/80d1514c-43a5-4bed-a94f-e264899fd4ad-kube-api-access-rmm8n\") pod \"test-curl-7484-140031903357616\" (UID: \"80d1514c-43a5-4bed-a94f-e264899fd4ad\") " pod="opendatahub/test-curl-7484-140031903357616"
Jul 24 16:20:55.176681 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:55.176569    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031903357616"
Jul 24 16:20:55.326693 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:55.326634    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7484-140031903357616"]
Jul 24 16:20:55.333365 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:20:55.333329    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80d1514c_43a5_4bed_a94f_e264899fd4ad.slice/crio-5b6dd57064b499b2e97baecda489eb1a4567193e0cf8dad56e46b75d152ea4e9 WatchSource:0}: Error finding container 5b6dd57064b499b2e97baecda489eb1a4567193e0cf8dad56e46b75d152ea4e9: Status 404 returned error can't find the container with id 5b6dd57064b499b2e97baecda489eb1a4567193e0cf8dad56e46b75d152ea4e9
Jul 24 16:20:55.334852 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:55.334832    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 16:20:55.853284 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:55.853245    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7484-140031903357616" event={"ID":"80d1514c-43a5-4bed-a94f-e264899fd4ad","Type":"ContainerStarted","Data":"5b6dd57064b499b2e97baecda489eb1a4567193e0cf8dad56e46b75d152ea4e9"}
Jul 24 16:20:56.859914 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:56.859872    2561 generic.go:356] "Generic (PLEG): container finished" podID="80d1514c-43a5-4bed-a94f-e264899fd4ad" containerID="ea734ed2cd9514a9a610fa3d7ea2a876fbe4a063ea0343d90c360373117ba170" exitCode=0
Jul 24 16:20:56.860316 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:56.859996    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7484-140031903357616" event={"ID":"80d1514c-43a5-4bed-a94f-e264899fd4ad","Type":"ContainerDied","Data":"ea734ed2cd9514a9a610fa3d7ea2a876fbe4a063ea0343d90c360373117ba170"}
Jul 24 16:20:58.010734 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.010698    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031903357616"
Jul 24 16:20:58.049411 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.049373    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7484-140031903357616_80d1514c-43a5-4bed-a94f-e264899fd4ad/test-curl-7484-140031903357616/0.log"
Jul 24 16:20:58.073353 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.073314    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7484-140031903357616"]
Jul 24 16:20:58.078173 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.078134    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7484-140031903357616"]
Jul 24 16:20:58.124088 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.123980    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmm8n\" (UniqueName: \"kubernetes.io/projected/80d1514c-43a5-4bed-a94f-e264899fd4ad-kube-api-access-rmm8n\") pod \"80d1514c-43a5-4bed-a94f-e264899fd4ad\" (UID: \"80d1514c-43a5-4bed-a94f-e264899fd4ad\") "
Jul 24 16:20:58.126344 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.126302    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80d1514c-43a5-4bed-a94f-e264899fd4ad-kube-api-access-rmm8n" (OuterVolumeSpecName: "kube-api-access-rmm8n") pod "80d1514c-43a5-4bed-a94f-e264899fd4ad" (UID: "80d1514c-43a5-4bed-a94f-e264899fd4ad"). InnerVolumeSpecName "kube-api-access-rmm8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:20:58.202229 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.202188    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7484-140031903357488"]
Jul 24 16:20:58.202694 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.202644    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80d1514c-43a5-4bed-a94f-e264899fd4ad" containerName="test-curl-7484-140031903357616"
Jul 24 16:20:58.202788 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.202697    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="80d1514c-43a5-4bed-a94f-e264899fd4ad" containerName="test-curl-7484-140031903357616"
Jul 24 16:20:58.202859 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.202800    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="80d1514c-43a5-4bed-a94f-e264899fd4ad" containerName="test-curl-7484-140031903357616"
Jul 24 16:20:58.205915 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.205890    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031903357488"
Jul 24 16:20:58.213213 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.213170    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7484-140031903357488"]
Jul 24 16:20:58.224903 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.224869    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmm8n\" (UniqueName: \"kubernetes.io/projected/80d1514c-43a5-4bed-a94f-e264899fd4ad-kube-api-access-rmm8n\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:20:58.325456 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.325414    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2vkr\" (UniqueName: \"kubernetes.io/projected/2181a45a-d3eb-47ec-985e-7d308c2c8174-kube-api-access-m2vkr\") pod \"test-curl-7484-140031903357488\" (UID: \"2181a45a-d3eb-47ec-985e-7d308c2c8174\") " pod="opendatahub/test-curl-7484-140031903357488"
Jul 24 16:20:58.426649 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.426549    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2vkr\" (UniqueName: \"kubernetes.io/projected/2181a45a-d3eb-47ec-985e-7d308c2c8174-kube-api-access-m2vkr\") pod \"test-curl-7484-140031903357488\" (UID: \"2181a45a-d3eb-47ec-985e-7d308c2c8174\") " pod="opendatahub/test-curl-7484-140031903357488"
Jul 24 16:20:58.435713 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.435644    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2vkr\" (UniqueName: \"kubernetes.io/projected/2181a45a-d3eb-47ec-985e-7d308c2c8174-kube-api-access-m2vkr\") pod \"test-curl-7484-140031903357488\" (UID: \"2181a45a-d3eb-47ec-985e-7d308c2c8174\") " pod="opendatahub/test-curl-7484-140031903357488"
Jul 24 16:20:58.517767 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.517729    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031903357488"
Jul 24 16:20:58.672977 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.672927    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7484-140031903357488"]
Jul 24 16:20:58.675251 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:20:58.675212    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2181a45a_d3eb_47ec_985e_7d308c2c8174.slice/crio-4e0a97b53c6568390293ca026c8b02f5847c0723fdd2963e8772c32b91ee6332 WatchSource:0}: Error finding container 4e0a97b53c6568390293ca026c8b02f5847c0723fdd2963e8772c32b91ee6332: Status 404 returned error can't find the container with id 4e0a97b53c6568390293ca026c8b02f5847c0723fdd2963e8772c32b91ee6332
Jul 24 16:20:58.872094 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.872061    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031903357616"
Jul 24 16:20:58.872094 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.872077    2561 scope.go:117] "RemoveContainer" containerID="ea734ed2cd9514a9a610fa3d7ea2a876fbe4a063ea0343d90c360373117ba170"
Jul 24 16:20:58.873475 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:58.873446    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7484-140031903357488" event={"ID":"2181a45a-d3eb-47ec-985e-7d308c2c8174","Type":"ContainerStarted","Data":"4e0a97b53c6568390293ca026c8b02f5847c0723fdd2963e8772c32b91ee6332"}
Jul 24 16:20:59.881291 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:59.881247    2561 generic.go:356] "Generic (PLEG): container finished" podID="2181a45a-d3eb-47ec-985e-7d308c2c8174" containerID="d38abdbc8dfcdaa1c27f5f954c9813efac699de25ad96abd4e1406cfd9ef626f" exitCode=0
Jul 24 16:20:59.881723 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:20:59.881335    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7484-140031903357488" event={"ID":"2181a45a-d3eb-47ec-985e-7d308c2c8174","Type":"ContainerDied","Data":"d38abdbc8dfcdaa1c27f5f954c9813efac699de25ad96abd4e1406cfd9ef626f"}
Jul 24 16:21:00.106984 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:00.106950    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80d1514c-43a5-4bed-a94f-e264899fd4ad" path="/var/lib/kubelet/pods/80d1514c-43a5-4bed-a94f-e264899fd4ad/volumes"
Jul 24 16:21:01.032966 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.032938    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031903357488"
Jul 24 16:21:01.075540 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.075503    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7484-140031903357488_2181a45a-d3eb-47ec-985e-7d308c2c8174/test-curl-7484-140031903357488/0.log"
Jul 24 16:21:01.104388 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.104352    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7484-140031903357488"]
Jul 24 16:21:01.111222 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.111193    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7484-140031903357488"]
Jul 24 16:21:01.152415 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.152315    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2vkr\" (UniqueName: \"kubernetes.io/projected/2181a45a-d3eb-47ec-985e-7d308c2c8174-kube-api-access-m2vkr\") pod \"2181a45a-d3eb-47ec-985e-7d308c2c8174\" (UID: \"2181a45a-d3eb-47ec-985e-7d308c2c8174\") "
Jul 24 16:21:01.154620 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.154588    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2181a45a-d3eb-47ec-985e-7d308c2c8174-kube-api-access-m2vkr" (OuterVolumeSpecName: "kube-api-access-m2vkr") pod "2181a45a-d3eb-47ec-985e-7d308c2c8174" (UID: "2181a45a-d3eb-47ec-985e-7d308c2c8174"). InnerVolumeSpecName "kube-api-access-m2vkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:21:01.253046 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.253012    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2vkr\" (UniqueName: \"kubernetes.io/projected/2181a45a-d3eb-47ec-985e-7d308c2c8174-kube-api-access-m2vkr\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:21:01.315341 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.315298    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7484-140031902815984"]
Jul 24 16:21:01.315768 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.315753    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2181a45a-d3eb-47ec-985e-7d308c2c8174" containerName="test-curl-7484-140031903357488"
Jul 24 16:21:01.315825 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.315770    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2181a45a-d3eb-47ec-985e-7d308c2c8174" containerName="test-curl-7484-140031903357488"
Jul 24 16:21:01.315869 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.315848    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2181a45a-d3eb-47ec-985e-7d308c2c8174" containerName="test-curl-7484-140031903357488"
Jul 24 16:21:01.318898 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.318879    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031902815984"
Jul 24 16:21:01.333512 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.333477    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7484-140031902815984"]
Jul 24 16:21:01.454167 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.454072    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6jh9\" (UniqueName: \"kubernetes.io/projected/8c9211e2-a43c-4f86-b6d3-034f771ca83a-kube-api-access-r6jh9\") pod \"test-curl-7484-140031902815984\" (UID: \"8c9211e2-a43c-4f86-b6d3-034f771ca83a\") " pod="opendatahub/test-curl-7484-140031902815984"
Jul 24 16:21:01.555501 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.555457    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6jh9\" (UniqueName: \"kubernetes.io/projected/8c9211e2-a43c-4f86-b6d3-034f771ca83a-kube-api-access-r6jh9\") pod \"test-curl-7484-140031902815984\" (UID: \"8c9211e2-a43c-4f86-b6d3-034f771ca83a\") " pod="opendatahub/test-curl-7484-140031902815984"
Jul 24 16:21:01.565581 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.565546    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6jh9\" (UniqueName: \"kubernetes.io/projected/8c9211e2-a43c-4f86-b6d3-034f771ca83a-kube-api-access-r6jh9\") pod \"test-curl-7484-140031902815984\" (UID: \"8c9211e2-a43c-4f86-b6d3-034f771ca83a\") " pod="opendatahub/test-curl-7484-140031902815984"
Jul 24 16:21:01.628972 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.628930    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031902815984"
Jul 24 16:21:01.795479 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.795301    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7484-140031902815984"]
Jul 24 16:21:01.798175 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:21:01.798145    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c9211e2_a43c_4f86_b6d3_034f771ca83a.slice/crio-e3815d5021dea8af3b6cfc7b796ec1be93ccd26467f0212288f3aed0dbc54ab2 WatchSource:0}: Error finding container e3815d5021dea8af3b6cfc7b796ec1be93ccd26467f0212288f3aed0dbc54ab2: Status 404 returned error can't find the container with id e3815d5021dea8af3b6cfc7b796ec1be93ccd26467f0212288f3aed0dbc54ab2
Jul 24 16:21:01.893628 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.893598    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e0a97b53c6568390293ca026c8b02f5847c0723fdd2963e8772c32b91ee6332"
Jul 24 16:21:01.893628 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.893594    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031903357488"
Jul 24 16:21:01.895031 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:01.894996    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7484-140031902815984" event={"ID":"8c9211e2-a43c-4f86-b6d3-034f771ca83a","Type":"ContainerStarted","Data":"e3815d5021dea8af3b6cfc7b796ec1be93ccd26467f0212288f3aed0dbc54ab2"}
Jul 24 16:21:01.904721 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:21:01.904691    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7484-140031903357488\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="2181a45a-d3eb-47ec-985e-7d308c2c8174" pod="opendatahub/test-curl-7484-140031903357488"
Jul 24 16:21:02.105598 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:21:02.105565    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7484-140031903357488\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="2181a45a-d3eb-47ec-985e-7d308c2c8174" pod="opendatahub/test-curl-7484-140031903357488"
Jul 24 16:21:02.107601 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:02.107569    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2181a45a-d3eb-47ec-985e-7d308c2c8174" path="/var/lib/kubelet/pods/2181a45a-d3eb-47ec-985e-7d308c2c8174/volumes"
Jul 24 16:21:02.901466 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:02.901383    2561 generic.go:356] "Generic (PLEG): container finished" podID="8c9211e2-a43c-4f86-b6d3-034f771ca83a" containerID="972ed05a6c9423e8da7acd551b9b457e9a00ca257bef1406a444fcdfcf9ac7d8" exitCode=0
Jul 24 16:21:02.901654 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:02.901474    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7484-140031902815984" event={"ID":"8c9211e2-a43c-4f86-b6d3-034f771ca83a","Type":"ContainerDied","Data":"972ed05a6c9423e8da7acd551b9b457e9a00ca257bef1406a444fcdfcf9ac7d8"}
Jul 24 16:21:04.077927 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.077892    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031902815984"
Jul 24 16:21:04.097457 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.097427    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7484-140031902815984_8c9211e2-a43c-4f86-b6d3-034f771ca83a/test-curl-7484-140031902815984/0.log"
Jul 24 16:21:04.120317 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.120281    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7484-140031902815984"]
Jul 24 16:21:04.123759 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.123722    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7484-140031902815984"]
Jul 24 16:21:04.178990 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.178898    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6jh9\" (UniqueName: \"kubernetes.io/projected/8c9211e2-a43c-4f86-b6d3-034f771ca83a-kube-api-access-r6jh9\") pod \"8c9211e2-a43c-4f86-b6d3-034f771ca83a\" (UID: \"8c9211e2-a43c-4f86-b6d3-034f771ca83a\") "
Jul 24 16:21:04.181294 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.181259    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c9211e2-a43c-4f86-b6d3-034f771ca83a-kube-api-access-r6jh9" (OuterVolumeSpecName: "kube-api-access-r6jh9") pod "8c9211e2-a43c-4f86-b6d3-034f771ca83a" (UID: "8c9211e2-a43c-4f86-b6d3-034f771ca83a"). InnerVolumeSpecName "kube-api-access-r6jh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:21:04.254862 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.254824    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7484-140031902815984"]
Jul 24 16:21:04.255241 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.255227    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9211e2-a43c-4f86-b6d3-034f771ca83a" containerName="test-curl-7484-140031902815984"
Jul 24 16:21:04.255282 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.255242    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9211e2-a43c-4f86-b6d3-034f771ca83a" containerName="test-curl-7484-140031902815984"
Jul 24 16:21:04.255340 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.255319    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c9211e2-a43c-4f86-b6d3-034f771ca83a" containerName="test-curl-7484-140031902815984"
Jul 24 16:21:04.258331 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.258311    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031902815984"
Jul 24 16:21:04.265207 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.265177    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7484-140031902815984"]
Jul 24 16:21:04.279646 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.279612    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6jh9\" (UniqueName: \"kubernetes.io/projected/8c9211e2-a43c-4f86-b6d3-034f771ca83a-kube-api-access-r6jh9\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:21:04.380468 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.380413    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqznv\" (UniqueName: \"kubernetes.io/projected/802e432f-0278-4f01-b422-e44e02186f5c-kube-api-access-fqznv\") pod \"test-curl-7484-140031902815984\" (UID: \"802e432f-0278-4f01-b422-e44e02186f5c\") " pod="opendatahub/test-curl-7484-140031902815984"
Jul 24 16:21:04.481614 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.481574    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqznv\" (UniqueName: \"kubernetes.io/projected/802e432f-0278-4f01-b422-e44e02186f5c-kube-api-access-fqznv\") pod \"test-curl-7484-140031902815984\" (UID: \"802e432f-0278-4f01-b422-e44e02186f5c\") " pod="opendatahub/test-curl-7484-140031902815984"
Jul 24 16:21:04.490029 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.490003    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqznv\" (UniqueName: \"kubernetes.io/projected/802e432f-0278-4f01-b422-e44e02186f5c-kube-api-access-fqznv\") pod \"test-curl-7484-140031902815984\" (UID: \"802e432f-0278-4f01-b422-e44e02186f5c\") " pod="opendatahub/test-curl-7484-140031902815984"
Jul 24 16:21:04.570056 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.570010    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031902815984"
Jul 24 16:21:04.724080 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.724033    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7484-140031902815984"]
Jul 24 16:21:04.732295 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:21:04.732233    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod802e432f_0278_4f01_b422_e44e02186f5c.slice/crio-01d612c7fbbce77857ae999b33f3de9823b9cf41b04e8ca428d74485c5c37470 WatchSource:0}: Error finding container 01d612c7fbbce77857ae999b33f3de9823b9cf41b04e8ca428d74485c5c37470: Status 404 returned error can't find the container with id 01d612c7fbbce77857ae999b33f3de9823b9cf41b04e8ca428d74485c5c37470
Jul 24 16:21:04.913992 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.913960    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031902815984"
Jul 24 16:21:04.913992 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.913977    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3815d5021dea8af3b6cfc7b796ec1be93ccd26467f0212288f3aed0dbc54ab2"
Jul 24 16:21:04.915261 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.915222    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7484-140031902815984" event={"ID":"802e432f-0278-4f01-b422-e44e02186f5c","Type":"ContainerStarted","Data":"01d612c7fbbce77857ae999b33f3de9823b9cf41b04e8ca428d74485c5c37470"}
Jul 24 16:21:04.925980 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:04.925938    2561 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7484-140031902815984" oldPodUID="8c9211e2-a43c-4f86-b6d3-034f771ca83a" podUID="802e432f-0278-4f01-b422-e44e02186f5c"
Jul 24 16:21:05.922082 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:05.922047    2561 generic.go:356] "Generic (PLEG): container finished" podID="802e432f-0278-4f01-b422-e44e02186f5c" containerID="d14ae8f5f0902fa1a3dad36e77212a1bfc5101ba11c364b9e2537270e7a6da73" exitCode=0
Jul 24 16:21:05.922463 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:05.922138    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7484-140031902815984" event={"ID":"802e432f-0278-4f01-b422-e44e02186f5c","Type":"ContainerDied","Data":"d14ae8f5f0902fa1a3dad36e77212a1bfc5101ba11c364b9e2537270e7a6da73"}
Jul 24 16:21:06.107989 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:06.107951    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c9211e2-a43c-4f86-b6d3-034f771ca83a" path="/var/lib/kubelet/pods/8c9211e2-a43c-4f86-b6d3-034f771ca83a/volumes"
Jul 24 16:21:07.082203 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.082178    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031902815984"
Jul 24 16:21:07.103434 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.103352    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7484-140031902815984_802e432f-0278-4f01-b422-e44e02186f5c/test-curl-7484-140031902815984/0.log"
Jul 24 16:21:07.128384 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.128347    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7484-140031902815984"]
Jul 24 16:21:07.131914 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.131880    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7484-140031902815984"]
Jul 24 16:21:07.208007 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.207976    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqznv\" (UniqueName: \"kubernetes.io/projected/802e432f-0278-4f01-b422-e44e02186f5c-kube-api-access-fqznv\") pod \"802e432f-0278-4f01-b422-e44e02186f5c\" (UID: \"802e432f-0278-4f01-b422-e44e02186f5c\") "
Jul 24 16:21:07.210165 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.210138    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/802e432f-0278-4f01-b422-e44e02186f5c-kube-api-access-fqznv" (OuterVolumeSpecName: "kube-api-access-fqznv") pod "802e432f-0278-4f01-b422-e44e02186f5c" (UID: "802e432f-0278-4f01-b422-e44e02186f5c"). InnerVolumeSpecName "kube-api-access-fqznv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:21:07.257803 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.257765    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7484-140031903270352"]
Jul 24 16:21:07.258189 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.258175    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="802e432f-0278-4f01-b422-e44e02186f5c" containerName="test-curl-7484-140031902815984"
Jul 24 16:21:07.258239 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.258191    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="802e432f-0278-4f01-b422-e44e02186f5c" containerName="test-curl-7484-140031902815984"
Jul 24 16:21:07.258281 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.258272    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="802e432f-0278-4f01-b422-e44e02186f5c" containerName="test-curl-7484-140031902815984"
Jul 24 16:21:07.261309 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.261287    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031903270352"
Jul 24 16:21:07.269626 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.269598    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7484-140031903270352"]
Jul 24 16:21:07.308975 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.308937    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqznv\" (UniqueName: \"kubernetes.io/projected/802e432f-0278-4f01-b422-e44e02186f5c-kube-api-access-fqznv\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:21:07.410408 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.410369    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2mzq\" (UniqueName: \"kubernetes.io/projected/0f5629d4-842f-4d32-a440-47e4144767af-kube-api-access-s2mzq\") pod \"test-curl-7484-140031903270352\" (UID: \"0f5629d4-842f-4d32-a440-47e4144767af\") " pod="opendatahub/test-curl-7484-140031903270352"
Jul 24 16:21:07.511337 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.511255    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2mzq\" (UniqueName: \"kubernetes.io/projected/0f5629d4-842f-4d32-a440-47e4144767af-kube-api-access-s2mzq\") pod \"test-curl-7484-140031903270352\" (UID: \"0f5629d4-842f-4d32-a440-47e4144767af\") " pod="opendatahub/test-curl-7484-140031903270352"
Jul 24 16:21:07.519473 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.519447    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2mzq\" (UniqueName: \"kubernetes.io/projected/0f5629d4-842f-4d32-a440-47e4144767af-kube-api-access-s2mzq\") pod \"test-curl-7484-140031903270352\" (UID: \"0f5629d4-842f-4d32-a440-47e4144767af\") " pod="opendatahub/test-curl-7484-140031903270352"
Jul 24 16:21:07.572905 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.572868    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031903270352"
Jul 24 16:21:07.721463 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.721432    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7484-140031903270352"]
Jul 24 16:21:07.724242 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:21:07.724103    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f5629d4_842f_4d32_a440_47e4144767af.slice/crio-f2ec17720c122fe8cbd1963d234d257987aa5adba2a38eb5aa396ad5e7a58097 WatchSource:0}: Error finding container f2ec17720c122fe8cbd1963d234d257987aa5adba2a38eb5aa396ad5e7a58097: Status 404 returned error can't find the container with id f2ec17720c122fe8cbd1963d234d257987aa5adba2a38eb5aa396ad5e7a58097
Jul 24 16:21:07.932955 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.932919    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031902815984"
Jul 24 16:21:07.933170 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.932920    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01d612c7fbbce77857ae999b33f3de9823b9cf41b04e8ca428d74485c5c37470"
Jul 24 16:21:07.934205 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:07.934179    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7484-140031903270352" event={"ID":"0f5629d4-842f-4d32-a440-47e4144767af","Type":"ContainerStarted","Data":"f2ec17720c122fe8cbd1963d234d257987aa5adba2a38eb5aa396ad5e7a58097"}
Jul 24 16:21:07.943644 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:21:07.943607    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7484-140031902815984\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="802e432f-0278-4f01-b422-e44e02186f5c" pod="opendatahub/test-curl-7484-140031902815984"
Jul 24 16:21:08.107636 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:08.107602    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="802e432f-0278-4f01-b422-e44e02186f5c" path="/var/lib/kubelet/pods/802e432f-0278-4f01-b422-e44e02186f5c/volumes"
Jul 24 16:21:08.840917 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:08.840874    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c" podUID="7e832d20-0edd-4703-a141-55b948c73667" containerName="main" containerID="cri-o://33ec0be45f626cabd9f4db2bd0594c015a99faf7c3732fa36d939686f8d52c79" gracePeriod=45
Jul 24 16:21:08.942386 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:08.942349    2561 generic.go:356] "Generic (PLEG): container finished" podID="0f5629d4-842f-4d32-a440-47e4144767af" containerID="62c9c9afe24b75af14c110efee50db2594f7add49d2e2d3c6b861ae48d858d5f" exitCode=0
Jul 24 16:21:08.942573 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:08.942442    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7484-140031903270352" event={"ID":"0f5629d4-842f-4d32-a440-47e4144767af","Type":"ContainerDied","Data":"62c9c9afe24b75af14c110efee50db2594f7add49d2e2d3c6b861ae48d858d5f"}
Jul 24 16:21:09.106861 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.106781    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:21:09.228012 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.227980    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-dshm\") pod \"7e832d20-0edd-4703-a141-55b948c73667\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") "
Jul 24 16:21:09.228012 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.228018    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-model-cache\") pod \"7e832d20-0edd-4703-a141-55b948c73667\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") "
Jul 24 16:21:09.228573 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.228038    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-tmp-dir\") pod \"7e832d20-0edd-4703-a141-55b948c73667\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") "
Jul 24 16:21:09.228573 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.228065    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e832d20-0edd-4703-a141-55b948c73667-tls-certs\") pod \"7e832d20-0edd-4703-a141-55b948c73667\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") "
Jul 24 16:21:09.228573 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.228106    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dm7mn\" (UniqueName: \"kubernetes.io/projected/7e832d20-0edd-4703-a141-55b948c73667-kube-api-access-dm7mn\") pod \"7e832d20-0edd-4703-a141-55b948c73667\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") "
Jul 24 16:21:09.228573 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.228128    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-home\") pod \"7e832d20-0edd-4703-a141-55b948c73667\" (UID: \"7e832d20-0edd-4703-a141-55b948c73667\") "
Jul 24 16:21:09.228573 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.228316    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7e832d20-0edd-4703-a141-55b948c73667" (UID: "7e832d20-0edd-4703-a141-55b948c73667"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:21:09.228573 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.228348    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-model-cache" (OuterVolumeSpecName: "model-cache") pod "7e832d20-0edd-4703-a141-55b948c73667" (UID: "7e832d20-0edd-4703-a141-55b948c73667"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:21:09.228573 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.228501    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-model-cache\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:21:09.228573 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.228546    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-tmp-dir\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:21:09.228573 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.228569    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-home" (OuterVolumeSpecName: "home") pod "7e832d20-0edd-4703-a141-55b948c73667" (UID: "7e832d20-0edd-4703-a141-55b948c73667"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:21:09.230594 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.230564    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e832d20-0edd-4703-a141-55b948c73667-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7e832d20-0edd-4703-a141-55b948c73667" (UID: "7e832d20-0edd-4703-a141-55b948c73667"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:21:09.230732 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.230634    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e832d20-0edd-4703-a141-55b948c73667-kube-api-access-dm7mn" (OuterVolumeSpecName: "kube-api-access-dm7mn") pod "7e832d20-0edd-4703-a141-55b948c73667" (UID: "7e832d20-0edd-4703-a141-55b948c73667"). InnerVolumeSpecName "kube-api-access-dm7mn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:21:09.230910 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.230890    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-dshm" (OuterVolumeSpecName: "dshm") pod "7e832d20-0edd-4703-a141-55b948c73667" (UID: "7e832d20-0edd-4703-a141-55b948c73667"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:21:09.329483 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.329444    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-dshm\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:21:09.329483 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.329475    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e832d20-0edd-4703-a141-55b948c73667-tls-certs\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:21:09.329483 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.329486    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dm7mn\" (UniqueName: \"kubernetes.io/projected/7e832d20-0edd-4703-a141-55b948c73667-kube-api-access-dm7mn\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:21:09.329766 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.329494    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e832d20-0edd-4703-a141-55b948c73667-home\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:21:09.949477 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.949436    2561 generic.go:356] "Generic (PLEG): container finished" podID="7e832d20-0edd-4703-a141-55b948c73667" containerID="33ec0be45f626cabd9f4db2bd0594c015a99faf7c3732fa36d939686f8d52c79" exitCode=0
Jul 24 16:21:09.949477 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.949480    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"
Jul 24 16:21:09.949723 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.949513    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c" event={"ID":"7e832d20-0edd-4703-a141-55b948c73667","Type":"ContainerDied","Data":"33ec0be45f626cabd9f4db2bd0594c015a99faf7c3732fa36d939686f8d52c79"}
Jul 24 16:21:09.949723 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.949544    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c" event={"ID":"7e832d20-0edd-4703-a141-55b948c73667","Type":"ContainerDied","Data":"0890a8b52fde82968980335e5b1ba54fcd96ff9189ca1a424917de4907610e9d"}
Jul 24 16:21:09.949723 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.949563    2561 scope.go:117] "RemoveContainer" containerID="33ec0be45f626cabd9f4db2bd0594c015a99faf7c3732fa36d939686f8d52c79"
Jul 24 16:21:09.967169 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.966986    2561 scope.go:117] "RemoveContainer" containerID="33ec0be45f626cabd9f4db2bd0594c015a99faf7c3732fa36d939686f8d52c79"
Jul 24 16:21:09.967541 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:21:09.967513    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33ec0be45f626cabd9f4db2bd0594c015a99faf7c3732fa36d939686f8d52c79\": container with ID starting with 33ec0be45f626cabd9f4db2bd0594c015a99faf7c3732fa36d939686f8d52c79 not found: ID does not exist" containerID="33ec0be45f626cabd9f4db2bd0594c015a99faf7c3732fa36d939686f8d52c79"
Jul 24 16:21:09.967687 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.967553    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33ec0be45f626cabd9f4db2bd0594c015a99faf7c3732fa36d939686f8d52c79"} err="failed to get container status \"33ec0be45f626cabd9f4db2bd0594c015a99faf7c3732fa36d939686f8d52c79\": rpc error: code = NotFound desc = could not find container \"33ec0be45f626cabd9f4db2bd0594c015a99faf7c3732fa36d939686f8d52c79\": container with ID starting with 33ec0be45f626cabd9f4db2bd0594c015a99faf7c3732fa36d939686f8d52c79 not found: ID does not exist"
Jul 24 16:21:09.980699 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.980642    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"]
Jul 24 16:21:09.987879 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:09.987840    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c6c91e/rate-test-model-c6c91e-kserve-c77d66d47-bbl8c"]
Jul 24 16:21:10.101009 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:10.100973    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031903270352"
Jul 24 16:21:10.107589 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:10.107553    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e832d20-0edd-4703-a141-55b948c73667" path="/var/lib/kubelet/pods/7e832d20-0edd-4703-a141-55b948c73667/volumes"
Jul 24 16:21:10.124078 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:10.124050    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7484-140031903270352_0f5629d4-842f-4d32-a440-47e4144767af/test-curl-7484-140031903270352/0.log"
Jul 24 16:21:10.150132 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:10.150087    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7484-140031903270352"]
Jul 24 16:21:10.152316 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:10.152289    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7484-140031903270352"]
Jul 24 16:21:10.237707 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:10.237670    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2mzq\" (UniqueName: \"kubernetes.io/projected/0f5629d4-842f-4d32-a440-47e4144767af-kube-api-access-s2mzq\") pod \"0f5629d4-842f-4d32-a440-47e4144767af\" (UID: \"0f5629d4-842f-4d32-a440-47e4144767af\") "
Jul 24 16:21:10.239860 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:10.239832    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f5629d4-842f-4d32-a440-47e4144767af-kube-api-access-s2mzq" (OuterVolumeSpecName: "kube-api-access-s2mzq") pod "0f5629d4-842f-4d32-a440-47e4144767af" (UID: "0f5629d4-842f-4d32-a440-47e4144767af"). InnerVolumeSpecName "kube-api-access-s2mzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:21:10.338635 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:10.338595    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2mzq\" (UniqueName: \"kubernetes.io/projected/0f5629d4-842f-4d32-a440-47e4144767af-kube-api-access-s2mzq\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:21:10.956976 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:10.956941    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7484-140031903270352"
Jul 24 16:21:10.957145 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:10.956948    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f2ec17720c122fe8cbd1963d234d257987aa5adba2a38eb5aa396ad5e7a58097"
Jul 24 16:21:10.967165 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:21:10.967133    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7484-140031903270352\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="0f5629d4-842f-4d32-a440-47e4144767af" pod="opendatahub/test-curl-7484-140031903270352"
Jul 24 16:21:12.105268 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:21:12.105223    2561 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7484-140031903270352\" is forbidden: User \"system:node:ip-10-0-135-164.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-164.ec2.internal' and this object" podUID="0f5629d4-842f-4d32-a440-47e4144767af" pod="opendatahub/test-curl-7484-140031903270352"
Jul 24 16:21:12.107734 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:12.107708    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f5629d4-842f-4d32-a440-47e4144767af" path="/var/lib/kubelet/pods/0f5629d4-842f-4d32-a440-47e4144767af/volumes"
Jul 24 16:21:18.710604 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.710557    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"]
Jul 24 16:21:18.711145 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.711125    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e832d20-0edd-4703-a141-55b948c73667" containerName="main"
Jul 24 16:21:18.711242 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.711147    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e832d20-0edd-4703-a141-55b948c73667" containerName="main"
Jul 24 16:21:18.711242 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.711198    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f5629d4-842f-4d32-a440-47e4144767af" containerName="test-curl-7484-140031903270352"
Jul 24 16:21:18.711242 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.711209    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f5629d4-842f-4d32-a440-47e4144767af" containerName="test-curl-7484-140031903270352"
Jul 24 16:21:18.711405 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.711296    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f5629d4-842f-4d32-a440-47e4144767af" containerName="test-curl-7484-140031903270352"
Jul 24 16:21:18.711405 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.711314    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e832d20-0edd-4703-a141-55b948c73667" containerName="main"
Jul 24 16:21:18.716075 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.716048    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"
Jul 24 16:21:18.719314 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.719291    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-5d0582-dockercfg-hm524\""
Jul 24 16:21:18.719694 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.719676    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-5d0582\""
Jul 24 16:21:18.726012 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.725987    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"]
Jul 24 16:21:18.750503 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.750463    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"]
Jul 24 16:21:18.753981 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.753959    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"
Jul 24 16:21:18.773456 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.773423    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"]
Jul 24 16:21:18.817235 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.817204    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72757fa1-7cc2-417e-aab3-cd0d0ce37c4b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b\" (UID: \"72757fa1-7cc2-417e-aab3-cd0d0ce37c4b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"
Jul 24 16:21:18.817401 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.817244    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnsxx\" (UniqueName: \"kubernetes.io/projected/72757fa1-7cc2-417e-aab3-cd0d0ce37c4b-kube-api-access-fnsxx\") pod \"payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b\" (UID: \"72757fa1-7cc2-417e-aab3-cd0d0ce37c4b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"
Jul 24 16:21:18.918481 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.918439    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72757fa1-7cc2-417e-aab3-cd0d0ce37c4b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b\" (UID: \"72757fa1-7cc2-417e-aab3-cd0d0ce37c4b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"
Jul 24 16:21:18.918651 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.918487    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnsxx\" (UniqueName: \"kubernetes.io/projected/72757fa1-7cc2-417e-aab3-cd0d0ce37c4b-kube-api-access-fnsxx\") pod \"payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b\" (UID: \"72757fa1-7cc2-417e-aab3-cd0d0ce37c4b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"
Jul 24 16:21:18.918651 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.918548    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc9a6c50-a23e-434e-8893-36d6122e8012-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4\" (UID: \"bc9a6c50-a23e-434e-8893-36d6122e8012\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"
Jul 24 16:21:18.918651 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.918570    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpqf2\" (UniqueName: \"kubernetes.io/projected/bc9a6c50-a23e-434e-8893-36d6122e8012-kube-api-access-lpqf2\") pod \"payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4\" (UID: \"bc9a6c50-a23e-434e-8893-36d6122e8012\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"
Jul 24 16:21:18.919224 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.919202    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72757fa1-7cc2-417e-aab3-cd0d0ce37c4b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b\" (UID: \"72757fa1-7cc2-417e-aab3-cd0d0ce37c4b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"
Jul 24 16:21:18.927208 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:18.927174    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnsxx\" (UniqueName: \"kubernetes.io/projected/72757fa1-7cc2-417e-aab3-cd0d0ce37c4b-kube-api-access-fnsxx\") pod \"payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b\" (UID: \"72757fa1-7cc2-417e-aab3-cd0d0ce37c4b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"
Jul 24 16:21:19.019597 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:19.019496    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc9a6c50-a23e-434e-8893-36d6122e8012-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4\" (UID: \"bc9a6c50-a23e-434e-8893-36d6122e8012\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"
Jul 24 16:21:19.019597 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:19.019538    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpqf2\" (UniqueName: \"kubernetes.io/projected/bc9a6c50-a23e-434e-8893-36d6122e8012-kube-api-access-lpqf2\") pod \"payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4\" (UID: \"bc9a6c50-a23e-434e-8893-36d6122e8012\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"
Jul 24 16:21:19.020272 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:19.020248    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc9a6c50-a23e-434e-8893-36d6122e8012-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4\" (UID: \"bc9a6c50-a23e-434e-8893-36d6122e8012\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"
Jul 24 16:21:19.027368 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:19.027341    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"
Jul 24 16:21:19.029459 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:19.029435    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpqf2\" (UniqueName: \"kubernetes.io/projected/bc9a6c50-a23e-434e-8893-36d6122e8012-kube-api-access-lpqf2\") pod \"payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4\" (UID: \"bc9a6c50-a23e-434e-8893-36d6122e8012\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"
Jul 24 16:21:19.064754 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:19.064711    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"
Jul 24 16:21:19.204874 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:19.204842    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"]
Jul 24 16:21:19.208182 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:21:19.208134    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72757fa1_7cc2_417e_aab3_cd0d0ce37c4b.slice/crio-48951ee380d9b2adc0542e0bc9ba07b27cab3beb80a44be7caf865b01ed90595 WatchSource:0}: Error finding container 48951ee380d9b2adc0542e0bc9ba07b27cab3beb80a44be7caf865b01ed90595: Status 404 returned error can't find the container with id 48951ee380d9b2adc0542e0bc9ba07b27cab3beb80a44be7caf865b01ed90595
Jul 24 16:21:19.240995 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:19.239133    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"]
Jul 24 16:21:19.249036 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:21:19.248890    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc9a6c50_a23e_434e_8893_36d6122e8012.slice/crio-e63fc8d2c569388471a444037e7608e183e21e7d129a3aba32489958a7d3f23b WatchSource:0}: Error finding container e63fc8d2c569388471a444037e7608e183e21e7d129a3aba32489958a7d3f23b: Status 404 returned error can't find the container with id e63fc8d2c569388471a444037e7608e183e21e7d129a3aba32489958a7d3f23b
Jul 24 16:21:20.009679 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:20.009629    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b" event={"ID":"72757fa1-7cc2-417e-aab3-cd0d0ce37c4b","Type":"ContainerStarted","Data":"7013fae434416b612b2617ab0edb1aa3658f6c3c6061e5d4af45a4c4589c270a"}
Jul 24 16:21:20.010150 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:20.009716    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b" event={"ID":"72757fa1-7cc2-417e-aab3-cd0d0ce37c4b","Type":"ContainerStarted","Data":"48951ee380d9b2adc0542e0bc9ba07b27cab3beb80a44be7caf865b01ed90595"}
Jul 24 16:21:20.010150 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:20.009824    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"
Jul 24 16:21:20.011328 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:20.011300    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4" event={"ID":"bc9a6c50-a23e-434e-8893-36d6122e8012","Type":"ContainerStarted","Data":"1116ddd4db5e7ef30b3b4bd7f16183d34f4d7b24dc4ee202268417497d06ca1d"}
Jul 24 16:21:20.011410 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:20.011345    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4" event={"ID":"bc9a6c50-a23e-434e-8893-36d6122e8012","Type":"ContainerStarted","Data":"e63fc8d2c569388471a444037e7608e183e21e7d129a3aba32489958a7d3f23b"}
Jul 24 16:21:20.011410 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:20.011364    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"
Jul 24 16:21:20.036964 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:20.036890    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b" podStartSLOduration=2.036867568 podStartE2EDuration="2.036867568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:21:20.034497741 +0000 UTC m=+2708.445208370" watchObservedRunningTime="2026-07-24 16:21:20.036867568 +0000 UTC m=+2708.447578185"
Jul 24 16:21:20.052772 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:20.052717    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4" podStartSLOduration=2.052700166 podStartE2EDuration="2.052700166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:21:20.051150731 +0000 UTC m=+2708.461861347" watchObservedRunningTime="2026-07-24 16:21:20.052700166 +0000 UTC m=+2708.463410837"
Jul 24 16:21:31.018471 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:31.018435    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"
Jul 24 16:21:31.019061 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:31.018522    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"
Jul 24 16:21:38.372911 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.372868    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"]
Jul 24 16:21:38.378698 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.378643    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"
Jul 24 16:21:38.381583 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.381560    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-07e89c-dockercfg-lmm8r\""
Jul 24 16:21:38.381727 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.381560    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-07e89c\""
Jul 24 16:21:38.392962 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.392929    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"]
Jul 24 16:21:38.425736 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.425689    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"]
Jul 24 16:21:38.429639 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.429605    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"
Jul 24 16:21:38.441508 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.441473    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"]
Jul 24 16:21:38.467199 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.467123    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ace890b-3e64-4b06-ab44-eeaa22c634b9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l\" (UID: \"9ace890b-3e64-4b06-ab44-eeaa22c634b9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"
Jul 24 16:21:38.467450 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.467280    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63820e82-989b-4489-87df-107e7bc2bb18-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs\" (UID: \"63820e82-989b-4489-87df-107e7bc2bb18\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"
Jul 24 16:21:38.467450 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.467314    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcp9n\" (UniqueName: \"kubernetes.io/projected/63820e82-989b-4489-87df-107e7bc2bb18-kube-api-access-pcp9n\") pod \"payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs\" (UID: \"63820e82-989b-4489-87df-107e7bc2bb18\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"
Jul 24 16:21:38.568714 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.568672    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtvgl\" (UniqueName: \"kubernetes.io/projected/9ace890b-3e64-4b06-ab44-eeaa22c634b9-kube-api-access-wtvgl\") pod \"payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l\" (UID: \"9ace890b-3e64-4b06-ab44-eeaa22c634b9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"
Jul 24 16:21:38.568905 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.568728    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ace890b-3e64-4b06-ab44-eeaa22c634b9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l\" (UID: \"9ace890b-3e64-4b06-ab44-eeaa22c634b9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"
Jul 24 16:21:38.568905 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.568764    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63820e82-989b-4489-87df-107e7bc2bb18-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs\" (UID: \"63820e82-989b-4489-87df-107e7bc2bb18\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"
Jul 24 16:21:38.568905 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.568786    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcp9n\" (UniqueName: \"kubernetes.io/projected/63820e82-989b-4489-87df-107e7bc2bb18-kube-api-access-pcp9n\") pod \"payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs\" (UID: \"63820e82-989b-4489-87df-107e7bc2bb18\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"
Jul 24 16:21:38.569464 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.569445    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ace890b-3e64-4b06-ab44-eeaa22c634b9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l\" (UID: \"9ace890b-3e64-4b06-ab44-eeaa22c634b9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"
Jul 24 16:21:38.569464 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.569458    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63820e82-989b-4489-87df-107e7bc2bb18-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs\" (UID: \"63820e82-989b-4489-87df-107e7bc2bb18\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"
Jul 24 16:21:38.580421 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.580382    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcp9n\" (UniqueName: \"kubernetes.io/projected/63820e82-989b-4489-87df-107e7bc2bb18-kube-api-access-pcp9n\") pod \"payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs\" (UID: \"63820e82-989b-4489-87df-107e7bc2bb18\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"
Jul 24 16:21:38.669402 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.669306    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtvgl\" (UniqueName: \"kubernetes.io/projected/9ace890b-3e64-4b06-ab44-eeaa22c634b9-kube-api-access-wtvgl\") pod \"payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l\" (UID: \"9ace890b-3e64-4b06-ab44-eeaa22c634b9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"
Jul 24 16:21:38.679404 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.679369    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtvgl\" (UniqueName: \"kubernetes.io/projected/9ace890b-3e64-4b06-ab44-eeaa22c634b9-kube-api-access-wtvgl\") pod \"payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l\" (UID: \"9ace890b-3e64-4b06-ab44-eeaa22c634b9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"
Jul 24 16:21:38.692030 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.691993    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"
Jul 24 16:21:38.744047 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.744008    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"
Jul 24 16:21:38.866710 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.866563    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"]
Jul 24 16:21:38.870014 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:21:38.869975    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63820e82_989b_4489_87df_107e7bc2bb18.slice/crio-ee9e28d7996a6489c1dc50bdd5716c6a3086f8d45f21c695d9b9ac68de35b4f9 WatchSource:0}: Error finding container ee9e28d7996a6489c1dc50bdd5716c6a3086f8d45f21c695d9b9ac68de35b4f9: Status 404 returned error can't find the container with id ee9e28d7996a6489c1dc50bdd5716c6a3086f8d45f21c695d9b9ac68de35b4f9
Jul 24 16:21:38.924411 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:38.924379    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"]
Jul 24 16:21:38.926964 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:21:38.926929    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ace890b_3e64_4b06_ab44_eeaa22c634b9.slice/crio-2b5cd5076f8c1ccd5bb3ad6f5bd6b9a4b73cf4ff78247720885cda4a3eb9bdd6 WatchSource:0}: Error finding container 2b5cd5076f8c1ccd5bb3ad6f5bd6b9a4b73cf4ff78247720885cda4a3eb9bdd6: Status 404 returned error can't find the container with id 2b5cd5076f8c1ccd5bb3ad6f5bd6b9a4b73cf4ff78247720885cda4a3eb9bdd6
Jul 24 16:21:39.120672 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:39.120614    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l" event={"ID":"9ace890b-3e64-4b06-ab44-eeaa22c634b9","Type":"ContainerStarted","Data":"810b898d024976a4fafbe10baab27526925c67a1396b09f1222c6d3d14e76055"}
Jul 24 16:21:39.120873 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:39.120699    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l" event={"ID":"9ace890b-3e64-4b06-ab44-eeaa22c634b9","Type":"ContainerStarted","Data":"2b5cd5076f8c1ccd5bb3ad6f5bd6b9a4b73cf4ff78247720885cda4a3eb9bdd6"}
Jul 24 16:21:39.120873 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:39.120815    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"
Jul 24 16:21:39.122303 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:39.122276    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs" event={"ID":"63820e82-989b-4489-87df-107e7bc2bb18","Type":"ContainerStarted","Data":"b3e93c6bf9e3b41980ee4f5f318e32bfa01aa350d8cabc69bcb9e2b8630aad3d"}
Jul 24 16:21:39.122460 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:39.122319    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs" event={"ID":"63820e82-989b-4489-87df-107e7bc2bb18","Type":"ContainerStarted","Data":"ee9e28d7996a6489c1dc50bdd5716c6a3086f8d45f21c695d9b9ac68de35b4f9"}
Jul 24 16:21:39.122460 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:39.122350    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"
Jul 24 16:21:39.145370 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:39.145312    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l" podStartSLOduration=1.145290514 podStartE2EDuration="1.145290514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:21:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:21:39.145086774 +0000 UTC m=+2727.555797393" watchObservedRunningTime="2026-07-24 16:21:39.145290514 +0000 UTC m=+2727.556001133"
Jul 24 16:21:39.174534 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:39.174420    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs" podStartSLOduration=1.174400894 podStartE2EDuration="1.174400894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:21:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:21:39.172576456 +0000 UTC m=+2727.583287073" watchObservedRunningTime="2026-07-24 16:21:39.174400894 +0000 UTC m=+2727.585111511"
Jul 24 16:21:50.130270 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:50.130239    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"
Jul 24 16:21:50.130790 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:50.130323    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"
Jul 24 16:21:57.781057 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:57.781019    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-qf26j_fa01d160-d791-465e-9488-0ae89aca50b3/payload-processing/0.log"
Jul 24 16:21:57.904792 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:21:57.904748    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l_9ace890b-3e64-4b06-ab44-eeaa22c634b9/payload-processing/0.log"
Jul 24 16:22:08.706039 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.706004    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"]
Jul 24 16:22:08.710349 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.710325    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.714636 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.714554    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-5d0582\"/\"ipp-route-5d0582-kserve-self-signed-certs\""
Jul 24 16:22:08.714636 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.714580    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-5d0582\"/\"kube-root-ca.crt\""
Jul 24 16:22:08.714848 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.714714    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-5d0582\"/\"openshift-service-ca.crt\""
Jul 24 16:22:08.714848 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.714734    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-5d0582\"/\"default-dockercfg-sgrzd\""
Jul 24 16:22:08.718405 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.718375    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"]
Jul 24 16:22:08.848274 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.848236    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd5958ed-4ded-4bce-8881-0a1915de8e69-tls-certs\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.848477 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.848283    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-model-cache\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.848477 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.848354    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-dshm\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.848477 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.848379    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f7r4\" (UniqueName: \"kubernetes.io/projected/cd5958ed-4ded-4bce-8881-0a1915de8e69-kube-api-access-7f7r4\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.848622 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.848481    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-tmp-dir\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.848622 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.848522    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-home\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.949217 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.949179    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd5958ed-4ded-4bce-8881-0a1915de8e69-tls-certs\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.949217 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.949217    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-model-cache\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.949492 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.949249    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-dshm\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.949492 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.949271    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f7r4\" (UniqueName: \"kubernetes.io/projected/cd5958ed-4ded-4bce-8881-0a1915de8e69-kube-api-access-7f7r4\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.949492 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.949309    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-tmp-dir\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.949492 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.949336    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-home\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.949748 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.949702    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-model-cache\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.949805 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.949766    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-tmp-dir\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.949805 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.949793    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-home\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.951973 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.951949    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-dshm\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.952077 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.952059    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd5958ed-4ded-4bce-8881-0a1915de8e69-tls-certs\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:08.957766 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:08.957643    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f7r4\" (UniqueName: \"kubernetes.io/projected/cd5958ed-4ded-4bce-8881-0a1915de8e69-kube-api-access-7f7r4\") pod \"ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") " pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:09.024307 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:09.024271    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:09.172991 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:09.172964    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"]
Jul 24 16:22:09.175146 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:22:09.175115    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd5958ed_4ded_4bce_8881_0a1915de8e69.slice/crio-d18719a7486d5bfac17fe84654b21ef363f6888cc867288e1cdf0a860f362d45 WatchSource:0}: Error finding container d18719a7486d5bfac17fe84654b21ef363f6888cc867288e1cdf0a860f362d45: Status 404 returned error can't find the container with id d18719a7486d5bfac17fe84654b21ef363f6888cc867288e1cdf0a860f362d45
Jul 24 16:22:09.281251 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:09.281207    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq" event={"ID":"cd5958ed-4ded-4bce-8881-0a1915de8e69","Type":"ContainerStarted","Data":"a7bd92311b9979074323d4d5196748ea996d93ee75c44e537d5c27f7f5619739"}
Jul 24 16:22:09.281443 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:09.281263    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq" event={"ID":"cd5958ed-4ded-4bce-8881-0a1915de8e69","Type":"ContainerStarted","Data":"d18719a7486d5bfac17fe84654b21ef363f6888cc867288e1cdf0a860f362d45"}
Jul 24 16:22:09.299039 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:09.298982    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq" podStartSLOduration=1.298963024 podStartE2EDuration="1.298963024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:22:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:22:09.296875035 +0000 UTC m=+2757.707585654" watchObservedRunningTime="2026-07-24 16:22:09.298963024 +0000 UTC m=+2757.709673663"
Jul 24 16:22:19.024825 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:19.024789    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:19.025255 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:19.024840    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:19.038027 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:19.037998    2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:19.343310 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:19.343223    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:22:39.697863 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:39.697766    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4_bc9a6c50-a23e-434e-8893-36d6122e8012/payload-processing/0.log"
Jul 24 16:22:39.824897 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:39.824863    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-qf26j_fa01d160-d791-465e-9488-0ae89aca50b3/payload-processing/0.log"
Jul 24 16:22:39.961448 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:39.961420    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l_9ace890b-3e64-4b06-ab44-eeaa22c634b9/payload-processing/0.log"
Jul 24 16:22:46.935859 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:46.935820    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"]
Jul 24 16:22:46.939927 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:46.939879    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"
Jul 24 16:22:46.942433 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:46.942385    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-ebf027\""
Jul 24 16:22:46.948320 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:46.948278    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"]
Jul 24 16:22:46.989044 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:46.988998    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"]
Jul 24 16:22:46.992862 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:46.992837    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"
Jul 24 16:22:46.995203 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:46.995163    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-ebf027-dockercfg-95jmg\""
Jul 24 16:22:47.003522 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.003486    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"]
Jul 24 16:22:47.017295 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.017257    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8cmb\" (UniqueName: \"kubernetes.io/projected/56b87d2b-b9e3-4a29-9774-86a54af78dc3-kube-api-access-g8cmb\") pod \"payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn\" (UID: \"56b87d2b-b9e3-4a29-9774-86a54af78dc3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"
Jul 24 16:22:47.017295 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.017303    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b87d2b-b9e3-4a29-9774-86a54af78dc3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn\" (UID: \"56b87d2b-b9e3-4a29-9774-86a54af78dc3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"
Jul 24 16:22:47.118624 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.118573    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2p5p\" (UniqueName: \"kubernetes.io/projected/059819f1-faff-423d-b800-6d56f0ff38e5-kube-api-access-v2p5p\") pod \"payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m\" (UID: \"059819f1-faff-423d-b800-6d56f0ff38e5\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"
Jul 24 16:22:47.118862 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.118646    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/059819f1-faff-423d-b800-6d56f0ff38e5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m\" (UID: \"059819f1-faff-423d-b800-6d56f0ff38e5\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"
Jul 24 16:22:47.118862 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.118723    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8cmb\" (UniqueName: \"kubernetes.io/projected/56b87d2b-b9e3-4a29-9774-86a54af78dc3-kube-api-access-g8cmb\") pod \"payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn\" (UID: \"56b87d2b-b9e3-4a29-9774-86a54af78dc3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"
Jul 24 16:22:47.118862 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.118762    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b87d2b-b9e3-4a29-9774-86a54af78dc3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn\" (UID: \"56b87d2b-b9e3-4a29-9774-86a54af78dc3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"
Jul 24 16:22:47.119833 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.119805    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b87d2b-b9e3-4a29-9774-86a54af78dc3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn\" (UID: \"56b87d2b-b9e3-4a29-9774-86a54af78dc3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"
Jul 24 16:22:47.128161 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.128117    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8cmb\" (UniqueName: \"kubernetes.io/projected/56b87d2b-b9e3-4a29-9774-86a54af78dc3-kube-api-access-g8cmb\") pod \"payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn\" (UID: \"56b87d2b-b9e3-4a29-9774-86a54af78dc3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"
Jul 24 16:22:47.219310 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.219273    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2p5p\" (UniqueName: \"kubernetes.io/projected/059819f1-faff-423d-b800-6d56f0ff38e5-kube-api-access-v2p5p\") pod \"payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m\" (UID: \"059819f1-faff-423d-b800-6d56f0ff38e5\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"
Jul 24 16:22:47.219538 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.219320    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/059819f1-faff-423d-b800-6d56f0ff38e5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m\" (UID: \"059819f1-faff-423d-b800-6d56f0ff38e5\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"
Jul 24 16:22:47.220122 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.220098    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/059819f1-faff-423d-b800-6d56f0ff38e5-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m\" (UID: \"059819f1-faff-423d-b800-6d56f0ff38e5\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"
Jul 24 16:22:47.227425 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.227392    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2p5p\" (UniqueName: \"kubernetes.io/projected/059819f1-faff-423d-b800-6d56f0ff38e5-kube-api-access-v2p5p\") pod \"payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m\" (UID: \"059819f1-faff-423d-b800-6d56f0ff38e5\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"
Jul 24 16:22:47.255318 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.255274    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"
Jul 24 16:22:47.305952 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.304893    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"
Jul 24 16:22:47.450797 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.450748    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"]
Jul 24 16:22:47.456893 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:22:47.456856    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56b87d2b_b9e3_4a29_9774_86a54af78dc3.slice/crio-109c41690c4124f255675469e7dfeb025e8a4949e6bbb702f6012734ae7c9c09 WatchSource:0}: Error finding container 109c41690c4124f255675469e7dfeb025e8a4949e6bbb702f6012734ae7c9c09: Status 404 returned error can't find the container with id 109c41690c4124f255675469e7dfeb025e8a4949e6bbb702f6012734ae7c9c09
Jul 24 16:22:47.493068 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.493030    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn" event={"ID":"56b87d2b-b9e3-4a29-9774-86a54af78dc3","Type":"ContainerStarted","Data":"109c41690c4124f255675469e7dfeb025e8a4949e6bbb702f6012734ae7c9c09"}
Jul 24 16:22:47.517899 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:47.517855    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"]
Jul 24 16:22:47.538559 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:22:47.538508    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod059819f1_faff_423d_b800_6d56f0ff38e5.slice/crio-5a92b5c515db61c46abb587fbaca2aac478fdabce861afeb5abfcc775b5c1dcb WatchSource:0}: Error finding container 5a92b5c515db61c46abb587fbaca2aac478fdabce861afeb5abfcc775b5c1dcb: Status 404 returned error can't find the container with id 5a92b5c515db61c46abb587fbaca2aac478fdabce861afeb5abfcc775b5c1dcb
Jul 24 16:22:48.502718 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:48.502680    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m" event={"ID":"059819f1-faff-423d-b800-6d56f0ff38e5","Type":"ContainerStarted","Data":"90d26433f6c82afd29d0ae193a40987d893152ea6912183faeb721fcc6be19bf"}
Jul 24 16:22:48.503254 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:48.502755    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"
Jul 24 16:22:48.503254 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:48.502779    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m" event={"ID":"059819f1-faff-423d-b800-6d56f0ff38e5","Type":"ContainerStarted","Data":"5a92b5c515db61c46abb587fbaca2aac478fdabce861afeb5abfcc775b5c1dcb"}
Jul 24 16:22:48.504458 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:48.504428    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn" event={"ID":"56b87d2b-b9e3-4a29-9774-86a54af78dc3","Type":"ContainerStarted","Data":"de735b54e9f4c805d58ba720d8356bed788bbfbc7e3f0f58f917810f77d7df65"}
Jul 24 16:22:48.504601 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:48.504512    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"
Jul 24 16:22:48.522505 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:48.522419    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m" podStartSLOduration=2.522400196 podStartE2EDuration="2.522400196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:22:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:22:48.521368176 +0000 UTC m=+2796.932078792" watchObservedRunningTime="2026-07-24 16:22:48.522400196 +0000 UTC m=+2796.933110870"
Jul 24 16:22:48.538091 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:48.538025    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn" podStartSLOduration=2.538001589 podStartE2EDuration="2.538001589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:22:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:22:48.536258504 +0000 UTC m=+2796.946969122" watchObservedRunningTime="2026-07-24 16:22:48.538001589 +0000 UTC m=+2796.948712209"
Jul 24 16:22:59.511951 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:59.511889    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn" podUID="56b87d2b-b9e3-4a29-9774-86a54af78dc3" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.85:9004: connect: connection refused"
Jul 24 16:22:59.512422 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:22:59.512400    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"
Jul 24 16:23:09.510889 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:09.510843    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"
Jul 24 16:23:26.982169 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:26.982115    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"]
Jul 24 16:23:26.983007 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:26.982471    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m" podUID="059819f1-faff-423d-b800-6d56f0ff38e5" containerName="payload-processing" containerID="cri-o://90d26433f6c82afd29d0ae193a40987d893152ea6912183faeb721fcc6be19bf" gracePeriod=30
Jul 24 16:23:26.988199 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:26.988166    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"]
Jul 24 16:23:26.988436 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:26.988389    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn" podUID="56b87d2b-b9e3-4a29-9774-86a54af78dc3" containerName="payload-pre-processing" containerID="cri-o://de735b54e9f4c805d58ba720d8356bed788bbfbc7e3f0f58f917810f77d7df65" gracePeriod=30
Jul 24 16:23:27.271371 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.271337    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"
Jul 24 16:23:27.299103 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.299073    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"
Jul 24 16:23:27.417498 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.417452    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2p5p\" (UniqueName: \"kubernetes.io/projected/059819f1-faff-423d-b800-6d56f0ff38e5-kube-api-access-v2p5p\") pod \"059819f1-faff-423d-b800-6d56f0ff38e5\" (UID: \"059819f1-faff-423d-b800-6d56f0ff38e5\") "
Jul 24 16:23:27.417722 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.417508    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/059819f1-faff-423d-b800-6d56f0ff38e5-plugins-config-volume\") pod \"059819f1-faff-423d-b800-6d56f0ff38e5\" (UID: \"059819f1-faff-423d-b800-6d56f0ff38e5\") "
Jul 24 16:23:27.417722 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.417571    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8cmb\" (UniqueName: \"kubernetes.io/projected/56b87d2b-b9e3-4a29-9774-86a54af78dc3-kube-api-access-g8cmb\") pod \"56b87d2b-b9e3-4a29-9774-86a54af78dc3\" (UID: \"56b87d2b-b9e3-4a29-9774-86a54af78dc3\") "
Jul 24 16:23:27.417722 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.417620    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b87d2b-b9e3-4a29-9774-86a54af78dc3-plugins-config-volume\") pod \"56b87d2b-b9e3-4a29-9774-86a54af78dc3\" (UID: \"56b87d2b-b9e3-4a29-9774-86a54af78dc3\") "
Jul 24 16:23:27.418154 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.418122    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/059819f1-faff-423d-b800-6d56f0ff38e5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "059819f1-faff-423d-b800-6d56f0ff38e5" (UID: "059819f1-faff-423d-b800-6d56f0ff38e5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:23:27.418154 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.418141    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56b87d2b-b9e3-4a29-9774-86a54af78dc3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "56b87d2b-b9e3-4a29-9774-86a54af78dc3" (UID: "56b87d2b-b9e3-4a29-9774-86a54af78dc3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:23:27.420059 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.420033    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/059819f1-faff-423d-b800-6d56f0ff38e5-kube-api-access-v2p5p" (OuterVolumeSpecName: "kube-api-access-v2p5p") pod "059819f1-faff-423d-b800-6d56f0ff38e5" (UID: "059819f1-faff-423d-b800-6d56f0ff38e5"). InnerVolumeSpecName "kube-api-access-v2p5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:23:27.420236 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.420212    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56b87d2b-b9e3-4a29-9774-86a54af78dc3-kube-api-access-g8cmb" (OuterVolumeSpecName: "kube-api-access-g8cmb") pod "56b87d2b-b9e3-4a29-9774-86a54af78dc3" (UID: "56b87d2b-b9e3-4a29-9774-86a54af78dc3"). InnerVolumeSpecName "kube-api-access-g8cmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:23:27.518974 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.518870    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8cmb\" (UniqueName: \"kubernetes.io/projected/56b87d2b-b9e3-4a29-9774-86a54af78dc3-kube-api-access-g8cmb\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:23:27.518974 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.518903    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56b87d2b-b9e3-4a29-9774-86a54af78dc3-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:23:27.518974 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.518918    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2p5p\" (UniqueName: \"kubernetes.io/projected/059819f1-faff-423d-b800-6d56f0ff38e5-kube-api-access-v2p5p\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:23:27.518974 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.518929    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/059819f1-faff-423d-b800-6d56f0ff38e5-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:23:27.728521 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.728491    2561 generic.go:356] "Generic (PLEG): container finished" podID="56b87d2b-b9e3-4a29-9774-86a54af78dc3" containerID="de735b54e9f4c805d58ba720d8356bed788bbfbc7e3f0f58f917810f77d7df65" exitCode=0
Jul 24 16:23:27.728758 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.728559    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"
Jul 24 16:23:27.728758 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.728574    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn" event={"ID":"56b87d2b-b9e3-4a29-9774-86a54af78dc3","Type":"ContainerDied","Data":"de735b54e9f4c805d58ba720d8356bed788bbfbc7e3f0f58f917810f77d7df65"}
Jul 24 16:23:27.728758 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.728643    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn" event={"ID":"56b87d2b-b9e3-4a29-9774-86a54af78dc3","Type":"ContainerDied","Data":"109c41690c4124f255675469e7dfeb025e8a4949e6bbb702f6012734ae7c9c09"}
Jul 24 16:23:27.728758 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.728691    2561 scope.go:117] "RemoveContainer" containerID="de735b54e9f4c805d58ba720d8356bed788bbfbc7e3f0f58f917810f77d7df65"
Jul 24 16:23:27.730181 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.730151    2561 generic.go:356] "Generic (PLEG): container finished" podID="059819f1-faff-423d-b800-6d56f0ff38e5" containerID="90d26433f6c82afd29d0ae193a40987d893152ea6912183faeb721fcc6be19bf" exitCode=0
Jul 24 16:23:27.730297 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.730189    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m" event={"ID":"059819f1-faff-423d-b800-6d56f0ff38e5","Type":"ContainerDied","Data":"90d26433f6c82afd29d0ae193a40987d893152ea6912183faeb721fcc6be19bf"}
Jul 24 16:23:27.730297 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.730207    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"
Jul 24 16:23:27.730297 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.730233    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m" event={"ID":"059819f1-faff-423d-b800-6d56f0ff38e5","Type":"ContainerDied","Data":"5a92b5c515db61c46abb587fbaca2aac478fdabce861afeb5abfcc775b5c1dcb"}
Jul 24 16:23:27.748370 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.748347    2561 scope.go:117] "RemoveContainer" containerID="de735b54e9f4c805d58ba720d8356bed788bbfbc7e3f0f58f917810f77d7df65"
Jul 24 16:23:27.748835 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:23:27.748809    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de735b54e9f4c805d58ba720d8356bed788bbfbc7e3f0f58f917810f77d7df65\": container with ID starting with de735b54e9f4c805d58ba720d8356bed788bbfbc7e3f0f58f917810f77d7df65 not found: ID does not exist" containerID="de735b54e9f4c805d58ba720d8356bed788bbfbc7e3f0f58f917810f77d7df65"
Jul 24 16:23:27.748946 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.748843    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de735b54e9f4c805d58ba720d8356bed788bbfbc7e3f0f58f917810f77d7df65"} err="failed to get container status \"de735b54e9f4c805d58ba720d8356bed788bbfbc7e3f0f58f917810f77d7df65\": rpc error: code = NotFound desc = could not find container \"de735b54e9f4c805d58ba720d8356bed788bbfbc7e3f0f58f917810f77d7df65\": container with ID starting with de735b54e9f4c805d58ba720d8356bed788bbfbc7e3f0f58f917810f77d7df65 not found: ID does not exist"
Jul 24 16:23:27.748946 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.748862    2561 scope.go:117] "RemoveContainer" containerID="90d26433f6c82afd29d0ae193a40987d893152ea6912183faeb721fcc6be19bf"
Jul 24 16:23:27.758169 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.758130    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"]
Jul 24 16:23:27.759377 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.759296    2561 scope.go:117] "RemoveContainer" containerID="90d26433f6c82afd29d0ae193a40987d893152ea6912183faeb721fcc6be19bf"
Jul 24 16:23:27.760005 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:23:27.759968    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90d26433f6c82afd29d0ae193a40987d893152ea6912183faeb721fcc6be19bf\": container with ID starting with 90d26433f6c82afd29d0ae193a40987d893152ea6912183faeb721fcc6be19bf not found: ID does not exist" containerID="90d26433f6c82afd29d0ae193a40987d893152ea6912183faeb721fcc6be19bf"
Jul 24 16:23:27.760136 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.760018    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90d26433f6c82afd29d0ae193a40987d893152ea6912183faeb721fcc6be19bf"} err="failed to get container status \"90d26433f6c82afd29d0ae193a40987d893152ea6912183faeb721fcc6be19bf\": rpc error: code = NotFound desc = could not find container \"90d26433f6c82afd29d0ae193a40987d893152ea6912183faeb721fcc6be19bf\": container with ID starting with 90d26433f6c82afd29d0ae193a40987d893152ea6912183faeb721fcc6be19bf not found: ID does not exist"
Jul 24 16:23:27.761873 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.761847    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ebf027-578cb5496d-nvr7m"]
Jul 24 16:23:27.772981 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.772876    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"]
Jul 24 16:23:27.775139 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:27.775107    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ebf027-6d89c4d49b-mc8gn"]
Jul 24 16:23:28.109952 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:28.109849    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="059819f1-faff-423d-b800-6d56f0ff38e5" path="/var/lib/kubelet/pods/059819f1-faff-423d-b800-6d56f0ff38e5/volumes"
Jul 24 16:23:28.110393 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:28.110267    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56b87d2b-b9e3-4a29-9774-86a54af78dc3" path="/var/lib/kubelet/pods/56b87d2b-b9e3-4a29-9774-86a54af78dc3/volumes"
Jul 24 16:23:59.776816 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:59.776779    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"]
Jul 24 16:23:59.777380 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:59.777017    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4" podUID="bc9a6c50-a23e-434e-8893-36d6122e8012" containerName="payload-processing" containerID="cri-o://1116ddd4db5e7ef30b3b4bd7f16183d34f4d7b24dc4ee202268417497d06ca1d" gracePeriod=30
Jul 24 16:23:59.783605 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:59.783572    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"]
Jul 24 16:23:59.783846 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:59.783815    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b" podUID="72757fa1-7cc2-417e-aab3-cd0d0ce37c4b" containerName="payload-pre-processing" containerID="cri-o://7013fae434416b612b2617ab0edb1aa3658f6c3c6061e5d4af45a4c4589c270a" gracePeriod=30
Jul 24 16:23:59.916005 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:59.915964    2561 generic.go:356] "Generic (PLEG): container finished" podID="72757fa1-7cc2-417e-aab3-cd0d0ce37c4b" containerID="7013fae434416b612b2617ab0edb1aa3658f6c3c6061e5d4af45a4c4589c270a" exitCode=0
Jul 24 16:23:59.916193 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:59.916036    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b" event={"ID":"72757fa1-7cc2-417e-aab3-cd0d0ce37c4b","Type":"ContainerDied","Data":"7013fae434416b612b2617ab0edb1aa3658f6c3c6061e5d4af45a4c4589c270a"}
Jul 24 16:23:59.917871 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:59.917820    2561 generic.go:356] "Generic (PLEG): container finished" podID="bc9a6c50-a23e-434e-8893-36d6122e8012" containerID="1116ddd4db5e7ef30b3b4bd7f16183d34f4d7b24dc4ee202268417497d06ca1d" exitCode=0
Jul 24 16:23:59.917871 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:23:59.917874    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4" event={"ID":"bc9a6c50-a23e-434e-8893-36d6122e8012","Type":"ContainerDied","Data":"1116ddd4db5e7ef30b3b4bd7f16183d34f4d7b24dc4ee202268417497d06ca1d"}
Jul 24 16:24:00.050701 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.050645    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"
Jul 24 16:24:00.082497 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.082469    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"
Jul 24 16:24:00.135909 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.135872    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72757fa1-7cc2-417e-aab3-cd0d0ce37c4b-plugins-config-volume\") pod \"72757fa1-7cc2-417e-aab3-cd0d0ce37c4b\" (UID: \"72757fa1-7cc2-417e-aab3-cd0d0ce37c4b\") "
Jul 24 16:24:00.136122 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.135922    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc9a6c50-a23e-434e-8893-36d6122e8012-plugins-config-volume\") pod \"bc9a6c50-a23e-434e-8893-36d6122e8012\" (UID: \"bc9a6c50-a23e-434e-8893-36d6122e8012\") "
Jul 24 16:24:00.136122 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.136047    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnsxx\" (UniqueName: \"kubernetes.io/projected/72757fa1-7cc2-417e-aab3-cd0d0ce37c4b-kube-api-access-fnsxx\") pod \"72757fa1-7cc2-417e-aab3-cd0d0ce37c4b\" (UID: \"72757fa1-7cc2-417e-aab3-cd0d0ce37c4b\") "
Jul 24 16:24:00.136339 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.136310    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72757fa1-7cc2-417e-aab3-cd0d0ce37c4b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "72757fa1-7cc2-417e-aab3-cd0d0ce37c4b" (UID: "72757fa1-7cc2-417e-aab3-cd0d0ce37c4b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:24:00.136474 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.136419    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc9a6c50-a23e-434e-8893-36d6122e8012-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bc9a6c50-a23e-434e-8893-36d6122e8012" (UID: "bc9a6c50-a23e-434e-8893-36d6122e8012"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:24:00.136557 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.136523    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpqf2\" (UniqueName: \"kubernetes.io/projected/bc9a6c50-a23e-434e-8893-36d6122e8012-kube-api-access-lpqf2\") pod \"bc9a6c50-a23e-434e-8893-36d6122e8012\" (UID: \"bc9a6c50-a23e-434e-8893-36d6122e8012\") "
Jul 24 16:24:00.136893 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.136871    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72757fa1-7cc2-417e-aab3-cd0d0ce37c4b-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:24:00.136995 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.136894    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc9a6c50-a23e-434e-8893-36d6122e8012-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:24:00.138942 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.138912    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72757fa1-7cc2-417e-aab3-cd0d0ce37c4b-kube-api-access-fnsxx" (OuterVolumeSpecName: "kube-api-access-fnsxx") pod "72757fa1-7cc2-417e-aab3-cd0d0ce37c4b" (UID: "72757fa1-7cc2-417e-aab3-cd0d0ce37c4b"). InnerVolumeSpecName "kube-api-access-fnsxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:24:00.139037 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.138934    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc9a6c50-a23e-434e-8893-36d6122e8012-kube-api-access-lpqf2" (OuterVolumeSpecName: "kube-api-access-lpqf2") pod "bc9a6c50-a23e-434e-8893-36d6122e8012" (UID: "bc9a6c50-a23e-434e-8893-36d6122e8012"). InnerVolumeSpecName "kube-api-access-lpqf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:24:00.237944 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.237901    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpqf2\" (UniqueName: \"kubernetes.io/projected/bc9a6c50-a23e-434e-8893-36d6122e8012-kube-api-access-lpqf2\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:24:00.237944 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.237937    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnsxx\" (UniqueName: \"kubernetes.io/projected/72757fa1-7cc2-417e-aab3-cd0d0ce37c4b-kube-api-access-fnsxx\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:24:00.925118 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.925082    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"
Jul 24 16:24:00.925118 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.925105    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b" event={"ID":"72757fa1-7cc2-417e-aab3-cd0d0ce37c4b","Type":"ContainerDied","Data":"48951ee380d9b2adc0542e0bc9ba07b27cab3beb80a44be7caf865b01ed90595"}
Jul 24 16:24:00.925696 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.925183    2561 scope.go:117] "RemoveContainer" containerID="7013fae434416b612b2617ab0edb1aa3658f6c3c6061e5d4af45a4c4589c270a"
Jul 24 16:24:00.926766 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.926598    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"
Jul 24 16:24:00.926766 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.926635    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4" event={"ID":"bc9a6c50-a23e-434e-8893-36d6122e8012","Type":"ContainerDied","Data":"e63fc8d2c569388471a444037e7608e183e21e7d129a3aba32489958a7d3f23b"}
Jul 24 16:24:00.936503 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.936476    2561 scope.go:117] "RemoveContainer" containerID="1116ddd4db5e7ef30b3b4bd7f16183d34f4d7b24dc4ee202268417497d06ca1d"
Jul 24 16:24:00.960404 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.960365    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"]
Jul 24 16:24:00.963543 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.963502    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-5d0582-6bcc95df74-xp6d4"]
Jul 24 16:24:00.984837 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.984801    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"]
Jul 24 16:24:00.988276 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:00.988238    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-5d0582-6fdd79978d-ln87b"]
Jul 24 16:24:02.108066 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:02.108025    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72757fa1-7cc2-417e-aab3-cd0d0ce37c4b" path="/var/lib/kubelet/pods/72757fa1-7cc2-417e-aab3-cd0d0ce37c4b/volumes"
Jul 24 16:24:02.108465 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:02.108378    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc9a6c50-a23e-434e-8893-36d6122e8012" path="/var/lib/kubelet/pods/bc9a6c50-a23e-434e-8893-36d6122e8012/volumes"
Jul 24 16:24:05.559311 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:05.559208    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"]
Jul 24 16:24:20.579275 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:20.579233    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq" podUID="cd5958ed-4ded-4bce-8881-0a1915de8e69" containerName="main" containerID="cri-o://a7bd92311b9979074323d4d5196748ea996d93ee75c44e537d5c27f7f5619739" gracePeriod=45
Jul 24 16:24:20.854867 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:20.854836    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:24:20.928450 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:20.928419    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-dshm\") pod \"cd5958ed-4ded-4bce-8881-0a1915de8e69\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") "
Jul 24 16:24:20.928450 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:20.928454    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-home\") pod \"cd5958ed-4ded-4bce-8881-0a1915de8e69\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") "
Jul 24 16:24:20.928647 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:20.928489    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-model-cache\") pod \"cd5958ed-4ded-4bce-8881-0a1915de8e69\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") "
Jul 24 16:24:20.928647 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:20.928537    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd5958ed-4ded-4bce-8881-0a1915de8e69-tls-certs\") pod \"cd5958ed-4ded-4bce-8881-0a1915de8e69\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") "
Jul 24 16:24:20.928647 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:20.928602    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-tmp-dir\") pod \"cd5958ed-4ded-4bce-8881-0a1915de8e69\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") "
Jul 24 16:24:20.928840 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:20.928653    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7f7r4\" (UniqueName: \"kubernetes.io/projected/cd5958ed-4ded-4bce-8881-0a1915de8e69-kube-api-access-7f7r4\") pod \"cd5958ed-4ded-4bce-8881-0a1915de8e69\" (UID: \"cd5958ed-4ded-4bce-8881-0a1915de8e69\") "
Jul 24 16:24:20.928840 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:20.928727    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-home" (OuterVolumeSpecName: "home") pod "cd5958ed-4ded-4bce-8881-0a1915de8e69" (UID: "cd5958ed-4ded-4bce-8881-0a1915de8e69"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:24:20.928840 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:20.928803    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-model-cache" (OuterVolumeSpecName: "model-cache") pod "cd5958ed-4ded-4bce-8881-0a1915de8e69" (UID: "cd5958ed-4ded-4bce-8881-0a1915de8e69"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:24:20.929019 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:20.928918    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cd5958ed-4ded-4bce-8881-0a1915de8e69" (UID: "cd5958ed-4ded-4bce-8881-0a1915de8e69"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:24:20.929019 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:20.928972    2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-home\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:24:20.929019 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:20.928991    2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-model-cache\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:24:20.930953 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:20.930920    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd5958ed-4ded-4bce-8881-0a1915de8e69-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cd5958ed-4ded-4bce-8881-0a1915de8e69" (UID: "cd5958ed-4ded-4bce-8881-0a1915de8e69"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:24:20.931089 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:20.930977    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-dshm" (OuterVolumeSpecName: "dshm") pod "cd5958ed-4ded-4bce-8881-0a1915de8e69" (UID: "cd5958ed-4ded-4bce-8881-0a1915de8e69"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:24:20.931089 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:20.931025    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd5958ed-4ded-4bce-8881-0a1915de8e69-kube-api-access-7f7r4" (OuterVolumeSpecName: "kube-api-access-7f7r4") pod "cd5958ed-4ded-4bce-8881-0a1915de8e69" (UID: "cd5958ed-4ded-4bce-8881-0a1915de8e69"). InnerVolumeSpecName "kube-api-access-7f7r4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:24:21.030320 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:21.030281    2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd5958ed-4ded-4bce-8881-0a1915de8e69-tls-certs\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:24:21.030320 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:21.030312    2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-tmp-dir\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:24:21.030320 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:21.030322    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f7r4\" (UniqueName: \"kubernetes.io/projected/cd5958ed-4ded-4bce-8881-0a1915de8e69-kube-api-access-7f7r4\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:24:21.030320 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:21.030332    2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd5958ed-4ded-4bce-8881-0a1915de8e69-dshm\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:24:21.037958 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:21.037926    2561 generic.go:356] "Generic (PLEG): container finished" podID="cd5958ed-4ded-4bce-8881-0a1915de8e69" containerID="a7bd92311b9979074323d4d5196748ea996d93ee75c44e537d5c27f7f5619739" exitCode=0
Jul 24 16:24:21.038168 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:21.037978    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"
Jul 24 16:24:21.038168 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:21.038017    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq" event={"ID":"cd5958ed-4ded-4bce-8881-0a1915de8e69","Type":"ContainerDied","Data":"a7bd92311b9979074323d4d5196748ea996d93ee75c44e537d5c27f7f5619739"}
Jul 24 16:24:21.038168 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:21.038074    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq" event={"ID":"cd5958ed-4ded-4bce-8881-0a1915de8e69","Type":"ContainerDied","Data":"d18719a7486d5bfac17fe84654b21ef363f6888cc867288e1cdf0a860f362d45"}
Jul 24 16:24:21.038168 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:21.038100    2561 scope.go:117] "RemoveContainer" containerID="a7bd92311b9979074323d4d5196748ea996d93ee75c44e537d5c27f7f5619739"
Jul 24 16:24:21.049633 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:21.049608    2561 scope.go:117] "RemoveContainer" containerID="a7bd92311b9979074323d4d5196748ea996d93ee75c44e537d5c27f7f5619739"
Jul 24 16:24:21.050032 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:24:21.050006    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7bd92311b9979074323d4d5196748ea996d93ee75c44e537d5c27f7f5619739\": container with ID starting with a7bd92311b9979074323d4d5196748ea996d93ee75c44e537d5c27f7f5619739 not found: ID does not exist" containerID="a7bd92311b9979074323d4d5196748ea996d93ee75c44e537d5c27f7f5619739"
Jul 24 16:24:21.050118 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:21.050046    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7bd92311b9979074323d4d5196748ea996d93ee75c44e537d5c27f7f5619739"} err="failed to get container status \"a7bd92311b9979074323d4d5196748ea996d93ee75c44e537d5c27f7f5619739\": rpc error: code = NotFound desc = could not find container \"a7bd92311b9979074323d4d5196748ea996d93ee75c44e537d5c27f7f5619739\": container with ID starting with a7bd92311b9979074323d4d5196748ea996d93ee75c44e537d5c27f7f5619739 not found: ID does not exist"
Jul 24 16:24:21.062099 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:21.062057    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"]
Jul 24 16:24:21.067314 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:21.067274    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-5d0582/ipp-route-5d0582-kserve-8f4d4fdcf-4j8gq"]
Jul 24 16:24:22.107067 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:22.107032    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd5958ed-4ded-4bce-8881-0a1915de8e69" path="/var/lib/kubelet/pods/cd5958ed-4ded-4bce-8881-0a1915de8e69/volumes"
Jul 24 16:24:56.672126 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.672089    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz"]
Jul 24 16:24:56.672591 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.672478    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72757fa1-7cc2-417e-aab3-cd0d0ce37c4b" containerName="payload-pre-processing"
Jul 24 16:24:56.672591 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.672490    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="72757fa1-7cc2-417e-aab3-cd0d0ce37c4b" containerName="payload-pre-processing"
Jul 24 16:24:56.672591 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.672497    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd5958ed-4ded-4bce-8881-0a1915de8e69" containerName="main"
Jul 24 16:24:56.672591 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.672503    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd5958ed-4ded-4bce-8881-0a1915de8e69" containerName="main"
Jul 24 16:24:56.672591 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.672508    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56b87d2b-b9e3-4a29-9774-86a54af78dc3" containerName="payload-pre-processing"
Jul 24 16:24:56.672591 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.672515    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="56b87d2b-b9e3-4a29-9774-86a54af78dc3" containerName="payload-pre-processing"
Jul 24 16:24:56.672591 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.672526    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="059819f1-faff-423d-b800-6d56f0ff38e5" containerName="payload-processing"
Jul 24 16:24:56.672591 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.672531    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="059819f1-faff-423d-b800-6d56f0ff38e5" containerName="payload-processing"
Jul 24 16:24:56.672591 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.672541    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc9a6c50-a23e-434e-8893-36d6122e8012" containerName="payload-processing"
Jul 24 16:24:56.672591 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.672546    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc9a6c50-a23e-434e-8893-36d6122e8012" containerName="payload-processing"
Jul 24 16:24:56.672944 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.672617    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc9a6c50-a23e-434e-8893-36d6122e8012" containerName="payload-processing"
Jul 24 16:24:56.672944 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.672626    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd5958ed-4ded-4bce-8881-0a1915de8e69" containerName="main"
Jul 24 16:24:56.672944 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.672634    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="72757fa1-7cc2-417e-aab3-cd0d0ce37c4b" containerName="payload-pre-processing"
Jul 24 16:24:56.672944 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.672640    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="56b87d2b-b9e3-4a29-9774-86a54af78dc3" containerName="payload-pre-processing"
Jul 24 16:24:56.672944 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.672646    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="059819f1-faff-423d-b800-6d56f0ff38e5" containerName="payload-processing"
Jul 24 16:24:56.675883 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.675864    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz"
Jul 24 16:24:56.678413 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.678393    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xnkqn\""
Jul 24 16:24:56.684736 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.684708    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz"]
Jul 24 16:24:56.747962 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.747921    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea9e632-630e-4bc1-8e23-c788b66ac17d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz\" (UID: \"6ea9e632-630e-4bc1-8e23-c788b66ac17d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz"
Jul 24 16:24:56.848946 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.848901    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea9e632-630e-4bc1-8e23-c788b66ac17d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz\" (UID: \"6ea9e632-630e-4bc1-8e23-c788b66ac17d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz"
Jul 24 16:24:56.849525 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.849498    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea9e632-630e-4bc1-8e23-c788b66ac17d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz\" (UID: \"6ea9e632-630e-4bc1-8e23-c788b66ac17d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz"
Jul 24 16:24:56.987059 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:56.987023    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz"
Jul 24 16:24:57.130399 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:57.130314    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz"]
Jul 24 16:24:57.132967 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:24:57.132941    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ea9e632_630e_4bc1_8e23_c788b66ac17d.slice/crio-4dadb31f54445a56f1fe9dd173622c268cc2b235d7903f41e9298eaaeabebfe3 WatchSource:0}: Error finding container 4dadb31f54445a56f1fe9dd173622c268cc2b235d7903f41e9298eaaeabebfe3: Status 404 returned error can't find the container with id 4dadb31f54445a56f1fe9dd173622c268cc2b235d7903f41e9298eaaeabebfe3
Jul 24 16:24:57.221133 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:24:57.221099    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz" event={"ID":"6ea9e632-630e-4bc1-8e23-c788b66ac17d","Type":"ContainerStarted","Data":"4dadb31f54445a56f1fe9dd173622c268cc2b235d7903f41e9298eaaeabebfe3"}
Jul 24 16:25:00.241476 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:00.241442    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz" event={"ID":"6ea9e632-630e-4bc1-8e23-c788b66ac17d","Type":"ContainerStarted","Data":"128e1ef7217893d284f1cea9ab2145680251068d35a6276ac3c9659b174d6ef1"}
Jul 24 16:25:00.257779 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:00.257728    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz" podStartSLOduration=1.732815209 podStartE2EDuration="4.257712142s" totalImagesPullingTime="2.524896933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:24:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:25:00.256083603 +0000 UTC m=+2928.666794220" watchObservedRunningTime="2026-07-24 16:25:00.257712142 +0000 UTC m=+2928.668422759"
Jul 24 16:25:01.248056 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:01.248024    2561 generic.go:356] "Generic (PLEG): container finished" podID="6ea9e632-630e-4bc1-8e23-c788b66ac17d" containerID="128e1ef7217893d284f1cea9ab2145680251068d35a6276ac3c9659b174d6ef1" exitCode=0
Jul 24 16:25:01.248448 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:01.248115    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz" event={"ID":"6ea9e632-630e-4bc1-8e23-c788b66ac17d","Type":"ContainerDied","Data":"128e1ef7217893d284f1cea9ab2145680251068d35a6276ac3c9659b174d6ef1"}
Jul 24 16:25:02.392311 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:02.392287    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz"
Jul 24 16:25:02.501562 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:02.501527    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea9e632-630e-4bc1-8e23-c788b66ac17d-maas-api-service-ca\") pod \"6ea9e632-630e-4bc1-8e23-c788b66ac17d\" (UID: \"6ea9e632-630e-4bc1-8e23-c788b66ac17d\") "
Jul 24 16:25:02.501938 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:02.501906    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ea9e632-630e-4bc1-8e23-c788b66ac17d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6ea9e632-630e-4bc1-8e23-c788b66ac17d" (UID: "6ea9e632-630e-4bc1-8e23-c788b66ac17d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:25:02.602768 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:02.602649    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ea9e632-630e-4bc1-8e23-c788b66ac17d-maas-api-service-ca\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:25:03.259315 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:03.259289    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz"
Jul 24 16:25:03.259315 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:03.259302    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz" event={"ID":"6ea9e632-630e-4bc1-8e23-c788b66ac17d","Type":"ContainerDied","Data":"4dadb31f54445a56f1fe9dd173622c268cc2b235d7903f41e9298eaaeabebfe3"}
Jul 24 16:25:03.259525 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:03.259344    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4dadb31f54445a56f1fe9dd173622c268cc2b235d7903f41e9298eaaeabebfe3"
Jul 24 16:25:06.776709 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:06.776641    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"]
Jul 24 16:25:06.777196 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:06.776991    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l" podUID="9ace890b-3e64-4b06-ab44-eeaa22c634b9" containerName="payload-processing" containerID="cri-o://810b898d024976a4fafbe10baab27526925c67a1396b09f1222c6d3d14e76055" gracePeriod=30
Jul 24 16:25:06.785919 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:06.785893    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"]
Jul 24 16:25:06.786141 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:06.786100    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs" podUID="63820e82-989b-4489-87df-107e7bc2bb18" containerName="payload-pre-processing" containerID="cri-o://b3e93c6bf9e3b41980ee4f5f318e32bfa01aa350d8cabc69bcb9e2b8630aad3d" gracePeriod=30
Jul 24 16:25:07.069940 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.069904    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"
Jul 24 16:25:07.073026 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.072994    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"
Jul 24 16:25:07.142344 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.142309    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ace890b-3e64-4b06-ab44-eeaa22c634b9-plugins-config-volume\") pod \"9ace890b-3e64-4b06-ab44-eeaa22c634b9\" (UID: \"9ace890b-3e64-4b06-ab44-eeaa22c634b9\") "
Jul 24 16:25:07.142344 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.142347    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtvgl\" (UniqueName: \"kubernetes.io/projected/9ace890b-3e64-4b06-ab44-eeaa22c634b9-kube-api-access-wtvgl\") pod \"9ace890b-3e64-4b06-ab44-eeaa22c634b9\" (UID: \"9ace890b-3e64-4b06-ab44-eeaa22c634b9\") "
Jul 24 16:25:07.142559 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.142402    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63820e82-989b-4489-87df-107e7bc2bb18-plugins-config-volume\") pod \"63820e82-989b-4489-87df-107e7bc2bb18\" (UID: \"63820e82-989b-4489-87df-107e7bc2bb18\") "
Jul 24 16:25:07.142559 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.142469    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcp9n\" (UniqueName: \"kubernetes.io/projected/63820e82-989b-4489-87df-107e7bc2bb18-kube-api-access-pcp9n\") pod \"63820e82-989b-4489-87df-107e7bc2bb18\" (UID: \"63820e82-989b-4489-87df-107e7bc2bb18\") "
Jul 24 16:25:07.142902 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.142864    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ace890b-3e64-4b06-ab44-eeaa22c634b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9ace890b-3e64-4b06-ab44-eeaa22c634b9" (UID: "9ace890b-3e64-4b06-ab44-eeaa22c634b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:25:07.143411 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.143379    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63820e82-989b-4489-87df-107e7bc2bb18-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "63820e82-989b-4489-87df-107e7bc2bb18" (UID: "63820e82-989b-4489-87df-107e7bc2bb18"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:25:07.145618 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.145593    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ace890b-3e64-4b06-ab44-eeaa22c634b9-kube-api-access-wtvgl" (OuterVolumeSpecName: "kube-api-access-wtvgl") pod "9ace890b-3e64-4b06-ab44-eeaa22c634b9" (UID: "9ace890b-3e64-4b06-ab44-eeaa22c634b9"). InnerVolumeSpecName "kube-api-access-wtvgl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:25:07.145770 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.145620    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63820e82-989b-4489-87df-107e7bc2bb18-kube-api-access-pcp9n" (OuterVolumeSpecName: "kube-api-access-pcp9n") pod "63820e82-989b-4489-87df-107e7bc2bb18" (UID: "63820e82-989b-4489-87df-107e7bc2bb18"). InnerVolumeSpecName "kube-api-access-pcp9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:25:07.243239 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.243202    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcp9n\" (UniqueName: \"kubernetes.io/projected/63820e82-989b-4489-87df-107e7bc2bb18-kube-api-access-pcp9n\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:25:07.243239 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.243235    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ace890b-3e64-4b06-ab44-eeaa22c634b9-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:25:07.243239 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.243245    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtvgl\" (UniqueName: \"kubernetes.io/projected/9ace890b-3e64-4b06-ab44-eeaa22c634b9-kube-api-access-wtvgl\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:25:07.243482 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.243254    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63820e82-989b-4489-87df-107e7bc2bb18-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:25:07.288932 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.288901    2561 generic.go:356] "Generic (PLEG): container finished" podID="9ace890b-3e64-4b06-ab44-eeaa22c634b9" containerID="810b898d024976a4fafbe10baab27526925c67a1396b09f1222c6d3d14e76055" exitCode=0
Jul 24 16:25:07.289111 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.288945    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"
Jul 24 16:25:07.289111 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.288972    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l" event={"ID":"9ace890b-3e64-4b06-ab44-eeaa22c634b9","Type":"ContainerDied","Data":"810b898d024976a4fafbe10baab27526925c67a1396b09f1222c6d3d14e76055"}
Jul 24 16:25:07.289111 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.289019    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l" event={"ID":"9ace890b-3e64-4b06-ab44-eeaa22c634b9","Type":"ContainerDied","Data":"2b5cd5076f8c1ccd5bb3ad6f5bd6b9a4b73cf4ff78247720885cda4a3eb9bdd6"}
Jul 24 16:25:07.289111 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.289041    2561 scope.go:117] "RemoveContainer" containerID="810b898d024976a4fafbe10baab27526925c67a1396b09f1222c6d3d14e76055"
Jul 24 16:25:07.290320 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.290209    2561 generic.go:356] "Generic (PLEG): container finished" podID="63820e82-989b-4489-87df-107e7bc2bb18" containerID="b3e93c6bf9e3b41980ee4f5f318e32bfa01aa350d8cabc69bcb9e2b8630aad3d" exitCode=0
Jul 24 16:25:07.290320 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.290245    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"
Jul 24 16:25:07.290320 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.290304    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs" event={"ID":"63820e82-989b-4489-87df-107e7bc2bb18","Type":"ContainerDied","Data":"b3e93c6bf9e3b41980ee4f5f318e32bfa01aa350d8cabc69bcb9e2b8630aad3d"}
Jul 24 16:25:07.290580 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.290362    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs" event={"ID":"63820e82-989b-4489-87df-107e7bc2bb18","Type":"ContainerDied","Data":"ee9e28d7996a6489c1dc50bdd5716c6a3086f8d45f21c695d9b9ac68de35b4f9"}
Jul 24 16:25:07.298474 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.298454    2561 scope.go:117] "RemoveContainer" containerID="810b898d024976a4fafbe10baab27526925c67a1396b09f1222c6d3d14e76055"
Jul 24 16:25:07.298927 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:25:07.298904    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"810b898d024976a4fafbe10baab27526925c67a1396b09f1222c6d3d14e76055\": container with ID starting with 810b898d024976a4fafbe10baab27526925c67a1396b09f1222c6d3d14e76055 not found: ID does not exist" containerID="810b898d024976a4fafbe10baab27526925c67a1396b09f1222c6d3d14e76055"
Jul 24 16:25:07.299006 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.298934    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810b898d024976a4fafbe10baab27526925c67a1396b09f1222c6d3d14e76055"} err="failed to get container status \"810b898d024976a4fafbe10baab27526925c67a1396b09f1222c6d3d14e76055\": rpc error: code = NotFound desc = could not find container \"810b898d024976a4fafbe10baab27526925c67a1396b09f1222c6d3d14e76055\": container with ID starting with 810b898d024976a4fafbe10baab27526925c67a1396b09f1222c6d3d14e76055 not found: ID does not exist"
Jul 24 16:25:07.299006 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.298952    2561 scope.go:117] "RemoveContainer" containerID="b3e93c6bf9e3b41980ee4f5f318e32bfa01aa350d8cabc69bcb9e2b8630aad3d"
Jul 24 16:25:07.307998 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.307978    2561 scope.go:117] "RemoveContainer" containerID="b3e93c6bf9e3b41980ee4f5f318e32bfa01aa350d8cabc69bcb9e2b8630aad3d"
Jul 24 16:25:07.308311 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:25:07.308289    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3e93c6bf9e3b41980ee4f5f318e32bfa01aa350d8cabc69bcb9e2b8630aad3d\": container with ID starting with b3e93c6bf9e3b41980ee4f5f318e32bfa01aa350d8cabc69bcb9e2b8630aad3d not found: ID does not exist" containerID="b3e93c6bf9e3b41980ee4f5f318e32bfa01aa350d8cabc69bcb9e2b8630aad3d"
Jul 24 16:25:07.308347 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.308322    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3e93c6bf9e3b41980ee4f5f318e32bfa01aa350d8cabc69bcb9e2b8630aad3d"} err="failed to get container status \"b3e93c6bf9e3b41980ee4f5f318e32bfa01aa350d8cabc69bcb9e2b8630aad3d\": rpc error: code = NotFound desc = could not find container \"b3e93c6bf9e3b41980ee4f5f318e32bfa01aa350d8cabc69bcb9e2b8630aad3d\": container with ID starting with b3e93c6bf9e3b41980ee4f5f318e32bfa01aa350d8cabc69bcb9e2b8630aad3d not found: ID does not exist"
Jul 24 16:25:07.317787 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.317756    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"]
Jul 24 16:25:07.325378 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.325351    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-07e89c-7874f6f648-l847l"]
Jul 24 16:25:07.335036 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.335011    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"]
Jul 24 16:25:07.339608 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:07.339583    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-07e89c-98c57f5b4-78wcs"]
Jul 24 16:25:08.107117 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:08.107039    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63820e82-989b-4489-87df-107e7bc2bb18" path="/var/lib/kubelet/pods/63820e82-989b-4489-87df-107e7bc2bb18/volumes"
Jul 24 16:25:08.107476 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:08.107374    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ace890b-3e64-4b06-ab44-eeaa22c634b9" path="/var/lib/kubelet/pods/9ace890b-3e64-4b06-ab44-eeaa22c634b9/volumes"
Jul 24 16:25:11.776551 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:11.776508    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz"]
Jul 24 16:25:11.782012 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:11.781980    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-07e89c-rg4vz"]
Jul 24 16:25:12.106854 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:12.106767    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ea9e632-630e-4bc1-8e23-c788b66ac17d" path="/var/lib/kubelet/pods/6ea9e632-630e-4bc1-8e23-c788b66ac17d/volumes"
Jul 24 16:25:19.184774 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.184738    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv"]
Jul 24 16:25:19.185274 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.185106    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ea9e632-630e-4bc1-8e23-c788b66ac17d" containerName="revoke-keys"
Jul 24 16:25:19.185274 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.185117    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ea9e632-630e-4bc1-8e23-c788b66ac17d" containerName="revoke-keys"
Jul 24 16:25:19.185274 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.185127    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ace890b-3e64-4b06-ab44-eeaa22c634b9" containerName="payload-processing"
Jul 24 16:25:19.185274 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.185133    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ace890b-3e64-4b06-ab44-eeaa22c634b9" containerName="payload-processing"
Jul 24 16:25:19.185274 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.185145    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63820e82-989b-4489-87df-107e7bc2bb18" containerName="payload-pre-processing"
Jul 24 16:25:19.185274 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.185150    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="63820e82-989b-4489-87df-107e7bc2bb18" containerName="payload-pre-processing"
Jul 24 16:25:19.185274 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.185215    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="63820e82-989b-4489-87df-107e7bc2bb18" containerName="payload-pre-processing"
Jul 24 16:25:19.185274 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.185223    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ea9e632-630e-4bc1-8e23-c788b66ac17d" containerName="revoke-keys"
Jul 24 16:25:19.185274 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.185232    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ace890b-3e64-4b06-ab44-eeaa22c634b9" containerName="payload-processing"
Jul 24 16:25:19.189972 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.189956    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv"
Jul 24 16:25:19.192870 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.192846    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xnkqn\""
Jul 24 16:25:19.198107 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.198081    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv"]
Jul 24 16:25:19.351045 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.351005    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb59402d-d6b8-4341-b85f-a3037bcf16df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv\" (UID: \"cb59402d-d6b8-4341-b85f-a3037bcf16df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv"
Jul 24 16:25:19.452281 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.452178    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb59402d-d6b8-4341-b85f-a3037bcf16df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv\" (UID: \"cb59402d-d6b8-4341-b85f-a3037bcf16df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv"
Jul 24 16:25:19.452905 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.452882    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb59402d-d6b8-4341-b85f-a3037bcf16df-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv\" (UID: \"cb59402d-d6b8-4341-b85f-a3037bcf16df\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv"
Jul 24 16:25:19.502119 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.502080    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv"
Jul 24 16:25:19.662425 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:19.662236    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv"]
Jul 24 16:25:19.665317 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:25:19.665285    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb59402d_d6b8_4341_b85f_a3037bcf16df.slice/crio-860dad84b4edd4b708c7f1565f80b5a2f7f000f062a46b8edaccea1f1d786484 WatchSource:0}: Error finding container 860dad84b4edd4b708c7f1565f80b5a2f7f000f062a46b8edaccea1f1d786484: Status 404 returned error can't find the container with id 860dad84b4edd4b708c7f1565f80b5a2f7f000f062a46b8edaccea1f1d786484
Jul 24 16:25:20.366770 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:20.366733    2561 generic.go:356] "Generic (PLEG): container finished" podID="cb59402d-d6b8-4341-b85f-a3037bcf16df" containerID="07ee4be038c08f1ae8353cb7629bf3e16372ecc2d6ac3d111ec422e1e8c7c106" exitCode=0
Jul 24 16:25:20.367142 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:20.366813    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv" event={"ID":"cb59402d-d6b8-4341-b85f-a3037bcf16df","Type":"ContainerDied","Data":"07ee4be038c08f1ae8353cb7629bf3e16372ecc2d6ac3d111ec422e1e8c7c106"}
Jul 24 16:25:20.367142 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:20.366870    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv" event={"ID":"cb59402d-d6b8-4341-b85f-a3037bcf16df","Type":"ContainerStarted","Data":"860dad84b4edd4b708c7f1565f80b5a2f7f000f062a46b8edaccea1f1d786484"}
Jul 24 16:25:21.516830 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:21.516800    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv"
Jul 24 16:25:21.672695 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:21.672581    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb59402d-d6b8-4341-b85f-a3037bcf16df-maas-api-service-ca\") pod \"cb59402d-d6b8-4341-b85f-a3037bcf16df\" (UID: \"cb59402d-d6b8-4341-b85f-a3037bcf16df\") "
Jul 24 16:25:21.672997 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:21.672975    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb59402d-d6b8-4341-b85f-a3037bcf16df-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cb59402d-d6b8-4341-b85f-a3037bcf16df" (UID: "cb59402d-d6b8-4341-b85f-a3037bcf16df"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:25:21.773516 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:21.773477    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb59402d-d6b8-4341-b85f-a3037bcf16df-maas-api-service-ca\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:25:22.379393 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:22.379303    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv"
Jul 24 16:25:22.379393 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:22.379313    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv" event={"ID":"cb59402d-d6b8-4341-b85f-a3037bcf16df","Type":"ContainerDied","Data":"860dad84b4edd4b708c7f1565f80b5a2f7f000f062a46b8edaccea1f1d786484"}
Jul 24 16:25:22.379393 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:22.379366    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="860dad84b4edd4b708c7f1565f80b5a2f7f000f062a46b8edaccea1f1d786484"
Jul 24 16:25:31.494143 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.494103    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"]
Jul 24 16:25:31.494962 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.494919    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp" podUID="94e6b954-4694-4b82-933d-54651f0456c9" containerName="payload-processing" containerID="cri-o://77c0326e81dc5c8e0dfd8e8fc76d96c29423189f894a301ec6d08d4048547320" gracePeriod=30
Jul 24 16:25:31.502199 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.502168    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"]
Jul 24 16:25:31.502413 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.502390    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk" podUID="a89ac429-adc7-43ad-a61a-77f2674a73d1" containerName="payload-pre-processing" containerID="cri-o://435344ffc1cf2e5a5a659fbb7917c27661c805b4e3ad964e7fbbfed37cf79688" gracePeriod=30
Jul 24 16:25:31.807074 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.807046    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"
Jul 24 16:25:31.809838 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.809801    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"
Jul 24 16:25:31.864920 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.864894    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a89ac429-adc7-43ad-a61a-77f2674a73d1-plugins-config-volume\") pod \"a89ac429-adc7-43ad-a61a-77f2674a73d1\" (UID: \"a89ac429-adc7-43ad-a61a-77f2674a73d1\") "
Jul 24 16:25:31.865107 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.864940    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94e6b954-4694-4b82-933d-54651f0456c9-plugins-config-volume\") pod \"94e6b954-4694-4b82-933d-54651f0456c9\" (UID: \"94e6b954-4694-4b82-933d-54651f0456c9\") "
Jul 24 16:25:31.865107 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.864968    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swbsn\" (UniqueName: \"kubernetes.io/projected/94e6b954-4694-4b82-933d-54651f0456c9-kube-api-access-swbsn\") pod \"94e6b954-4694-4b82-933d-54651f0456c9\" (UID: \"94e6b954-4694-4b82-933d-54651f0456c9\") "
Jul 24 16:25:31.865107 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.865070    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtwjl\" (UniqueName: \"kubernetes.io/projected/a89ac429-adc7-43ad-a61a-77f2674a73d1-kube-api-access-gtwjl\") pod \"a89ac429-adc7-43ad-a61a-77f2674a73d1\" (UID: \"a89ac429-adc7-43ad-a61a-77f2674a73d1\") "
Jul 24 16:25:31.865361 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.865334    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a89ac429-adc7-43ad-a61a-77f2674a73d1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a89ac429-adc7-43ad-a61a-77f2674a73d1" (UID: "a89ac429-adc7-43ad-a61a-77f2674a73d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:25:31.865431 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.865402    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94e6b954-4694-4b82-933d-54651f0456c9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "94e6b954-4694-4b82-933d-54651f0456c9" (UID: "94e6b954-4694-4b82-933d-54651f0456c9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:25:31.867351 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.867324    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a89ac429-adc7-43ad-a61a-77f2674a73d1-kube-api-access-gtwjl" (OuterVolumeSpecName: "kube-api-access-gtwjl") pod "a89ac429-adc7-43ad-a61a-77f2674a73d1" (UID: "a89ac429-adc7-43ad-a61a-77f2674a73d1"). InnerVolumeSpecName "kube-api-access-gtwjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:25:31.867433 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.867407    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94e6b954-4694-4b82-933d-54651f0456c9-kube-api-access-swbsn" (OuterVolumeSpecName: "kube-api-access-swbsn") pod "94e6b954-4694-4b82-933d-54651f0456c9" (UID: "94e6b954-4694-4b82-933d-54651f0456c9"). InnerVolumeSpecName "kube-api-access-swbsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:25:31.966402 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.966367    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtwjl\" (UniqueName: \"kubernetes.io/projected/a89ac429-adc7-43ad-a61a-77f2674a73d1-kube-api-access-gtwjl\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:25:31.966402 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.966394    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a89ac429-adc7-43ad-a61a-77f2674a73d1-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:25:31.966402 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.966408    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94e6b954-4694-4b82-933d-54651f0456c9-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:25:31.966649 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:31.966416    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swbsn\" (UniqueName: \"kubernetes.io/projected/94e6b954-4694-4b82-933d-54651f0456c9-kube-api-access-swbsn\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:25:32.431706 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.431588    2561 generic.go:356] "Generic (PLEG): container finished" podID="94e6b954-4694-4b82-933d-54651f0456c9" containerID="77c0326e81dc5c8e0dfd8e8fc76d96c29423189f894a301ec6d08d4048547320" exitCode=0
Jul 24 16:25:32.431706 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.431654    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"
Jul 24 16:25:32.431706 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.431696    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp" event={"ID":"94e6b954-4694-4b82-933d-54651f0456c9","Type":"ContainerDied","Data":"77c0326e81dc5c8e0dfd8e8fc76d96c29423189f894a301ec6d08d4048547320"}
Jul 24 16:25:32.431962 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.431748    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp" event={"ID":"94e6b954-4694-4b82-933d-54651f0456c9","Type":"ContainerDied","Data":"d18f59761455ef59d0c0adf509e932e9f8ce63a9453f914debcb824e0872fd4f"}
Jul 24 16:25:32.431962 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.431771    2561 scope.go:117] "RemoveContainer" containerID="77c0326e81dc5c8e0dfd8e8fc76d96c29423189f894a301ec6d08d4048547320"
Jul 24 16:25:32.432941 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.432924    2561 generic.go:356] "Generic (PLEG): container finished" podID="a89ac429-adc7-43ad-a61a-77f2674a73d1" containerID="435344ffc1cf2e5a5a659fbb7917c27661c805b4e3ad964e7fbbfed37cf79688" exitCode=0
Jul 24 16:25:32.433085 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.432971    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk" event={"ID":"a89ac429-adc7-43ad-a61a-77f2674a73d1","Type":"ContainerDied","Data":"435344ffc1cf2e5a5a659fbb7917c27661c805b4e3ad964e7fbbfed37cf79688"}
Jul 24 16:25:32.433085 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.432980    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"
Jul 24 16:25:32.433085 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.433006    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk" event={"ID":"a89ac429-adc7-43ad-a61a-77f2674a73d1","Type":"ContainerDied","Data":"90cb9927c6918f165c96ae78a36e967c6bf5cee9f3f0b39f9ad248db23790fa0"}
Jul 24 16:25:32.442641 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.442618    2561 scope.go:117] "RemoveContainer" containerID="77c0326e81dc5c8e0dfd8e8fc76d96c29423189f894a301ec6d08d4048547320"
Jul 24 16:25:32.443063 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:25:32.443034    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77c0326e81dc5c8e0dfd8e8fc76d96c29423189f894a301ec6d08d4048547320\": container with ID starting with 77c0326e81dc5c8e0dfd8e8fc76d96c29423189f894a301ec6d08d4048547320 not found: ID does not exist" containerID="77c0326e81dc5c8e0dfd8e8fc76d96c29423189f894a301ec6d08d4048547320"
Jul 24 16:25:32.443138 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.443073    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77c0326e81dc5c8e0dfd8e8fc76d96c29423189f894a301ec6d08d4048547320"} err="failed to get container status \"77c0326e81dc5c8e0dfd8e8fc76d96c29423189f894a301ec6d08d4048547320\": rpc error: code = NotFound desc = could not find container \"77c0326e81dc5c8e0dfd8e8fc76d96c29423189f894a301ec6d08d4048547320\": container with ID starting with 77c0326e81dc5c8e0dfd8e8fc76d96c29423189f894a301ec6d08d4048547320 not found: ID does not exist"
Jul 24 16:25:32.443138 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.443096    2561 scope.go:117] "RemoveContainer" containerID="435344ffc1cf2e5a5a659fbb7917c27661c805b4e3ad964e7fbbfed37cf79688"
Jul 24 16:25:32.453594 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.452682    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"]
Jul 24 16:25:32.456965 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.456927    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-fafc28-8f9759f74-86gjp"]
Jul 24 16:25:32.458631 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.458608    2561 scope.go:117] "RemoveContainer" containerID="435344ffc1cf2e5a5a659fbb7917c27661c805b4e3ad964e7fbbfed37cf79688"
Jul 24 16:25:32.459022 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:25:32.459003    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"435344ffc1cf2e5a5a659fbb7917c27661c805b4e3ad964e7fbbfed37cf79688\": container with ID starting with 435344ffc1cf2e5a5a659fbb7917c27661c805b4e3ad964e7fbbfed37cf79688 not found: ID does not exist" containerID="435344ffc1cf2e5a5a659fbb7917c27661c805b4e3ad964e7fbbfed37cf79688"
Jul 24 16:25:32.459104 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.459032    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"435344ffc1cf2e5a5a659fbb7917c27661c805b4e3ad964e7fbbfed37cf79688"} err="failed to get container status \"435344ffc1cf2e5a5a659fbb7917c27661c805b4e3ad964e7fbbfed37cf79688\": rpc error: code = NotFound desc = could not find container \"435344ffc1cf2e5a5a659fbb7917c27661c805b4e3ad964e7fbbfed37cf79688\": container with ID starting with 435344ffc1cf2e5a5a659fbb7917c27661c805b4e3ad964e7fbbfed37cf79688 not found: ID does not exist"
Jul 24 16:25:32.471234 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.471205    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"]
Jul 24 16:25:32.478185 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:32.478159    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-fafc28-594d97f8b8-2tjlk"]
Jul 24 16:25:34.106649 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:34.106615    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94e6b954-4694-4b82-933d-54651f0456c9" path="/var/lib/kubelet/pods/94e6b954-4694-4b82-933d-54651f0456c9/volumes"
Jul 24 16:25:34.107052 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:34.106994    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a89ac429-adc7-43ad-a61a-77f2674a73d1" path="/var/lib/kubelet/pods/a89ac429-adc7-43ad-a61a-77f2674a73d1/volumes"
Jul 24 16:25:34.450978 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:34.450889    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv"]
Jul 24 16:25:34.454414 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:34.454388    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-fafc28-jpmdv"]
Jul 24 16:25:36.106381 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:36.106346    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb59402d-d6b8-4341-b85f-a3037bcf16df" path="/var/lib/kubelet/pods/cb59402d-d6b8-4341-b85f-a3037bcf16df/volumes"
Jul 24 16:25:42.054285 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.054245    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24"]
Jul 24 16:25:42.054732 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.054629    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94e6b954-4694-4b82-933d-54651f0456c9" containerName="payload-processing"
Jul 24 16:25:42.054732 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.054644    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="94e6b954-4694-4b82-933d-54651f0456c9" containerName="payload-processing"
Jul 24 16:25:42.054732 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.054674    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a89ac429-adc7-43ad-a61a-77f2674a73d1" containerName="payload-pre-processing"
Jul 24 16:25:42.054732 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.054680    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a89ac429-adc7-43ad-a61a-77f2674a73d1" containerName="payload-pre-processing"
Jul 24 16:25:42.054732 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.054698    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb59402d-d6b8-4341-b85f-a3037bcf16df" containerName="revoke-keys"
Jul 24 16:25:42.054732 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.054703    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb59402d-d6b8-4341-b85f-a3037bcf16df" containerName="revoke-keys"
Jul 24 16:25:42.054925 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.054767    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a89ac429-adc7-43ad-a61a-77f2674a73d1" containerName="payload-pre-processing"
Jul 24 16:25:42.054925 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.054777    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="94e6b954-4694-4b82-933d-54651f0456c9" containerName="payload-processing"
Jul 24 16:25:42.054925 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.054783    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb59402d-d6b8-4341-b85f-a3037bcf16df" containerName="revoke-keys"
Jul 24 16:25:42.060895 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.060875    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24"
Jul 24 16:25:42.064110 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.064091    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xnkqn\""
Jul 24 16:25:42.067968 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.067939    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24"]
Jul 24 16:25:42.160022 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.159984    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8e13e21-ccf7-407c-9825-c71d6bc11b03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24\" (UID: \"a8e13e21-ccf7-407c-9825-c71d6bc11b03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24"
Jul 24 16:25:42.260902 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.260860    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8e13e21-ccf7-407c-9825-c71d6bc11b03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24\" (UID: \"a8e13e21-ccf7-407c-9825-c71d6bc11b03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24"
Jul 24 16:25:42.261474 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.261453    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8e13e21-ccf7-407c-9825-c71d6bc11b03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24\" (UID: \"a8e13e21-ccf7-407c-9825-c71d6bc11b03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24"
Jul 24 16:25:42.373418 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.373329    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24"
Jul 24 16:25:42.542714 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:42.542684    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24"]
Jul 24 16:25:42.545594 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:25:42.545556    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8e13e21_ccf7_407c_9825_c71d6bc11b03.slice/crio-ca707d85efdb1b6b8b5c9a90bbda64b5ce367f8ebf2f9ded542e9fe2865fac13 WatchSource:0}: Error finding container ca707d85efdb1b6b8b5c9a90bbda64b5ce367f8ebf2f9ded542e9fe2865fac13: Status 404 returned error can't find the container with id ca707d85efdb1b6b8b5c9a90bbda64b5ce367f8ebf2f9ded542e9fe2865fac13
Jul 24 16:25:43.497003 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:43.496967    2561 generic.go:356] "Generic (PLEG): container finished" podID="a8e13e21-ccf7-407c-9825-c71d6bc11b03" containerID="774dcc0ce40c7713b2516c62e94e0276d3ce72f4e81d76875c211a570d73052c" exitCode=0
Jul 24 16:25:43.497422 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:43.497056    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24" event={"ID":"a8e13e21-ccf7-407c-9825-c71d6bc11b03","Type":"ContainerDied","Data":"774dcc0ce40c7713b2516c62e94e0276d3ce72f4e81d76875c211a570d73052c"}
Jul 24 16:25:43.497422 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:43.497158    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24" event={"ID":"a8e13e21-ccf7-407c-9825-c71d6bc11b03","Type":"ContainerStarted","Data":"ca707d85efdb1b6b8b5c9a90bbda64b5ce367f8ebf2f9ded542e9fe2865fac13"}
Jul 24 16:25:44.648350 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:44.648324    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24"
Jul 24 16:25:44.783951 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:44.783844    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8e13e21-ccf7-407c-9825-c71d6bc11b03-maas-api-service-ca\") pod \"a8e13e21-ccf7-407c-9825-c71d6bc11b03\" (UID: \"a8e13e21-ccf7-407c-9825-c71d6bc11b03\") "
Jul 24 16:25:44.784255 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:44.784228    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8e13e21-ccf7-407c-9825-c71d6bc11b03-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a8e13e21-ccf7-407c-9825-c71d6bc11b03" (UID: "a8e13e21-ccf7-407c-9825-c71d6bc11b03"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:25:44.884507 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:44.884472    2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8e13e21-ccf7-407c-9825-c71d6bc11b03-maas-api-service-ca\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:25:45.509148 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:45.509115    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24"
Jul 24 16:25:45.509339 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:45.509164    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24" event={"ID":"a8e13e21-ccf7-407c-9825-c71d6bc11b03","Type":"ContainerDied","Data":"ca707d85efdb1b6b8b5c9a90bbda64b5ce367f8ebf2f9ded542e9fe2865fac13"}
Jul 24 16:25:45.509339 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:45.509221    2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca707d85efdb1b6b8b5c9a90bbda64b5ce367f8ebf2f9ded542e9fe2865fac13"
Jul 24 16:25:52.846613 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:52.846573    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"]
Jul 24 16:25:52.847563 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:52.846950    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b" podUID="eb1025b2-acfb-48e0-bdce-c80127a09ec1" containerName="payload-processing" containerID="cri-o://d52af8ea55dce8d7bc9e73e7dd8965695ee187e9682d355f6124d2d794f07d4a" gracePeriod=30
Jul 24 16:25:52.861776 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:52.861487    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"]
Jul 24 16:25:52.862292 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:52.862099    2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66" podUID="3f9bee4d-8b84-4ddd-8740-1db054920dbe" containerName="payload-pre-processing" containerID="cri-o://d31690e4e4930e27c4851170f8e895b7699f2786926b90a9f3a7f3485a3c3dbe" gracePeriod=30
Jul 24 16:25:53.030883 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.030393    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66" podUID="3f9bee4d-8b84-4ddd-8740-1db054920dbe" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.70:9004: connect: connection refused"
Jul 24 16:25:53.150081 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.150052    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"
Jul 24 16:25:53.154312 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.154287    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"
Jul 24 16:25:53.256322 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.256281    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7ppk\" (UniqueName: \"kubernetes.io/projected/eb1025b2-acfb-48e0-bdce-c80127a09ec1-kube-api-access-b7ppk\") pod \"eb1025b2-acfb-48e0-bdce-c80127a09ec1\" (UID: \"eb1025b2-acfb-48e0-bdce-c80127a09ec1\") "
Jul 24 16:25:53.256322 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.256319    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bswhb\" (UniqueName: \"kubernetes.io/projected/3f9bee4d-8b84-4ddd-8740-1db054920dbe-kube-api-access-bswhb\") pod \"3f9bee4d-8b84-4ddd-8740-1db054920dbe\" (UID: \"3f9bee4d-8b84-4ddd-8740-1db054920dbe\") "
Jul 24 16:25:53.256602 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.256381    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1025b2-acfb-48e0-bdce-c80127a09ec1-plugins-config-volume\") pod \"eb1025b2-acfb-48e0-bdce-c80127a09ec1\" (UID: \"eb1025b2-acfb-48e0-bdce-c80127a09ec1\") "
Jul 24 16:25:53.256602 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.256415    2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f9bee4d-8b84-4ddd-8740-1db054920dbe-plugins-config-volume\") pod \"3f9bee4d-8b84-4ddd-8740-1db054920dbe\" (UID: \"3f9bee4d-8b84-4ddd-8740-1db054920dbe\") "
Jul 24 16:25:53.256822 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.256797    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb1025b2-acfb-48e0-bdce-c80127a09ec1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eb1025b2-acfb-48e0-bdce-c80127a09ec1" (UID: "eb1025b2-acfb-48e0-bdce-c80127a09ec1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:25:53.256959 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.256836    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f9bee4d-8b84-4ddd-8740-1db054920dbe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3f9bee4d-8b84-4ddd-8740-1db054920dbe" (UID: "3f9bee4d-8b84-4ddd-8740-1db054920dbe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:25:53.258684 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.258644    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb1025b2-acfb-48e0-bdce-c80127a09ec1-kube-api-access-b7ppk" (OuterVolumeSpecName: "kube-api-access-b7ppk") pod "eb1025b2-acfb-48e0-bdce-c80127a09ec1" (UID: "eb1025b2-acfb-48e0-bdce-c80127a09ec1"). InnerVolumeSpecName "kube-api-access-b7ppk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:25:53.258684 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.258653    2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f9bee4d-8b84-4ddd-8740-1db054920dbe-kube-api-access-bswhb" (OuterVolumeSpecName: "kube-api-access-bswhb") pod "3f9bee4d-8b84-4ddd-8740-1db054920dbe" (UID: "3f9bee4d-8b84-4ddd-8740-1db054920dbe"). InnerVolumeSpecName "kube-api-access-bswhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:25:53.357093 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.357045    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1025b2-acfb-48e0-bdce-c80127a09ec1-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:25:53.357093 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.357080    2561 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f9bee4d-8b84-4ddd-8740-1db054920dbe-plugins-config-volume\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:25:53.357093 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.357091    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7ppk\" (UniqueName: \"kubernetes.io/projected/eb1025b2-acfb-48e0-bdce-c80127a09ec1-kube-api-access-b7ppk\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:25:53.357093 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.357101    2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bswhb\" (UniqueName: \"kubernetes.io/projected/3f9bee4d-8b84-4ddd-8740-1db054920dbe-kube-api-access-bswhb\") on node \"ip-10-0-135-164.ec2.internal\" DevicePath \"\""
Jul 24 16:25:53.549936 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.549895    2561 generic.go:356] "Generic (PLEG): container finished" podID="eb1025b2-acfb-48e0-bdce-c80127a09ec1" containerID="d52af8ea55dce8d7bc9e73e7dd8965695ee187e9682d355f6124d2d794f07d4a" exitCode=0
Jul 24 16:25:53.550115 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.549944    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"
Jul 24 16:25:53.550115 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.549980    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b" event={"ID":"eb1025b2-acfb-48e0-bdce-c80127a09ec1","Type":"ContainerDied","Data":"d52af8ea55dce8d7bc9e73e7dd8965695ee187e9682d355f6124d2d794f07d4a"}
Jul 24 16:25:53.550115 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.550034    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b" event={"ID":"eb1025b2-acfb-48e0-bdce-c80127a09ec1","Type":"ContainerDied","Data":"658b43759e57f5c9f85740d15d3965d631a37bedf490972ee4923a2f8b56f70f"}
Jul 24 16:25:53.550115 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.550064    2561 scope.go:117] "RemoveContainer" containerID="d52af8ea55dce8d7bc9e73e7dd8965695ee187e9682d355f6124d2d794f07d4a"
Jul 24 16:25:53.551551 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.551526    2561 generic.go:356] "Generic (PLEG): container finished" podID="3f9bee4d-8b84-4ddd-8740-1db054920dbe" containerID="d31690e4e4930e27c4851170f8e895b7699f2786926b90a9f3a7f3485a3c3dbe" exitCode=0
Jul 24 16:25:53.551718 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.551593    2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"
Jul 24 16:25:53.551718 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.551589    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66" event={"ID":"3f9bee4d-8b84-4ddd-8740-1db054920dbe","Type":"ContainerDied","Data":"d31690e4e4930e27c4851170f8e895b7699f2786926b90a9f3a7f3485a3c3dbe"}
Jul 24 16:25:53.551853 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.551735    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66" event={"ID":"3f9bee4d-8b84-4ddd-8740-1db054920dbe","Type":"ContainerDied","Data":"5b59ed1defb5fb960e98d653edd2b5522dcfe006e63479f7cb5ef0d3aa2d9c07"}
Jul 24 16:25:53.559559 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.559535    2561 scope.go:117] "RemoveContainer" containerID="d52af8ea55dce8d7bc9e73e7dd8965695ee187e9682d355f6124d2d794f07d4a"
Jul 24 16:25:53.559968 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:25:53.559937    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d52af8ea55dce8d7bc9e73e7dd8965695ee187e9682d355f6124d2d794f07d4a\": container with ID starting with d52af8ea55dce8d7bc9e73e7dd8965695ee187e9682d355f6124d2d794f07d4a not found: ID does not exist" containerID="d52af8ea55dce8d7bc9e73e7dd8965695ee187e9682d355f6124d2d794f07d4a"
Jul 24 16:25:53.560102 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.559975    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d52af8ea55dce8d7bc9e73e7dd8965695ee187e9682d355f6124d2d794f07d4a"} err="failed to get container status \"d52af8ea55dce8d7bc9e73e7dd8965695ee187e9682d355f6124d2d794f07d4a\": rpc error: code = NotFound desc = could not find container \"d52af8ea55dce8d7bc9e73e7dd8965695ee187e9682d355f6124d2d794f07d4a\": container with ID starting with d52af8ea55dce8d7bc9e73e7dd8965695ee187e9682d355f6124d2d794f07d4a not found: ID does not exist"
Jul 24 16:25:53.560102 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.560002    2561 scope.go:117] "RemoveContainer" containerID="d31690e4e4930e27c4851170f8e895b7699f2786926b90a9f3a7f3485a3c3dbe"
Jul 24 16:25:53.571878 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.571858    2561 scope.go:117] "RemoveContainer" containerID="d31690e4e4930e27c4851170f8e895b7699f2786926b90a9f3a7f3485a3c3dbe"
Jul 24 16:25:53.572217 ip-10-0-135-164 kubenswrapper[2561]: E0724 16:25:53.572197    2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d31690e4e4930e27c4851170f8e895b7699f2786926b90a9f3a7f3485a3c3dbe\": container with ID starting with d31690e4e4930e27c4851170f8e895b7699f2786926b90a9f3a7f3485a3c3dbe not found: ID does not exist" containerID="d31690e4e4930e27c4851170f8e895b7699f2786926b90a9f3a7f3485a3c3dbe"
Jul 24 16:25:53.572292 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.572228    2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d31690e4e4930e27c4851170f8e895b7699f2786926b90a9f3a7f3485a3c3dbe"} err="failed to get container status \"d31690e4e4930e27c4851170f8e895b7699f2786926b90a9f3a7f3485a3c3dbe\": rpc error: code = NotFound desc = could not find container \"d31690e4e4930e27c4851170f8e895b7699f2786926b90a9f3a7f3485a3c3dbe\": container with ID starting with d31690e4e4930e27c4851170f8e895b7699f2786926b90a9f3a7f3485a3c3dbe not found: ID does not exist"
Jul 24 16:25:53.586175 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.586133    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"]
Jul 24 16:25:53.589911 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.589879    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b"]
Jul 24 16:25:53.612478 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.612440    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"]
Jul 24 16:25:53.617321 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:53.617290    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c6c91e-66cc754b8d-hnd66"]
Jul 24 16:25:54.030765 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:54.030724    2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-c6c91e-bb646b46b-88f4b" podUID="eb1025b2-acfb-48e0-bdce-c80127a09ec1" containerName="payload-processing" probeResult="failure" output="dial tcp 10.133.0.69:9004: i/o timeout"
Jul 24 16:25:54.112240 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:54.112206    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f9bee4d-8b84-4ddd-8740-1db054920dbe" path="/var/lib/kubelet/pods/3f9bee4d-8b84-4ddd-8740-1db054920dbe/volumes"
Jul 24 16:25:54.112559 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:54.112546    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb1025b2-acfb-48e0-bdce-c80127a09ec1" path="/var/lib/kubelet/pods/eb1025b2-acfb-48e0-bdce-c80127a09ec1/volumes"
Jul 24 16:25:57.208442 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:57.208408    2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24"]
Jul 24 16:25:57.217804 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:57.217775    2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c6c91e-zgp24"]
Jul 24 16:25:58.106757 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:25:58.106725    2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8e13e21-ccf7-407c-9825-c71d6bc11b03" path="/var/lib/kubelet/pods/a8e13e21-ccf7-407c-9825-c71d6bc11b03/volumes"
Jul 24 16:26:05.970262 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:05.970222    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6f468f5-ztsjj_e96645d7-a4df-4e93-bd25-19b1ad71f158/authorino/0.log"
Jul 24 16:26:12.158870 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:12.158839    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-pqdnc_9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd/manager/0.log"
Jul 24 16:26:12.278003 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:12.277971    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-8zx68_9eac2967-63a8-4d6e-89a0-3fd21f1cee11/manager/0.log"
Jul 24 16:26:12.410624 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:12.410531    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67bb47cc5d-9df4d_d11df53b-f05a-40f2-82af-22a66735a556/manager/0.log"
Jul 24 16:26:12.530714 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:12.530685    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gbdjq_dc394fc0-3581-48d0-ba1e-9c70876d9a73/server/0.log"
Jul 24 16:26:13.087393 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:13.087363    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b7484f4dc-r9vps_472a5da2-28a3-4250-9eb9-98f1ae8efa46/copy-manifests/0.log"
Jul 24 16:26:13.095243 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:13.095215    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b7484f4dc-r9vps_472a5da2-28a3-4250-9eb9-98f1ae8efa46/manager/0.log"
Jul 24 16:26:13.828511 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:13.828478    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-f798k_e863f716-e88f-4b89-9fe8-9a47157fed96/postgres/0.log"
Jul 24 16:26:15.171587 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:15.171544    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6f468f5-ztsjj_e96645d7-a4df-4e93-bd25-19b1ad71f158/authorino/0.log"
Jul 24 16:26:15.292283 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:15.292251    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-hnnbz_5e7a3852-e1c7-42f7-a96a-05b6c00f18c9/manager/0.log"
Jul 24 16:26:15.659701 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:15.659642    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-zjvt9_5802f3ed-2120-4987-82e4-b3e420cf3ba3/registry-server/0.log"
Jul 24 16:26:15.783109 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:15.783077    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zcg6b_12ea3d30-44e2-460e-813a-fa7f80aae5ba/manager/0.log"
Jul 24 16:26:16.018943 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:16.018910    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-bbt4w_7b21dbee-df26-48c4-a68d-25a8feb04e6a/manager/0.log"
Jul 24 16:26:16.651922 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:16.651886    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59fb5479dc-kcpmq_5ef6a247-e63f-4351-906b-479a8ddb2294/kube-auth-proxy/0.log"
Jul 24 16:26:17.169680 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:17.169627    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-qf26j_fa01d160-d791-465e-9488-0ae89aca50b3/payload-processing/0.log"
Jul 24 16:26:17.679726 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:17.679698    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-nb5f5_160829c5-e8bf-4ed8-9380-cc0234f2023b/main/0.log"
Jul 24 16:26:17.921634 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:17.921606    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-wwgsn_38f9eab2-2be5-4fb3-b7e8-eb8001a4a09c/main/0.log"
Jul 24 16:26:24.216914 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.216877    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zsrwm/must-gather-dsnz7"]
Jul 24 16:26:24.217556 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.217394    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb1025b2-acfb-48e0-bdce-c80127a09ec1" containerName="payload-processing"
Jul 24 16:26:24.217556 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.217414    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb1025b2-acfb-48e0-bdce-c80127a09ec1" containerName="payload-processing"
Jul 24 16:26:24.217556 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.217430    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f9bee4d-8b84-4ddd-8740-1db054920dbe" containerName="payload-pre-processing"
Jul 24 16:26:24.217556 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.217439    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f9bee4d-8b84-4ddd-8740-1db054920dbe" containerName="payload-pre-processing"
Jul 24 16:26:24.217556 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.217462    2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8e13e21-ccf7-407c-9825-c71d6bc11b03" containerName="revoke-keys"
Jul 24 16:26:24.217556 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.217471    2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8e13e21-ccf7-407c-9825-c71d6bc11b03" containerName="revoke-keys"
Jul 24 16:26:24.217556 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.217549    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb1025b2-acfb-48e0-bdce-c80127a09ec1" containerName="payload-processing"
Jul 24 16:26:24.217556 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.217560    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8e13e21-ccf7-407c-9825-c71d6bc11b03" containerName="revoke-keys"
Jul 24 16:26:24.218015 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.217571    2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f9bee4d-8b84-4ddd-8740-1db054920dbe" containerName="payload-pre-processing"
Jul 24 16:26:24.220858 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.220826    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zsrwm/must-gather-dsnz7"
Jul 24 16:26:24.223304 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.223269    2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zsrwm\"/\"default-dockercfg-cl78s\""
Jul 24 16:26:24.223304 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.223267    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zsrwm\"/\"openshift-service-ca.crt\""
Jul 24 16:26:24.223693 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.223648    2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zsrwm\"/\"kube-root-ca.crt\""
Jul 24 16:26:24.232920 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.232887    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zsrwm/must-gather-dsnz7"]
Jul 24 16:26:24.334991 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.334953    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5vcb\" (UniqueName: \"kubernetes.io/projected/ec414b4e-b195-49df-83e2-501d3f32ffc7-kube-api-access-c5vcb\") pod \"must-gather-dsnz7\" (UID: \"ec414b4e-b195-49df-83e2-501d3f32ffc7\") " pod="openshift-must-gather-zsrwm/must-gather-dsnz7"
Jul 24 16:26:24.335203 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.335021    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ec414b4e-b195-49df-83e2-501d3f32ffc7-must-gather-output\") pod \"must-gather-dsnz7\" (UID: \"ec414b4e-b195-49df-83e2-501d3f32ffc7\") " pod="openshift-must-gather-zsrwm/must-gather-dsnz7"
Jul 24 16:26:24.435856 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.435818    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5vcb\" (UniqueName: \"kubernetes.io/projected/ec414b4e-b195-49df-83e2-501d3f32ffc7-kube-api-access-c5vcb\") pod \"must-gather-dsnz7\" (UID: \"ec414b4e-b195-49df-83e2-501d3f32ffc7\") " pod="openshift-must-gather-zsrwm/must-gather-dsnz7"
Jul 24 16:26:24.436088 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.435877    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ec414b4e-b195-49df-83e2-501d3f32ffc7-must-gather-output\") pod \"must-gather-dsnz7\" (UID: \"ec414b4e-b195-49df-83e2-501d3f32ffc7\") " pod="openshift-must-gather-zsrwm/must-gather-dsnz7"
Jul 24 16:26:24.436197 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.436181    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ec414b4e-b195-49df-83e2-501d3f32ffc7-must-gather-output\") pod \"must-gather-dsnz7\" (UID: \"ec414b4e-b195-49df-83e2-501d3f32ffc7\") " pod="openshift-must-gather-zsrwm/must-gather-dsnz7"
Jul 24 16:26:24.445010 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.444982    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5vcb\" (UniqueName: \"kubernetes.io/projected/ec414b4e-b195-49df-83e2-501d3f32ffc7-kube-api-access-c5vcb\") pod \"must-gather-dsnz7\" (UID: \"ec414b4e-b195-49df-83e2-501d3f32ffc7\") " pod="openshift-must-gather-zsrwm/must-gather-dsnz7"
Jul 24 16:26:24.530262 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.530164    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zsrwm/must-gather-dsnz7"
Jul 24 16:26:24.677080 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.677046    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zsrwm/must-gather-dsnz7"]
Jul 24 16:26:24.679350 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:26:24.679319    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec414b4e_b195_49df_83e2_501d3f32ffc7.slice/crio-1b56ccc1610cc5724404314f2070d2f7805380036fdb62f255888549cb474bb9 WatchSource:0}: Error finding container 1b56ccc1610cc5724404314f2070d2f7805380036fdb62f255888549cb474bb9: Status 404 returned error can't find the container with id 1b56ccc1610cc5724404314f2070d2f7805380036fdb62f255888549cb474bb9
Jul 24 16:26:24.683935 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.683913    2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 16:26:24.722429 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:24.722394    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zsrwm/must-gather-dsnz7" event={"ID":"ec414b4e-b195-49df-83e2-501d3f32ffc7","Type":"ContainerStarted","Data":"1b56ccc1610cc5724404314f2070d2f7805380036fdb62f255888549cb474bb9"}
Jul 24 16:26:25.745012 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:25.744972    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zsrwm/must-gather-dsnz7" event={"ID":"ec414b4e-b195-49df-83e2-501d3f32ffc7","Type":"ContainerStarted","Data":"1638257b4593e247bbf86194fc9621a7082f497fc57ea1337fbc24b0acbfced6"}
Jul 24 16:26:26.755209 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:26.755153    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zsrwm/must-gather-dsnz7" event={"ID":"ec414b4e-b195-49df-83e2-501d3f32ffc7","Type":"ContainerStarted","Data":"61bbd01ec7a7b0d24e958d95701b8dc8fd007d887dc3ead42e545e913f610bf4"}
Jul 24 16:26:26.772890 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:26.772815    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zsrwm/must-gather-dsnz7" podStartSLOduration=1.8263051479999999 podStartE2EDuration="2.772795193s" totalImagesPullingTime="946.490045ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:26:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:26:26.771529989 +0000 UTC m=+3015.182240627" watchObservedRunningTime="2026-07-24 16:26:26.772795193 +0000 UTC m=+3015.183505811"
Jul 24 16:26:27.258263 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:27.258231    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-j8fqw_94e9b736-6944-4781-9b40-500ec82d10e2/global-pull-secret-syncer/0.log"
Jul 24 16:26:27.401555 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:27.401517    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lvgwz_f144fc9d-a69b-4cb5-b99c-2092a7905b32/konnectivity-agent/0.log"
Jul 24 16:26:27.522714 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:27.522606    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-164.ec2.internal_b4237762195c7e99a7a069493539ac56/haproxy/0.log"
Jul 24 16:26:32.836687 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:32.835947    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6f468f5-ztsjj_e96645d7-a4df-4e93-bd25-19b1ad71f158/authorino/0.log"
Jul 24 16:26:32.869861 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:32.869829    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-hnnbz_5e7a3852-e1c7-42f7-a96a-05b6c00f18c9/manager/0.log"
Jul 24 16:26:32.994408 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:32.994361    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-zjvt9_5802f3ed-2120-4987-82e4-b3e420cf3ba3/registry-server/0.log"
Jul 24 16:26:33.204653 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:33.204547    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-zcg6b_12ea3d30-44e2-460e-813a-fa7f80aae5ba/manager/0.log"
Jul 24 16:26:33.317686 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:33.316825    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-bbt4w_7b21dbee-df26-48c4-a68d-25a8feb04e6a/manager/0.log"
Jul 24 16:26:35.571086 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:35.570994    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t2hxv_68bec78f-2086-4308-bb27-d18cb6b8c3fd/node-exporter/0.log"
Jul 24 16:26:35.601934 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:35.601906    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t2hxv_68bec78f-2086-4308-bb27-d18cb6b8c3fd/kube-rbac-proxy/0.log"
Jul 24 16:26:35.626709 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:35.626686    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t2hxv_68bec78f-2086-4308-bb27-d18cb6b8c3fd/init-textfile/0.log"
Jul 24 16:26:38.208446 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.208407    2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"]
Jul 24 16:26:38.217178 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.217147    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:38.221170 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.221131    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"]
Jul 24 16:26:38.286810 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.286753    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6723d81a-d798-4761-b613-c333b7ff900e-proc\") pod \"perf-node-gather-daemonset-f7fpb\" (UID: \"6723d81a-d798-4761-b613-c333b7ff900e\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:38.286810 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.286819    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6723d81a-d798-4761-b613-c333b7ff900e-lib-modules\") pod \"perf-node-gather-daemonset-f7fpb\" (UID: \"6723d81a-d798-4761-b613-c333b7ff900e\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:38.287085 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.286850    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6723d81a-d798-4761-b613-c333b7ff900e-podres\") pod \"perf-node-gather-daemonset-f7fpb\" (UID: \"6723d81a-d798-4761-b613-c333b7ff900e\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:38.287085 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.286878    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6723d81a-d798-4761-b613-c333b7ff900e-sys\") pod \"perf-node-gather-daemonset-f7fpb\" (UID: \"6723d81a-d798-4761-b613-c333b7ff900e\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:38.287085 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.286931    2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6q5f\" (UniqueName: \"kubernetes.io/projected/6723d81a-d798-4761-b613-c333b7ff900e-kube-api-access-p6q5f\") pod \"perf-node-gather-daemonset-f7fpb\" (UID: \"6723d81a-d798-4761-b613-c333b7ff900e\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:38.388215 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.388181    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6723d81a-d798-4761-b613-c333b7ff900e-lib-modules\") pod \"perf-node-gather-daemonset-f7fpb\" (UID: \"6723d81a-d798-4761-b613-c333b7ff900e\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:38.388215 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.388228    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6723d81a-d798-4761-b613-c333b7ff900e-podres\") pod \"perf-node-gather-daemonset-f7fpb\" (UID: \"6723d81a-d798-4761-b613-c333b7ff900e\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:38.388478 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.388261    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6723d81a-d798-4761-b613-c333b7ff900e-sys\") pod \"perf-node-gather-daemonset-f7fpb\" (UID: \"6723d81a-d798-4761-b613-c333b7ff900e\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:38.388478 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.388308    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6q5f\" (UniqueName: \"kubernetes.io/projected/6723d81a-d798-4761-b613-c333b7ff900e-kube-api-access-p6q5f\") pod \"perf-node-gather-daemonset-f7fpb\" (UID: \"6723d81a-d798-4761-b613-c333b7ff900e\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:38.388478 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.388366    2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6723d81a-d798-4761-b613-c333b7ff900e-proc\") pod \"perf-node-gather-daemonset-f7fpb\" (UID: \"6723d81a-d798-4761-b613-c333b7ff900e\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:38.388478 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.388435    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6723d81a-d798-4761-b613-c333b7ff900e-sys\") pod \"perf-node-gather-daemonset-f7fpb\" (UID: \"6723d81a-d798-4761-b613-c333b7ff900e\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:38.388478 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.388465    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6723d81a-d798-4761-b613-c333b7ff900e-proc\") pod \"perf-node-gather-daemonset-f7fpb\" (UID: \"6723d81a-d798-4761-b613-c333b7ff900e\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:38.388744 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.388465    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6723d81a-d798-4761-b613-c333b7ff900e-lib-modules\") pod \"perf-node-gather-daemonset-f7fpb\" (UID: \"6723d81a-d798-4761-b613-c333b7ff900e\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:38.388744 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.388486    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6723d81a-d798-4761-b613-c333b7ff900e-podres\") pod \"perf-node-gather-daemonset-f7fpb\" (UID: \"6723d81a-d798-4761-b613-c333b7ff900e\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:38.397697 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.397637    2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6q5f\" (UniqueName: \"kubernetes.io/projected/6723d81a-d798-4761-b613-c333b7ff900e-kube-api-access-p6q5f\") pod \"perf-node-gather-daemonset-f7fpb\" (UID: \"6723d81a-d798-4761-b613-c333b7ff900e\") " pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:38.538180 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.538073    2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:38.752100 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.752064    2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"]
Jul 24 16:26:38.756155 ip-10-0-135-164 kubenswrapper[2561]: W0724 16:26:38.756116    2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6723d81a_d798_4761_b613_c333b7ff900e.slice/crio-9eb1d783ae273e7b430b03f77f76c4a7e6bb1e125870fe278c5b53a9a69c2db3 WatchSource:0}: Error finding container 9eb1d783ae273e7b430b03f77f76c4a7e6bb1e125870fe278c5b53a9a69c2db3: Status 404 returned error can't find the container with id 9eb1d783ae273e7b430b03f77f76c4a7e6bb1e125870fe278c5b53a9a69c2db3
Jul 24 16:26:38.843228 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:38.843194    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb" event={"ID":"6723d81a-d798-4761-b613-c333b7ff900e","Type":"ContainerStarted","Data":"9eb1d783ae273e7b430b03f77f76c4a7e6bb1e125870fe278c5b53a9a69c2db3"}
Jul 24 16:26:39.852779 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:39.852723    2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb" event={"ID":"6723d81a-d798-4761-b613-c333b7ff900e","Type":"ContainerStarted","Data":"f9f90c4ddc0f115a398b5f6a09ffe6ec9b73c86249d44d607669b5c0c33e3b64"}
Jul 24 16:26:39.853276 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:39.852898    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:39.873273 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:39.873202    2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb" podStartSLOduration=1.87317608 podStartE2EDuration="1.87317608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 16:26:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:26:39.870416684 +0000 UTC m=+3028.281127301" watchObservedRunningTime="2026-07-24 16:26:39.87317608 +0000 UTC m=+3028.283886699"
Jul 24 16:26:40.512437 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:40.512411    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mf5pv_68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9/dns/0.log"
Jul 24 16:26:40.541595 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:40.541568    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mf5pv_68ecf7f0-483c-4c8d-bf84-e61e1bb89fb9/kube-rbac-proxy/0.log"
Jul 24 16:26:40.717384 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:40.717355    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-s6wn2_aa3224e2-ecb7-4164-aa12-48213a6a5210/dns-node-resolver/0.log"
Jul 24 16:26:41.269559 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:41.269530    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-697f546964-d25bc_97b984fd-3eae-4d5c-8ea2-73573fa71c96/registry/0.log"
Jul 24 16:26:41.345155 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:41.345127    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lsj8p_076c0fb1-5f86-4a0b-8725-100ea9966c41/node-ca/0.log"
Jul 24 16:26:42.655394 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:42.655365    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59fb5479dc-kcpmq_5ef6a247-e63f-4351-906b-479a8ddb2294/kube-auth-proxy/0.log"
Jul 24 16:26:42.847786 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:42.847741    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-qf26j_fa01d160-d791-465e-9488-0ae89aca50b3/payload-processing/0.log"
Jul 24 16:26:43.760699 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:43.760650    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kfwqt_9699ef5a-c5cc-4b62-9e8d-17c6f686571b/serve-healthcheck-canary/0.log"
Jul 24 16:26:44.418578 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:44.418539    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-88dmg_bad5c0b7-e451-4460-886d-d43b45404633/kube-rbac-proxy/0.log"
Jul 24 16:26:44.441979 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:44.441948    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-88dmg_bad5c0b7-e451-4460-886d-d43b45404633/exporter/0.log"
Jul 24 16:26:44.465483 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:44.465456    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-88dmg_bad5c0b7-e451-4460-886d-d43b45404633/extractor/0.log"
Jul 24 16:26:45.873603 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:45.872621    2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zsrwm/perf-node-gather-daemonset-f7fpb"
Jul 24 16:26:46.967787 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:46.967753    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-pqdnc_9a4f23fb-7e92-45fb-bc69-2a506e5fb6bd/manager/0.log"
Jul 24 16:26:47.241408 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:47.241319    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-8zx68_9eac2967-63a8-4d6e-89a0-3fd21f1cee11/manager/0.log"
Jul 24 16:26:47.519357 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:47.519228    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67bb47cc5d-9df4d_d11df53b-f05a-40f2-82af-22a66735a556/manager/0.log"
Jul 24 16:26:47.601412 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:47.601372    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gbdjq_dc394fc0-3581-48d0-ba1e-9c70876d9a73/server/0.log"
Jul 24 16:26:48.053982 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:48.053950    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b7484f4dc-r9vps_472a5da2-28a3-4250-9eb9-98f1ae8efa46/manager/0.log"
Jul 24 16:26:48.080737 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:48.080710    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b7484f4dc-r9vps_472a5da2-28a3-4250-9eb9-98f1ae8efa46/copy-manifests/0.log"
Jul 24 16:26:49.506595 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:49.506553    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-zgzsz_5cb16e01-2dfd-46f6-b147-e97092ed46d4/openshift-lws-operator/0.log"
Jul 24 16:26:49.653461 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:49.653429    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59fb5479dc-kcpmq_5ef6a247-e63f-4351-906b-479a8ddb2294/kube-auth-proxy/0.log"
Jul 24 16:26:49.811267 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:49.811152    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-qf26j_fa01d160-d791-465e-9488-0ae89aca50b3/payload-processing/0.log"
Jul 24 16:26:51.502728 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:51.502700    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-59fb5479dc-kcpmq_5ef6a247-e63f-4351-906b-479a8ddb2294/kube-auth-proxy/0.log"
Jul 24 16:26:51.653461 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:51.653425    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-qf26j_fa01d160-d791-465e-9488-0ae89aca50b3/payload-processing/0.log"
Jul 24 16:26:54.655101 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:54.655054    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-s4pwm_4cc2d387-c3f0-4b9d-a571-8d0e133ae85f/migrator/0.log"
Jul 24 16:26:54.675888 ip-10-0-135-164 kubenswrapper[2561]: I0724 16:26:54.675858    2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-s4pwm_4cc2d387-c3f0-4b9d-a571-8d0e133ae85f/graceful-termination/0.log"