Jul 16 20:32:33.911679 ip-10-0-143-245 systemd[1]: Starting Kubernetes Kubelet...
Jul 16 20:32:34.416737 ip-10-0-143-245 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 20:32:34.416737 ip-10-0-143-245 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 16 20:32:34.416737 ip-10-0-143-245 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 20:32:34.416737 ip-10-0-143-245 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 16 20:32:34.416737 ip-10-0-143-245 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 16 20:32:34.416737 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.320338    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 16 20:32:34.416737 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324735    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 20:32:34.416737 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324745    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 20:32:34.416737 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324749    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 20:32:34.416737 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324751    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 20:32:34.416737 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324754    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 20:32:34.416737 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324757    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 20:32:34.416737 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324760    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 20:32:34.416737 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324764    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 20:32:34.388521 ip-10-0-143-245 systemd[1]: Started Kubernetes Kubelet.
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324766    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324769    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324773    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324775    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324778    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324781    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324784    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324788    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324791    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324794    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324797    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324800    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324803    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324805    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324808    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324810    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324813    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324816    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 20:32:34.432072 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324818    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324821    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324823    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324826    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324828    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324831    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324834    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324836    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324839    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324841    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324845    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324849    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324853    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324856    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324859    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324863    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324866    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324869    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324872    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324874    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 20:32:34.434194 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324877    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324880    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324883    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324886    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324890    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324893    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324895    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324898    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324901    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324904    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324906    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324909    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324911    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324914    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324917    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324919    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324922    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324924    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324927    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324931    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 20:32:34.440228 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324934    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324936    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324938    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324942    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324944    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324947    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324949    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324951    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324954    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324956    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324958    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324961    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324964    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324967    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324969    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324972    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324974    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324977    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324979    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324982    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 20:32:34.452729 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324985    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324989    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324992    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324994    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324997    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.324999    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325002    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325004    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325006    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325009    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325011    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325015    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325019    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325692    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325699    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325702    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325708    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325711    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325714    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325717    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 20:32:34.600851 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325720    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325723    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325726    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325729    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325732    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325734    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325737    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325742    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325744    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325747    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325749    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325752    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325754    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325757    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325759    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325762    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325764    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325767    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325770    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325772    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 20:32:34.602250 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325775    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325777    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325780    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325782    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325785    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325787    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325790    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325792    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325795    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325798    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325800    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325803    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325807    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325811    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325814    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325817    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325820    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325823    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325826    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 20:32:34.603202 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325830    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325834    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325837    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325839    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325843    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325846    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325850    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325853    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325856    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325859    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325861    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325864    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325867    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325869    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325872    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325874    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325877    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325879    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325882    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 20:32:34.603855 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325884    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325886    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325889    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325891    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325895    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325897    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325900    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325902    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325905    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325907    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325909    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325913    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325915    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325919    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325923    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325925    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325928    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325930    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325933    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 20:32:34.604470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325937    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325940    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325943    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325946    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325949    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325952    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325954    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325956    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325959    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325962    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325964    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325967    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325969    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325972    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.325974    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327310    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327317    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327320    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327325    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 20:32:34.605027 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327328    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327332    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327334    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327337    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327340    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327342    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327345    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327350    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327354    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327371    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327374    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327376    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327379    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327382    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327385    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327387    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327390    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327393    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327395    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327398    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 20:32:34.605630 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327400    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327403    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327405    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327408    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327410    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327413    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327415    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327418    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327420    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327423    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327425    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327428    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327431    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327434    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327436    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327439    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327441    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327443    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327446    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327450    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 20:32:34.606200 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327454    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327458    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327460    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327463    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327465    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327468    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327470    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327472    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327475    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327478    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327480    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327483    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327485    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327488    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327490    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327492    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327495    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327498    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327500    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327502    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327505    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 20:32:34.606846 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327508    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327510    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327513    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327515    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327518    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327520    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327523    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327525    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327528    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327530    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327534    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327538    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327540    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327543    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327545    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327548    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327550    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327553    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327556    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327559    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 20:32:34.607448 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327561    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327565    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327568    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327570    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327572    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327575    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327580    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327583    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327586    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327588    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327591    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327594    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327597    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.327600    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327659    2576 flags.go:64] FLAG: --address="0.0.0.0"
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327666    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327675    2576 flags.go:64] FLAG: --anonymous-auth="true"
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327679    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327683    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327686    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 16 20:32:34.608030 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327690    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327694    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327698    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327702    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327706    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327709    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327712    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327715    2576 flags.go:64] FLAG: --cgroup-root=""
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327717    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327720    2576 flags.go:64] FLAG: --client-ca-file=""
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327723    2576 flags.go:64] FLAG: --cloud-provider="external"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327726    2576 flags.go:64] FLAG: --cluster-dns="[]"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327730    2576 flags.go:64] FLAG: --cluster-domain=""
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327733    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327736    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327739    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327742    2576 flags.go:64] FLAG: --container-log-max-files="5"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327745    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327748    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327751    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327754    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327758    2576 flags.go:64] FLAG: --contention-profiling="false"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327760    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327763    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 16 20:32:34.608570 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327766    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327769    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327773    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327776    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327778    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327782    2576 flags.go:64] FLAG: --enable-load-reader="false"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327784    2576 flags.go:64] FLAG: --enable-server="true"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327787    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327791    2576 flags.go:64] FLAG: --event-burst="100"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327794    2576 flags.go:64] FLAG: --event-qps="50"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327798    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327803    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327806    2576 flags.go:64] FLAG: --eviction-hard=""
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327809    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327812    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327814    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327817    2576 flags.go:64] FLAG: --eviction-soft=""
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327820    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327823    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327826    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327828    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327831    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327834    2576 flags.go:64] FLAG: --fail-swap-on="true"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327837    2576 flags.go:64] FLAG: --feature-gates=""
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327841    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 16 20:32:34.609228 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327844    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327846    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327849    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327852    2576 flags.go:64] FLAG: --healthz-port="10248"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327855    2576 flags.go:64] FLAG: --help="false"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327857    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-143-245.ec2.internal"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327860    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327863    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327866    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327869    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327872    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327875    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327878    2576 flags.go:64] FLAG: --image-service-endpoint=""
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327881    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327884    2576 flags.go:64] FLAG: --kube-api-burst="100"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327887    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327890    2576 flags.go:64] FLAG: --kube-api-qps="50"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327894    2576 flags.go:64] FLAG: --kube-reserved=""
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327898    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327900    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327903    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327906    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327909    2576 flags.go:64] FLAG: --lock-file=""
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327912    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 16 20:32:34.609897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327914    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327917    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327922    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327925    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327928    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327930    2576 flags.go:64] FLAG: --logging-format="text"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327933    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327936    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327939    2576 flags.go:64] FLAG: --manifest-url=""
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327941    2576 flags.go:64] FLAG: --manifest-url-header=""
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327946    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327948    2576 flags.go:64] FLAG: --max-open-files="1000000"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327952    2576 flags.go:64] FLAG: --max-pods="110"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327955    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327958    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327961    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327963    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327966    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327969    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327972    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327978    2576 flags.go:64] FLAG: --node-status-max-images="50"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327981    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327987    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 16 20:32:34.610581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327990    2576 flags.go:64] FLAG: --pod-cidr=""
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327993    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.327999    2576 flags.go:64] FLAG: --pod-manifest-path=""
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328002    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328006    2576 flags.go:64] FLAG: --pods-per-core="0"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328009    2576 flags.go:64] FLAG: --port="10250"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328012    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328015    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a54fdf47b062fc37"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328018    2576 flags.go:64] FLAG: --qos-reserved=""
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328020    2576 flags.go:64] FLAG: --read-only-port="10255"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328023    2576 flags.go:64] FLAG: --register-node="true"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328026    2576 flags.go:64] FLAG: --register-with-taints=""
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328029    2576 flags.go:64] FLAG: --registry-burst="10"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328032    2576 flags.go:64] FLAG: --registry-qps="5"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328035    2576 flags.go:64] FLAG: --reserved-cpus=""
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328038    2576 flags.go:64] FLAG: --reserved-memory=""
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328041    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328044    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328046    2576 flags.go:64] FLAG: --rotate-certificates="false"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328049    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328052    2576 flags.go:64] FLAG: --runonce="false"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328054    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328057    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328060    2576 flags.go:64] FLAG: --seccomp-default="false"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328063    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 16 20:32:34.614266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328065    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328068    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328071    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328074    2576 flags.go:64] FLAG: --storage-driver-password="root"
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328076    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328079    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328082    2576 flags.go:64] FLAG: --storage-driver-user="root"
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328085    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328088    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328092    2576 flags.go:64] FLAG: --system-cgroups=""
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328095    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328101    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328104    2576 flags.go:64] FLAG: --tls-cert-file=""
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328107    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328111    2576 flags.go:64] FLAG: --tls-min-version=""
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328114    2576 flags.go:64] FLAG: --tls-private-key-file=""
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328116    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328119    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328122    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328125    2576 flags.go:64] FLAG: --v="2"
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328129    2576 flags.go:64] FLAG: --version="false"
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328132    2576 flags.go:64] FLAG: --vmodule=""
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328136    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.328139    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328257    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 20:32:34.614989 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328260    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328263    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328266    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328269    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328272    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328274    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328277    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328279    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328282    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328284    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328287    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328291    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328295    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328297    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328300    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328303    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328308    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328311    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328315    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328318    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 20:32:34.765295 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328321    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328323    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328326    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328328    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328331    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328333    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328336    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328338    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328341    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328343    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328346    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328348    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328351    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328353    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328373    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328376    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328379    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328381    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328384    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328386    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 20:32:34.765910 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328389    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328391    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328394    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328397    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328399    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328403    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328406    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328410    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328418    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328421    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328425    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328428    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328430    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328433    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328436    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328438    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328441    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328444    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328446    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 20:32:34.766474 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328449    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328451    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328454    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328457    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328459    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328462    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328465    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328467    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328470    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328472    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328474    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328477    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328480    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328483    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328485    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328488    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328490    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328493    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328495    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328498    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 20:32:34.767129 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328500    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328504    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328507    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328510    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328513    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328516    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328518    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328521    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328524    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328526    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328529    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328531    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328534    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328536    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328539    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328541    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328543    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328546    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 20:32:34.767672 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.328549    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 20:32:34.768183 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.329130    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 20:32:34.768183 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.337578    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 16 20:32:34.768183 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.337600    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 16 20:32:34.768183 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.337626    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 16 20:32:34.768183 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.337656    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 16 20:32:34.768183 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337857    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 20:32:34.768183 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337867    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 20:32:34.768183 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337871    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 20:32:34.768183 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337876    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 20:32:34.768183 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337880    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 20:32:34.768183 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337885    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 20:32:34.768183 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337889    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 20:32:34.768183 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337894    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 20:32:34.768183 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337898    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 20:32:34.768183 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337902    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337907    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337910    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337915    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337919    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337924    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337928    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337932    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337936    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337940    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337945    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337949    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337953    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337957    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337961    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337966    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337970    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337974    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337978    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337982    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 20:32:34.768625 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337986    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337989    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337991    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337994    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.337997    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338000    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338003    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338006    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338010    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338014    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338017    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338020    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338023    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338027    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338030    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338033    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338036    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338038    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338041    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 20:32:34.769178 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338044    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338046    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338048    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338053    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338057    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338061    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338064    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338067    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338070    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338072    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338075    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338077    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338080    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338082    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338085    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338088    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338090    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338093    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338096    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338102    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 20:32:34.769703 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338104    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338107    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338110    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338112    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338115    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338118    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338120    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338123    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338125    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338128    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338130    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338136    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338139    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338142    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338144    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338147    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338150    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338152    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338155    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 20:32:34.770273 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338157    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 20:32:34.770788 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338160    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 20:32:34.770788 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338162    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 20:32:34.770788 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338165    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 20:32:34.770788 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338167    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 20:32:34.770788 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338170    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 20:32:34.770788 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338172    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 20:32:34.770788 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338175    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 20:32:34.770788 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338177    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 20:32:34.770788 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338180    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 20:32:34.770788 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338182    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 20:32:34.770788 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338186    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 20:32:34.770788 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.338193    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 20:32:34.770788 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338297    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 16 20:32:34.770788 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338303    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 16 20:32:34.770788 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338306    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338309    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338312    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338315    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338318    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338320    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338323    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338325    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338329    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338331    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338334    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338337    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338340    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338342    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338344    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338347    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338350    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338353    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338355    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 16 20:32:34.771234 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338376    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338380    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338382    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338385    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338387    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338390    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338393    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338395    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338398    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338402    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338405    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338410    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338412    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338415    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338418    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338420    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338423    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338427    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338431    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338434    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 16 20:32:34.771753 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338437    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338440    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338443    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338445    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338449    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338451    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338454    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338457    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338459    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338462    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338465    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338467    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338470    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338472    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338475    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338478    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338480    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338483    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338485    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338488    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 16 20:32:34.772310 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338490    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338494    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338497    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338500    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338503    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338505    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338508    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338511    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338513    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338516    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338518    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338521    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338523    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338526    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338528    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338531    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338533    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338537    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338539    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338542    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 16 20:32:34.772881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338544    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 16 20:32:34.773470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338547    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 16 20:32:34.773470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338550    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 16 20:32:34.773470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338552    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 16 20:32:34.773470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338554    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 16 20:32:34.773470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338557    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 16 20:32:34.773470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338560    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 16 20:32:34.773470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338562    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 16 20:32:34.773470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338565    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 16 20:32:34.773470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338567    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 16 20:32:34.773470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338570    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 16 20:32:34.773470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338572    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 16 20:32:34.773470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338576    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 16 20:32:34.773470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338583    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 16 20:32:34.773470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338586    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 16 20:32:34.773470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338590    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 16 20:32:34.773470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338594    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 16 20:32:34.773470 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:34.338598    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 16 20:32:34.773930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.338603    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 16 20:32:34.773930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.338695    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 16 20:32:34.773930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.340832    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 16 20:32:34.773930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.342028    2576 server.go:1019] "Starting client certificate rotation"
Jul 16 20:32:34.773930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.342121    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 20:32:34.773930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.342150    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 16 20:32:34.773930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.344289    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 20:32:34.773930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.345585    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 16 20:32:34.773930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.354085    2576 log.go:25] "Validated CRI v1 runtime API"
Jul 16 20:32:34.773930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.356041    2576 log.go:25] "Validated CRI v1 image API"
Jul 16 20:32:34.773930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.356817    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 16 20:32:34.773930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.360140    2576 fs.go:135] Filesystem UUIDs: map[0496620e-eda1-4778-8b54-8b3fbca975bf:/dev/nvme0n1p4 6046ac17-ff2c-4fe0-96fa-fab36293fae6:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 16 20:32:34.774301 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.360159    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 16 20:32:34.774386 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.365330    2576 manager.go:216] Machine: {Timestamp:2026-07-16 20:32:34.36406921 +0000 UTC m=+0.330003433 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106194 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ddbff9d1bc684b12a6c902cec5948 SystemUUID:ec2ddbff-9d1b-c684-b12a-6c902cec5948 BootID:217418a4-4add-4a41-9f85-e77ee37f84dc Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:51:09:19:17:59 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:51:09:19:17:59 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:21:42:e5:65:70 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 16 20:32:34.774386 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.365437    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 16 20:32:34.774386 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.365506    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 16 20:32:34.774386 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.368622    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 16 20:32:34.774386 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.368649    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-245.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 16 20:32:34.774386 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.369802    2576 topology_manager.go:138] "Creating topology manager with none policy"
Jul 16 20:32:34.774386 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.369820    2576 container_manager_linux.go:309] "Creating device plugin manager"
Jul 16 20:32:34.774386 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.369828    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.370500    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.370998    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.372313    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.374963    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.375069    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.375095    2576 kubelet.go:486] "Attempting to sync node with API server"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.375105    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.375116    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.375123    2576 kubelet.go:398] "Adding apiserver pod source"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.375132    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.375411    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.375426    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.380442    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.381920    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.381947    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.382583    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.382597    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.382603    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.382608    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.382616    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.382621    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.382628    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.382634    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.382640    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.382645    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.382653    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.382658    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.383761    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.383770    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.387804    2576 server.go:1268] "Started kubelet"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.387858    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.387898    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.387983    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.388091    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-245.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.388159    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-245.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.388304    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.389182    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.389730    2576 server.go:310] "Adding debug handlers to kubelet server"
Jul 16 20:32:34.774693 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.394774    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.395784    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.395792    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.395873    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-245.ec2.internal\" not found"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.395256    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-245.ec2.internal.18c2df540779649b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-245.ec2.internal,UID:ip-10-0-143-245.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-245.ec2.internal,},FirstTimestamp:2026-07-16 20:32:34.387780763 +0000 UTC m=+0.353714987,LastTimestamp:2026-07-16 20:32:34.387780763 +0000 UTC m=+0.353714987,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-245.ec2.internal,}"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.396519    2576 factory.go:55] Registering systemd factory
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.396535    2576 factory.go:223] Registration of the systemd container factory successfully
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.396541    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.396561    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.396572    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.396680    2576 reconstruct.go:97] "Volume reconstruction finished"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.396688    2576 reconciler.go:29] "Reconciler: start to sync state"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.396749    2576 factory.go:153] Registering CRI-O factory
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.396761    2576 factory.go:223] Registration of the crio container factory successfully
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.396833    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.396853    2576 factory.go:103] Registering Raw factory
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.396873    2576 manager.go:1196] Started watching for new ooms in manager
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.397202    2576 manager.go:349] Starting recovery of all containers
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.401170    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-245.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.401286    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.409071    2576 manager.go:354] Recovery completed
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.414160    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.417872    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.417901    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.417913    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasSufficientPID"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.418475    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.418485    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.418500    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.418499    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.419231    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.419254    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.419264    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasSufficientPID"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.419784    2576 policy_none.go:49] "None policy: Start"
Jul 16 20:32:34.778132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.419805    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.419736    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-245.ec2.internal.18c2df540944b9b9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-245.ec2.internal,UID:ip-10-0-143-245.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-245.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-245.ec2.internal,},FirstTimestamp:2026-07-16 20:32:34.417883577 +0000 UTC m=+0.383817809,LastTimestamp:2026-07-16 20:32:34.417883577 +0000 UTC m=+0.383817809,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-245.ec2.internal,}"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.419816    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.420852    2576 policy_none.go:47] "Start"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.422726    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kbgl8"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.428565    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kbgl8"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.429560    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-245.ec2.internal.18c2df5409451857  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-245.ec2.internal,UID:ip-10-0-143-245.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-245.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-245.ec2.internal,},FirstTimestamp:2026-07-16 20:32:34.417907799 +0000 UTC m=+0.383842023,LastTimestamp:2026-07-16 20:32:34.417907799 +0000 UTC m=+0.383842023,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-245.ec2.internal,}"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.478446    2576 manager.go:337] "Starting Device Plugin manager"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.478490    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.478499    2576 server.go:87] "Starting device plugin registration server"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.478692    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.478704    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.478818    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.478886    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.478893    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.479330    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.479379    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-245.ec2.internal\" not found"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.521142    2576 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2612 into the system container \"/system.slice\": <nil>"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.579080    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.579822    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.579844    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.579855    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasSufficientPID"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.579877    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.586765    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.008734 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.586780    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-245.ec2.internal\": node \"ip-10-0-143-245.ec2.internal\" not found"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.612286    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-245.ec2.internal\" not found"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.681993    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.683070    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.683101    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.683117    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.683146    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.685347    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.714702    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-245.ec2.internal\" not found"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.783857    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-245.ec2.internal"]
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.783905    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.784620    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.784642    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.784652    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasSufficientPID"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.785973    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.786111    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.786146    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.786578    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.786586    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.786598    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.786607    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasSufficientPID"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.786608    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.786622    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasSufficientPID"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.788017    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.788039    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.788692    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasSufficientMemory"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.788715    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasNoDiskPressure"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.788728    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeHasSufficientPID"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.800998    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d3d7cdf18b301318dd0bf2a5e9344c0d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal\" (UID: \"d3d7cdf18b301318dd0bf2a5e9344c0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.010755 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.801018    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3d7cdf18b301318dd0bf2a5e9344c0d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal\" (UID: \"d3d7cdf18b301318dd0bf2a5e9344c0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.011617 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.801037    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5861fc40cf4c0bb1ecbf0801c1a2d731-config\") pod \"kube-apiserver-proxy-ip-10-0-143-245.ec2.internal\" (UID: \"5861fc40cf4c0bb1ecbf0801c1a2d731\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.011617 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.814977    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-245.ec2.internal\" not found"
Jul 16 20:32:35.011617 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.818860    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-245.ec2.internal\" not found" node="ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.011617 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.823198    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-245.ec2.internal\" not found" node="ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.011617 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.901345    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d3d7cdf18b301318dd0bf2a5e9344c0d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal\" (UID: \"d3d7cdf18b301318dd0bf2a5e9344c0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.011617 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.901378    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3d7cdf18b301318dd0bf2a5e9344c0d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal\" (UID: \"d3d7cdf18b301318dd0bf2a5e9344c0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.011617 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.901395    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5861fc40cf4c0bb1ecbf0801c1a2d731-config\") pod \"kube-apiserver-proxy-ip-10-0-143-245.ec2.internal\" (UID: \"5861fc40cf4c0bb1ecbf0801c1a2d731\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.011617 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.901433    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5861fc40cf4c0bb1ecbf0801c1a2d731-config\") pod \"kube-apiserver-proxy-ip-10-0-143-245.ec2.internal\" (UID: \"5861fc40cf4c0bb1ecbf0801c1a2d731\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.011617 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.901460    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3d7cdf18b301318dd0bf2a5e9344c0d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal\" (UID: \"d3d7cdf18b301318dd0bf2a5e9344c0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.011617 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:34.901464    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d3d7cdf18b301318dd0bf2a5e9344c0d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal\" (UID: \"d3d7cdf18b301318dd0bf2a5e9344c0d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.011617 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:34.915450    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-245.ec2.internal\" not found"
Jul 16 20:32:35.226265 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:35.015888    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-245.ec2.internal\" not found"
Jul 16 20:32:35.226265 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:35.116380    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-245.ec2.internal\" not found"
Jul 16 20:32:35.226265 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.121846    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.226265 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.130317    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.226265 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.206351    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 20:32:35.226265 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.222277    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.296342    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.308119    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.309070    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-245.ec2.internal"
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.316826    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.342578    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.342672    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.342701    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.342711    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.375863    2576 apiserver.go:52] "Watching apiserver"
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.394127    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.394415    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc","openshift-cluster-node-tuning-operator/tuned-nwjsv","openshift-dns/node-resolver-9jr72","openshift-image-registry/node-ca-6vjn8","openshift-multus/multus-additional-cni-plugins-v4qls","openshift-multus/network-metrics-daemon-pr8rp","kube-system/global-pull-secret-syncer-q77jc","kube-system/kube-apiserver-proxy-ip-10-0-143-245.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal","openshift-multus/multus-w9582","openshift-network-diagnostics/network-check-target-xxh76","openshift-network-operator/iptables-alerter-67v9j","openshift-ovn-kubernetes/ovnkube-node-9lz24","kube-system/konnectivity-agent-r5zjc"]
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.395854    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.395878    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:35.395930    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.397404    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.398678    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9jr72"
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.399602    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.399843    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.399860    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-df7rt\""
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.401032    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.401036    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.401213    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2frcx\""
Jul 16 20:32:35.420670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.403345    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.421965 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.403343    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-modprobe-d\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.421965 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.403503    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-sysconfig\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.421965 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.403534    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-sysctl-conf\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.421965 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.403556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-systemd\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.421965 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.403581    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-sys\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.421965 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.403603    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-tmp\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.421965 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.403622    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-run\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.421965 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.403647    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcz7l\" (UniqueName: \"kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l\") pod \"network-check-target-xxh76\" (UID: \"30258292-80a3-4166-a0e5-1ad9bec7d240\") " pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:35.421965 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.403697    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-kubernetes\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.421965 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.403728    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-tuned\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.421965 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.403753    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-sysctl-d\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.421965 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.403767    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-host\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.421965 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.403797    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrrtb\" (UniqueName: \"kubernetes.io/projected/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-kube-api-access-rrrtb\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.421965 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.403813    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-lib-modules\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.403836    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-var-lib-kubelet\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.404872    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.405435    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.405445    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.405447    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.405447    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.405568    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.405765    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.405807    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-d2w8n\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.406020    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.406034    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-67v9j"
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.407165    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.407221    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:35.407240    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.407326    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.407794    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.407791    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.407977    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.408024    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.408035    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ck495\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.408044    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.408139    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-64bd6\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.408160    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.408433    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.409812    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.411193    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 16 20:32:35.422709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.411202    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:35.423898 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:35.411249    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:32:35.423898 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.411430    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-djmkv\""
Jul 16 20:32:35.423898 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.412198    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 16 20:32:35.423898 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.412243    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 16 20:32:35.423898 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.412294    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 16 20:32:35.423898 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.412439    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-prnz2\""
Jul 16 20:32:35.423898 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.412938    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6vjn8"
Jul 16 20:32:35.423898 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.415176    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7jngz\""
Jul 16 20:32:35.423898 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.415268    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 16 20:32:35.423898 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.415276    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 16 20:32:35.423898 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.420379    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r5zjc"
Jul 16 20:32:35.423898 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.420496    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 16 20:32:35.424850 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.424728    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 16 20:32:35.425168 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.425045    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wrmmq\""
Jul 16 20:32:35.425168 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.425138    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 16 20:32:35.620539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.432085    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hbcbs"
Jul 16 20:32:35.620539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.435184    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 20:27:34 +0000 UTC" deadline="2028-04-13 21:52:13.726970552 +0000 UTC"
Jul 16 20:32:35.620539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.435219    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15289h19m38.291754107s"
Jul 16 20:32:35.620539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.441151    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hbcbs"
Jul 16 20:32:35.620539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.452641    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 20:32:35.620539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.497582    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 16 20:32:35.620539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.503972    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.620539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.503993    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3b501cab-446b-4e3d-b9f8-3e2958d42293-serviceca\") pod \"node-ca-6vjn8\" (UID: \"3b501cab-446b-4e3d-b9f8-3e2958d42293\") " pod="openshift-image-registry/node-ca-6vjn8"
Jul 16 20:32:35.620539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504009    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91bc89fb-502f-4c3e-8aef-1c2f07998932-konnectivity-ca\") pod \"konnectivity-agent-r5zjc\" (UID: \"91bc89fb-502f-4c3e-8aef-1c2f07998932\") " pod="kube-system/konnectivity-agent-r5zjc"
Jul 16 20:32:35.620539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504024    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c77ecce4-9890-4cef-bf49-ebac541c6d6d-cnibin\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.620539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504048    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-tuned\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.620539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504081    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-cnibin\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.620539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504113    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-run-multus-certs\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.620539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504129    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-etc-kubernetes\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.620539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504146    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6ef282cd-38f3-4e0d-b862-b9d510e70226-kubelet-config\") pod \"global-pull-secret-syncer-q77jc\" (UID: \"6ef282cd-38f3-4e0d-b862-b9d510e70226\") " pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:35.620539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504215    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-multus-daemon-config\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.620539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504237    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs\") pod \"network-metrics-daemon-pr8rp\" (UID: \"2d50105b-d05a-487b-bfa1-2b5167015795\") " pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:35.624192 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504261    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-run-netns\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.624192 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504276    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/53dad2b0-4e1e-4fee-8022-8aceabe32607-host-slash\") pod \"iptables-alerter-67v9j\" (UID: \"53dad2b0-4e1e-4fee-8022-8aceabe32607\") " pod="openshift-network-operator/iptables-alerter-67v9j"
Jul 16 20:32:35.624192 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504299    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm6zh\" (UniqueName: \"kubernetes.io/projected/53dad2b0-4e1e-4fee-8022-8aceabe32607-kube-api-access-fm6zh\") pod \"iptables-alerter-67v9j\" (UID: \"53dad2b0-4e1e-4fee-8022-8aceabe32607\") " pod="openshift-network-operator/iptables-alerter-67v9j"
Jul 16 20:32:35.624192 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504329    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-device-dir\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.624192 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504348    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-268t5\" (UniqueName: \"kubernetes.io/projected/8902bd3a-2b67-43d5-ae89-2c0271731c9a-kube-api-access-268t5\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.624192 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504364    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-run-openvswitch\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.624192 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504405    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-var-lib-cni-multus\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.624192 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504428    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-socket-dir\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.624192 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504446    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-slash\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.624192 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504464    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-run-ovn\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.624192 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504478    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c77ecce4-9890-4cef-bf49-ebac541c6d6d-os-release\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.624192 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504494    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n548\" (UniqueName: \"kubernetes.io/projected/c77ecce4-9890-4cef-bf49-ebac541c6d6d-kube-api-access-5n548\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.624192 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504511    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-run\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.624722 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504527    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcz7l\" (UniqueName: \"kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l\") pod \"network-check-target-xxh76\" (UID: \"30258292-80a3-4166-a0e5-1ad9bec7d240\") " pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:35.624722 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504555    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-kubernetes\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.624722 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504579    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-multus-cni-dir\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.624722 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504596    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-hostroot\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.624722 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504617    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lhjb\" (UniqueName: \"kubernetes.io/projected/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-kube-api-access-5lhjb\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.624722 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504655    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-kubernetes\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.624722 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504680    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-node-log\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.624722 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504699    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-host\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.624722 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504705    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-run\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.624722 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504734    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-cni-binary-copy\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.624722 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504757    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-host\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.624722 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504781    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-run-netns\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.624722 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504797    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-multus-conf-dir\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.624722 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504823    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-sysconfig\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.625569 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504840    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-etc-selinux\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.625569 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504855    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6df4aad3-be34-4b4a-bd20-4b46e9d53737-hosts-file\") pod \"node-resolver-9jr72\" (UID: \"6df4aad3-be34-4b4a-bd20-4b46e9d53737\") " pod="openshift-dns/node-resolver-9jr72"
Jul 16 20:32:35.625569 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504871    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c77ecce4-9890-4cef-bf49-ebac541c6d6d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.625569 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504896    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/53dad2b0-4e1e-4fee-8022-8aceabe32607-iptables-alerter-script\") pod \"iptables-alerter-67v9j\" (UID: \"53dad2b0-4e1e-4fee-8022-8aceabe32607\") " pod="openshift-network-operator/iptables-alerter-67v9j"
Jul 16 20:32:35.625569 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504922    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-sys\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.625569 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504931    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-sysconfig\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.625569 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504947    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-registration-dir\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.625569 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504973    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c77ecce4-9890-4cef-bf49-ebac541c6d6d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.625569 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504980    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-sys\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.625569 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.504998    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-var-lib-kubelet\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.625569 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505049    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gztq2\" (UniqueName: \"kubernetes.io/projected/2d50105b-d05a-487b-bfa1-2b5167015795-kube-api-access-gztq2\") pod \"network-metrics-daemon-pr8rp\" (UID: \"2d50105b-d05a-487b-bfa1-2b5167015795\") " pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:35.625569 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505093    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6df4aad3-be34-4b4a-bd20-4b46e9d53737-tmp-dir\") pod \"node-resolver-9jr72\" (UID: \"6df4aad3-be34-4b4a-bd20-4b46e9d53737\") " pod="openshift-dns/node-resolver-9jr72"
Jul 16 20:32:35.625569 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505142    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-run-ovn-kubernetes\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.625569 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505181    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-cni-netd\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.626189 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505206    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/380d1ee9-2151-419f-a44f-35445c5abba7-env-overrides\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.626189 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505230    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/380d1ee9-2151-419f-a44f-35445c5abba7-ovnkube-script-lib\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.626189 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505251    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-sysctl-d\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.626189 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505299    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrrtb\" (UniqueName: \"kubernetes.io/projected/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-kube-api-access-rrrtb\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.626189 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505317    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-sysctl-d\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.626189 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505349    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-system-cni-dir\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.626189 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505383    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3b501cab-446b-4e3d-b9f8-3e2958d42293-host\") pod \"node-ca-6vjn8\" (UID: \"3b501cab-446b-4e3d-b9f8-3e2958d42293\") " pod="openshift-image-registry/node-ca-6vjn8"
Jul 16 20:32:35.626189 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505403    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-var-lib-openvswitch\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.626189 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505427    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.626189 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505443    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/380d1ee9-2151-419f-a44f-35445c5abba7-ovn-node-metrics-cert\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.626189 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505469    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c77ecce4-9890-4cef-bf49-ebac541c6d6d-cni-binary-copy\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.626189 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505487    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-kubelet\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.626189 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505501    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-os-release\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.626189 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505518    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-modprobe-d\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.626778 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505538    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-sysctl-conf\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.626778 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505560    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmncw\" (UniqueName: \"kubernetes.io/projected/3b501cab-446b-4e3d-b9f8-3e2958d42293-kube-api-access-hmncw\") pod \"node-ca-6vjn8\" (UID: \"3b501cab-446b-4e3d-b9f8-3e2958d42293\") " pod="openshift-image-registry/node-ca-6vjn8"
Jul 16 20:32:35.626778 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505583    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91bc89fb-502f-4c3e-8aef-1c2f07998932-agent-certs\") pod \"konnectivity-agent-r5zjc\" (UID: \"91bc89fb-502f-4c3e-8aef-1c2f07998932\") " pod="kube-system/konnectivity-agent-r5zjc"
Jul 16 20:32:35.626778 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505607    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-etc-openvswitch\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.626778 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505615    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-modprobe-d\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.626778 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505626    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9w5z\" (UniqueName: \"kubernetes.io/projected/380d1ee9-2151-419f-a44f-35445c5abba7-kube-api-access-v9w5z\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.626778 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505676    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-tmp\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.626778 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505709    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-var-lib-cni-bin\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.626778 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505731    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqxkk\" (UniqueName: \"kubernetes.io/projected/6df4aad3-be34-4b4a-bd20-4b46e9d53737-kube-api-access-qqxkk\") pod \"node-resolver-9jr72\" (UID: \"6df4aad3-be34-4b4a-bd20-4b46e9d53737\") " pod="openshift-dns/node-resolver-9jr72"
Jul 16 20:32:35.626778 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505740    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-sysctl-conf\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.626778 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505748    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-systemd-units\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.626778 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505813    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-cni-bin\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.626778 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505840    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c77ecce4-9890-4cef-bf49-ebac541c6d6d-system-cni-dir\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.626778 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505865    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret\") pod \"global-pull-secret-syncer-q77jc\" (UID: \"6ef282cd-38f3-4e0d-b862-b9d510e70226\") " pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:35.627332 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505885    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-multus-socket-dir-parent\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.627332 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505923    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-run-k8s-cni-cncf-io\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.627332 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505950    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-log-socket\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.627332 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.505978    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c77ecce4-9890-4cef-bf49-ebac541c6d6d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.627332 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.506024    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-lib-modules\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.627332 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.506051    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-var-lib-kubelet\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.627332 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.506079    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-systemd\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.627332 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.506110    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-lib-modules\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.627332 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.506107    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-sys-fs\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.627332 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.506145    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-var-lib-kubelet\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.627332 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.506147    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-run-systemd\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.627332 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.506176    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/380d1ee9-2151-419f-a44f-35445c5abba7-ovnkube-config\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.627332 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.506193    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6ef282cd-38f3-4e0d-b862-b9d510e70226-dbus\") pod \"global-pull-secret-syncer-q77jc\" (UID: \"6ef282cd-38f3-4e0d-b862-b9d510e70226\") " pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:35.627332 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.506194    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-systemd\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.627930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.506509    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 16 20:32:35.627930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.509547    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-tmp\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.627930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.509571    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-etc-tuned\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.627930 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:35.512890    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 20:32:35.627930 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:35.512910    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 20:32:35.627930 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:35.512919    2576 projected.go:196] Error preparing data for projected volume kube-api-access-kcz7l for pod openshift-network-diagnostics/network-check-target-xxh76: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:32:35.627930 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:35.512976    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l podName:30258292-80a3-4166-a0e5-1ad9bec7d240 nodeName:}" failed. No retries permitted until 2026-07-16 20:32:36.012959681 +0000 UTC m=+1.978893923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kcz7l" (UniqueName: "kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l") pod "network-check-target-xxh76" (UID: "30258292-80a3-4166-a0e5-1ad9bec7d240") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:32:35.627930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.515773    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrrtb\" (UniqueName: \"kubernetes.io/projected/c6be9422-38e1-4a0d-b2e9-fd3f77cea68f-kube-api-access-rrrtb\") pod \"tuned-nwjsv\" (UID: \"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f\") " pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.627930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.606614    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-kubelet\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.627930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.606663    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-kubelet\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.627930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.606685    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-os-release\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.627930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.606703    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmncw\" (UniqueName: \"kubernetes.io/projected/3b501cab-446b-4e3d-b9f8-3e2958d42293-kube-api-access-hmncw\") pod \"node-ca-6vjn8\" (UID: \"3b501cab-446b-4e3d-b9f8-3e2958d42293\") " pod="openshift-image-registry/node-ca-6vjn8"
Jul 16 20:32:35.627930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.606719    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91bc89fb-502f-4c3e-8aef-1c2f07998932-agent-certs\") pod \"konnectivity-agent-r5zjc\" (UID: \"91bc89fb-502f-4c3e-8aef-1c2f07998932\") " pod="kube-system/konnectivity-agent-r5zjc"
Jul 16 20:32:35.627930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.606734    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-etc-openvswitch\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.627930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.606748    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9w5z\" (UniqueName: \"kubernetes.io/projected/380d1ee9-2151-419f-a44f-35445c5abba7-kube-api-access-v9w5z\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.628600 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.606763    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-var-lib-cni-bin\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.628600 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.606791    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqxkk\" (UniqueName: \"kubernetes.io/projected/6df4aad3-be34-4b4a-bd20-4b46e9d53737-kube-api-access-qqxkk\") pod \"node-resolver-9jr72\" (UID: \"6df4aad3-be34-4b4a-bd20-4b46e9d53737\") " pod="openshift-dns/node-resolver-9jr72"
Jul 16 20:32:35.628600 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.606823    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-etc-openvswitch\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.628600 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.606856    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-var-lib-cni-bin\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.628600 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.606915    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-systemd-units\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.628600 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.606934    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-cni-bin\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.628600 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.606950    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c77ecce4-9890-4cef-bf49-ebac541c6d6d-system-cni-dir\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.628600 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.606953    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-os-release\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.628600 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.606967    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret\") pod \"global-pull-secret-syncer-q77jc\" (UID: \"6ef282cd-38f3-4e0d-b862-b9d510e70226\") " pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:35.628600 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.606966    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-systemd-units\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.628600 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607003    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-multus-socket-dir-parent\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.628600 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607013    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c77ecce4-9890-4cef-bf49-ebac541c6d6d-system-cni-dir\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.628600 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:35.607021    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 20:32:35.628600 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-run-k8s-cni-cncf-io\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.628600 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607026    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-cni-bin\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.629120 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607051    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-multus-socket-dir-parent\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.629120 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607042    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-log-socket\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.629120 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607085    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-log-socket\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.629120 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607088    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-run-k8s-cni-cncf-io\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.629120 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:35.607091    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret podName:6ef282cd-38f3-4e0d-b862-b9d510e70226 nodeName:}" failed. No retries permitted until 2026-07-16 20:32:36.107071235 +0000 UTC m=+2.073005468 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret") pod "global-pull-secret-syncer-q77jc" (UID: "6ef282cd-38f3-4e0d-b862-b9d510e70226") : object "kube-system"/"original-pull-secret" not registered
Jul 16 20:32:35.629120 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607120    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c77ecce4-9890-4cef-bf49-ebac541c6d6d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.629120 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607155    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-sys-fs\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.629120 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607178    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-run-systemd\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.629120 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607201    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/380d1ee9-2151-419f-a44f-35445c5abba7-ovnkube-config\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.629120 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607224    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6ef282cd-38f3-4e0d-b862-b9d510e70226-dbus\") pod \"global-pull-secret-syncer-q77jc\" (UID: \"6ef282cd-38f3-4e0d-b862-b9d510e70226\") " pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:35.629120 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607260    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-run-systemd\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.629120 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607285    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-sys-fs\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.629120 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.629120 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607334    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6ef282cd-38f3-4e0d-b862-b9d510e70226-dbus\") pod \"global-pull-secret-syncer-q77jc\" (UID: \"6ef282cd-38f3-4e0d-b862-b9d510e70226\") " pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:35.629623 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607347    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3b501cab-446b-4e3d-b9f8-3e2958d42293-serviceca\") pod \"node-ca-6vjn8\" (UID: \"3b501cab-446b-4e3d-b9f8-3e2958d42293\") " pod="openshift-image-registry/node-ca-6vjn8"
Jul 16 20:32:35.629623 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607368    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91bc89fb-502f-4c3e-8aef-1c2f07998932-konnectivity-ca\") pod \"konnectivity-agent-r5zjc\" (UID: \"91bc89fb-502f-4c3e-8aef-1c2f07998932\") " pod="kube-system/konnectivity-agent-r5zjc"
Jul 16 20:32:35.629623 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607405    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c77ecce4-9890-4cef-bf49-ebac541c6d6d-cnibin\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.629623 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607406    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.629623 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607431    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-cnibin\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.629623 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607458    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-run-multus-certs\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.629623 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607482    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-etc-kubernetes\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.629623 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607504    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-cnibin\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.629623 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607459    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c77ecce4-9890-4cef-bf49-ebac541c6d6d-cnibin\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.629623 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607507    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6ef282cd-38f3-4e0d-b862-b9d510e70226-kubelet-config\") pod \"global-pull-secret-syncer-q77jc\" (UID: \"6ef282cd-38f3-4e0d-b862-b9d510e70226\") " pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:35.629623 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607552    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6ef282cd-38f3-4e0d-b862-b9d510e70226-kubelet-config\") pod \"global-pull-secret-syncer-q77jc\" (UID: \"6ef282cd-38f3-4e0d-b862-b9d510e70226\") " pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:35.629623 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607567    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-multus-daemon-config\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.629623 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607577    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-etc-kubernetes\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.629623 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607594    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-run-multus-certs\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.629623 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607595    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs\") pod \"network-metrics-daemon-pr8rp\" (UID: \"2d50105b-d05a-487b-bfa1-2b5167015795\") " pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:35.630123 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607635    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-run-netns\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.630123 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:35.607660    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:32:35.630123 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607688    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/53dad2b0-4e1e-4fee-8022-8aceabe32607-host-slash\") pod \"iptables-alerter-67v9j\" (UID: \"53dad2b0-4e1e-4fee-8022-8aceabe32607\") " pod="openshift-network-operator/iptables-alerter-67v9j"
Jul 16 20:32:35.630123 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607697    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c77ecce4-9890-4cef-bf49-ebac541c6d6d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.630123 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:35.607714    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs podName:2d50105b-d05a-487b-bfa1-2b5167015795 nodeName:}" failed. No retries permitted until 2026-07-16 20:32:36.107698883 +0000 UTC m=+2.073633096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs") pod "network-metrics-daemon-pr8rp" (UID: "2d50105b-d05a-487b-bfa1-2b5167015795") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:32:35.630123 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607659    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/53dad2b0-4e1e-4fee-8022-8aceabe32607-host-slash\") pod \"iptables-alerter-67v9j\" (UID: \"53dad2b0-4e1e-4fee-8022-8aceabe32607\") " pod="openshift-network-operator/iptables-alerter-67v9j"
Jul 16 20:32:35.630123 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607740    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/380d1ee9-2151-419f-a44f-35445c5abba7-ovnkube-config\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.630123 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607771    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm6zh\" (UniqueName: \"kubernetes.io/projected/53dad2b0-4e1e-4fee-8022-8aceabe32607-kube-api-access-fm6zh\") pod \"iptables-alerter-67v9j\" (UID: \"53dad2b0-4e1e-4fee-8022-8aceabe32607\") " pod="openshift-network-operator/iptables-alerter-67v9j"
Jul 16 20:32:35.630123 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607729    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-run-netns\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.630123 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607801    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-device-dir\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.630123 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607820    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-268t5\" (UniqueName: \"kubernetes.io/projected/8902bd3a-2b67-43d5-ae89-2c0271731c9a-kube-api-access-268t5\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.630123 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607841    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-run-openvswitch\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.630123 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607860    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-var-lib-cni-multus\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.630123 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607872    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-device-dir\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.630624 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607880    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-socket-dir\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.630624 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607914    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-slash\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.630624 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607940    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-run-ovn\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.630624 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607942    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-run-openvswitch\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.630624 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608037    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-run-ovn\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.630624 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608048    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c77ecce4-9890-4cef-bf49-ebac541c6d6d-os-release\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.630624 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607996    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-var-lib-cni-multus\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.630624 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608014    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-slash\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.630624 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608020    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-socket-dir\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.630624 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.607954    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91bc89fb-502f-4c3e-8aef-1c2f07998932-konnectivity-ca\") pod \"konnectivity-agent-r5zjc\" (UID: \"91bc89fb-502f-4c3e-8aef-1c2f07998932\") " pod="kube-system/konnectivity-agent-r5zjc"
Jul 16 20:32:35.630624 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608000    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c77ecce4-9890-4cef-bf49-ebac541c6d6d-os-release\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.630624 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608104    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n548\" (UniqueName: \"kubernetes.io/projected/c77ecce4-9890-4cef-bf49-ebac541c6d6d-kube-api-access-5n548\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.630624 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608140    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-multus-cni-dir\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.630624 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608140    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-multus-daemon-config\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.630624 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608157    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-hostroot\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.631083 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608172    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lhjb\" (UniqueName: \"kubernetes.io/projected/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-kube-api-access-5lhjb\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.631083 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608189    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-node-log\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.631083 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608213    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-hostroot\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.631083 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608213    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-cni-binary-copy\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.631083 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608247    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-run-netns\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.631083 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608254    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-node-log\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.631083 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608274    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-multus-conf-dir\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.631083 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608232    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-multus-cni-dir\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.631083 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608294    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-run-netns\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.631083 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608304    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-etc-selinux\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.631083 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608340    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-multus-conf-dir\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.631083 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608390    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6df4aad3-be34-4b4a-bd20-4b46e9d53737-hosts-file\") pod \"node-resolver-9jr72\" (UID: \"6df4aad3-be34-4b4a-bd20-4b46e9d53737\") " pod="openshift-dns/node-resolver-9jr72"
Jul 16 20:32:35.631083 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608398    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3b501cab-446b-4e3d-b9f8-3e2958d42293-serviceca\") pod \"node-ca-6vjn8\" (UID: \"3b501cab-446b-4e3d-b9f8-3e2958d42293\") " pod="openshift-image-registry/node-ca-6vjn8"
Jul 16 20:32:35.631083 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608417    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c77ecce4-9890-4cef-bf49-ebac541c6d6d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.631083 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608442    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6df4aad3-be34-4b4a-bd20-4b46e9d53737-hosts-file\") pod \"node-resolver-9jr72\" (UID: \"6df4aad3-be34-4b4a-bd20-4b46e9d53737\") " pod="openshift-dns/node-resolver-9jr72"
Jul 16 20:32:35.631609 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608481    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-etc-selinux\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.631609 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608444    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/53dad2b0-4e1e-4fee-8022-8aceabe32607-iptables-alerter-script\") pod \"iptables-alerter-67v9j\" (UID: \"53dad2b0-4e1e-4fee-8022-8aceabe32607\") " pod="openshift-network-operator/iptables-alerter-67v9j"
Jul 16 20:32:35.631609 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608521    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-registration-dir\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.631609 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608553    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c77ecce4-9890-4cef-bf49-ebac541c6d6d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.631609 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608579    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-var-lib-kubelet\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.631609 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608606    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gztq2\" (UniqueName: \"kubernetes.io/projected/2d50105b-d05a-487b-bfa1-2b5167015795-kube-api-access-gztq2\") pod \"network-metrics-daemon-pr8rp\" (UID: \"2d50105b-d05a-487b-bfa1-2b5167015795\") " pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:35.631609 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608633    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6df4aad3-be34-4b4a-bd20-4b46e9d53737-tmp-dir\") pod \"node-resolver-9jr72\" (UID: \"6df4aad3-be34-4b4a-bd20-4b46e9d53737\") " pod="openshift-dns/node-resolver-9jr72"
Jul 16 20:32:35.631609 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608651    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-host-var-lib-kubelet\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.631609 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608659    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-run-ovn-kubernetes\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.631609 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608687    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c77ecce4-9890-4cef-bf49-ebac541c6d6d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.631609 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608692    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-cni-binary-copy\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.631609 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608698    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-cni-netd\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.631609 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608704    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-run-ovn-kubernetes\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.631609 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608735    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/380d1ee9-2151-419f-a44f-35445c5abba7-env-overrides\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.632044 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608741    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-cni-netd\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.632044 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608770    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/380d1ee9-2151-419f-a44f-35445c5abba7-ovnkube-script-lib\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.632044 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608798    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-system-cni-dir\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.632044 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608823    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3b501cab-446b-4e3d-b9f8-3e2958d42293-host\") pod \"node-ca-6vjn8\" (UID: \"3b501cab-446b-4e3d-b9f8-3e2958d42293\") " pod="openshift-image-registry/node-ca-6vjn8"
Jul 16 20:32:35.632044 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608848    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-var-lib-openvswitch\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.632044 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.632044 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608903    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/380d1ee9-2151-419f-a44f-35445c5abba7-ovn-node-metrics-cert\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.632044 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608582    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8902bd3a-2b67-43d5-ae89-2c0271731c9a-registration-dir\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.632044 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608917    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c77ecce4-9890-4cef-bf49-ebac541c6d6d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.632044 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.608946    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c77ecce4-9890-4cef-bf49-ebac541c6d6d-cni-binary-copy\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.632044 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.609004    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6df4aad3-be34-4b4a-bd20-4b46e9d53737-tmp-dir\") pod \"node-resolver-9jr72\" (UID: \"6df4aad3-be34-4b4a-bd20-4b46e9d53737\") " pod="openshift-dns/node-resolver-9jr72"
Jul 16 20:32:35.632044 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.609024    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/380d1ee9-2151-419f-a44f-35445c5abba7-env-overrides\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.632044 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.609079    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3b501cab-446b-4e3d-b9f8-3e2958d42293-host\") pod \"node-ca-6vjn8\" (UID: \"3b501cab-446b-4e3d-b9f8-3e2958d42293\") " pod="openshift-image-registry/node-ca-6vjn8"
Jul 16 20:32:35.632044 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.609186    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-system-cni-dir\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.878534 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.609196    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.878534 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.609247    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/380d1ee9-2151-419f-a44f-35445c5abba7-var-lib-openvswitch\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.878534 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.609454    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91bc89fb-502f-4c3e-8aef-1c2f07998932-agent-certs\") pod \"konnectivity-agent-r5zjc\" (UID: \"91bc89fb-502f-4c3e-8aef-1c2f07998932\") " pod="kube-system/konnectivity-agent-r5zjc"
Jul 16 20:32:35.878534 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.609495    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/380d1ee9-2151-419f-a44f-35445c5abba7-ovnkube-script-lib\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.878534 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.609518    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c77ecce4-9890-4cef-bf49-ebac541c6d6d-cni-binary-copy\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.878534 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.609581    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/53dad2b0-4e1e-4fee-8022-8aceabe32607-iptables-alerter-script\") pod \"iptables-alerter-67v9j\" (UID: \"53dad2b0-4e1e-4fee-8022-8aceabe32607\") " pod="openshift-network-operator/iptables-alerter-67v9j"
Jul 16 20:32:35.878534 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.610937    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/380d1ee9-2151-419f-a44f-35445c5abba7-ovn-node-metrics-cert\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.878534 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.616646    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqxkk\" (UniqueName: \"kubernetes.io/projected/6df4aad3-be34-4b4a-bd20-4b46e9d53737-kube-api-access-qqxkk\") pod \"node-resolver-9jr72\" (UID: \"6df4aad3-be34-4b4a-bd20-4b46e9d53737\") " pod="openshift-dns/node-resolver-9jr72"
Jul 16 20:32:35.878534 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.617066    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9w5z\" (UniqueName: \"kubernetes.io/projected/380d1ee9-2151-419f-a44f-35445c5abba7-kube-api-access-v9w5z\") pod \"ovnkube-node-9lz24\" (UID: \"380d1ee9-2151-419f-a44f-35445c5abba7\") " pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.878534 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.617447    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmncw\" (UniqueName: \"kubernetes.io/projected/3b501cab-446b-4e3d-b9f8-3e2958d42293-kube-api-access-hmncw\") pod \"node-ca-6vjn8\" (UID: \"3b501cab-446b-4e3d-b9f8-3e2958d42293\") " pod="openshift-image-registry/node-ca-6vjn8"
Jul 16 20:32:35.878534 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.618584    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n548\" (UniqueName: \"kubernetes.io/projected/c77ecce4-9890-4cef-bf49-ebac541c6d6d-kube-api-access-5n548\") pod \"multus-additional-cni-plugins-v4qls\" (UID: \"c77ecce4-9890-4cef-bf49-ebac541c6d6d\") " pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.878534 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.618589    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm6zh\" (UniqueName: \"kubernetes.io/projected/53dad2b0-4e1e-4fee-8022-8aceabe32607-kube-api-access-fm6zh\") pod \"iptables-alerter-67v9j\" (UID: \"53dad2b0-4e1e-4fee-8022-8aceabe32607\") " pod="openshift-network-operator/iptables-alerter-67v9j"
Jul 16 20:32:35.878534 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.618979    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-268t5\" (UniqueName: \"kubernetes.io/projected/8902bd3a-2b67-43d5-ae89-2c0271731c9a-kube-api-access-268t5\") pod \"aws-ebs-csi-driver-node-85qfc\" (UID: \"8902bd3a-2b67-43d5-ae89-2c0271731c9a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.878534 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.619198    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gztq2\" (UniqueName: \"kubernetes.io/projected/2d50105b-d05a-487b-bfa1-2b5167015795-kube-api-access-gztq2\") pod \"network-metrics-daemon-pr8rp\" (UID: \"2d50105b-d05a-487b-bfa1-2b5167015795\") " pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:35.881459 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.624847    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lhjb\" (UniqueName: \"kubernetes.io/projected/6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c-kube-api-access-5lhjb\") pod \"multus-w9582\" (UID: \"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c\") " pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.881459 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.724758    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nwjsv"
Jul 16 20:32:35.881459 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.740270    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9jr72"
Jul 16 20:32:35.881459 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.764120    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:32:35.881459 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.780727    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-v4qls"
Jul 16 20:32:35.881459 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.808589    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-67v9j"
Jul 16 20:32:35.881459 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:35.818509    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5861fc40cf4c0bb1ecbf0801c1a2d731.slice/crio-211bd6a3309b1bab26cd68a0d70893914ed96ae3b35b127e3bdf3aeac25316c4 WatchSource:0}: Error finding container 211bd6a3309b1bab26cd68a0d70893914ed96ae3b35b127e3bdf3aeac25316c4: Status 404 returned error can't find the container with id 211bd6a3309b1bab26cd68a0d70893914ed96ae3b35b127e3bdf3aeac25316c4
Jul 16 20:32:35.881459 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:35.821810    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53dad2b0_4e1e_4fee_8022_8aceabe32607.slice/crio-a932487ba02b44b1300898a9a314f5915ff073e9c882467665167f9552d148e9 WatchSource:0}: Error finding container a932487ba02b44b1300898a9a314f5915ff073e9c882467665167f9552d148e9: Status 404 returned error can't find the container with id a932487ba02b44b1300898a9a314f5915ff073e9c882467665167f9552d148e9
Jul 16 20:32:35.881459 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.823060    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 20:32:35.881459 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.833008    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w9582"
Jul 16 20:32:35.881459 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:35.840480    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c8a52d2_f4d2_45d4_ad2b_0b4fecb8bf7c.slice/crio-edcd255e057d5eed8146325b327f49b70ec39a8ddb8190779f0fe5bc05fd109b WatchSource:0}: Error finding container edcd255e057d5eed8146325b327f49b70ec39a8ddb8190779f0fe5bc05fd109b: Status 404 returned error can't find the container with id edcd255e057d5eed8146325b327f49b70ec39a8ddb8190779f0fe5bc05fd109b
Jul 16 20:32:35.881459 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.848960    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc"
Jul 16 20:32:35.881459 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:35.855360    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8902bd3a_2b67_43d5_ae89_2c0271731c9a.slice/crio-6ae41b9048753575fcef83056c006a0485d0e1ba5309437a4545dcd51c660c0c WatchSource:0}: Error finding container 6ae41b9048753575fcef83056c006a0485d0e1ba5309437a4545dcd51c660c0c: Status 404 returned error can't find the container with id 6ae41b9048753575fcef83056c006a0485d0e1ba5309437a4545dcd51c660c0c
Jul 16 20:32:35.881459 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.874945    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6vjn8"
Jul 16 20:32:35.881459 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:35.880971    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r5zjc"
Jul 16 20:32:35.882412 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:35.881465    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b501cab_446b_4e3d_b9f8_3e2958d42293.slice/crio-a4c57edc8b202c43f4e6fbb823ce7171285a0c5739992e25e923f19cf1a7bc6c WatchSource:0}: Error finding container a4c57edc8b202c43f4e6fbb823ce7171285a0c5739992e25e923f19cf1a7bc6c: Status 404 returned error can't find the container with id a4c57edc8b202c43f4e6fbb823ce7171285a0c5739992e25e923f19cf1a7bc6c
Jul 16 20:32:36.118045 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:36.112516    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs\") pod \"network-metrics-daemon-pr8rp\" (UID: \"2d50105b-d05a-487b-bfa1-2b5167015795\") " pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:36.118045 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:36.112552    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcz7l\" (UniqueName: \"kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l\") pod \"network-check-target-xxh76\" (UID: \"30258292-80a3-4166-a0e5-1ad9bec7d240\") " pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:36.118045 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:36.112583    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret\") pod \"global-pull-secret-syncer-q77jc\" (UID: \"6ef282cd-38f3-4e0d-b862-b9d510e70226\") " pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:36.118045 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:36.112654    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 20:32:36.118045 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:36.112655    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:32:36.118045 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:36.112670    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 20:32:36.118045 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:36.112688    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 20:32:36.118045 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:36.112695    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret podName:6ef282cd-38f3-4e0d-b862-b9d510e70226 nodeName:}" failed. No retries permitted until 2026-07-16 20:32:37.112682508 +0000 UTC m=+3.078616721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret") pod "global-pull-secret-syncer-q77jc" (UID: "6ef282cd-38f3-4e0d-b862-b9d510e70226") : object "kube-system"/"original-pull-secret" not registered
Jul 16 20:32:36.118045 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:36.112700    2576 projected.go:196] Error preparing data for projected volume kube-api-access-kcz7l for pod openshift-network-diagnostics/network-check-target-xxh76: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:32:36.118045 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:36.112713    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs podName:2d50105b-d05a-487b-bfa1-2b5167015795 nodeName:}" failed. No retries permitted until 2026-07-16 20:32:37.112703386 +0000 UTC m=+3.078637600 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs") pod "network-metrics-daemon-pr8rp" (UID: "2d50105b-d05a-487b-bfa1-2b5167015795") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:32:36.118045 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:36.112734    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l podName:30258292-80a3-4166-a0e5-1ad9bec7d240 nodeName:}" failed. No retries permitted until 2026-07-16 20:32:37.112722251 +0000 UTC m=+3.078656464 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kcz7l" (UniqueName: "kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l") pod "network-check-target-xxh76" (UID: "30258292-80a3-4166-a0e5-1ad9bec7d240") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:32:36.364240 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:36.126025    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc77ecce4_9890_4cef_bf49_ebac541c6d6d.slice/crio-80eb46985d1f0eb89985c5b1e93a66ba1f3c8c21ba00ab75d55df481af2ba2d9 WatchSource:0}: Error finding container 80eb46985d1f0eb89985c5b1e93a66ba1f3c8c21ba00ab75d55df481af2ba2d9: Status 404 returned error can't find the container with id 80eb46985d1f0eb89985c5b1e93a66ba1f3c8c21ba00ab75d55df481af2ba2d9
Jul 16 20:32:36.364240 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:36.167894    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6be9422_38e1_4a0d_b2e9_fd3f77cea68f.slice/crio-e873fd71d9e709063fbdc631560c042cbf44be70d502f77017d0500255adb16c WatchSource:0}: Error finding container e873fd71d9e709063fbdc631560c042cbf44be70d502f77017d0500255adb16c: Status 404 returned error can't find the container with id e873fd71d9e709063fbdc631560c042cbf44be70d502f77017d0500255adb16c
Jul 16 20:32:36.364240 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:36.254393    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df4aad3_be34_4b4a_bd20_4b46e9d53737.slice/crio-ff3478a0863a6c6dd09272d83fbc98945ca26ca77bd8be9c9e3afca232795943 WatchSource:0}: Error finding container ff3478a0863a6c6dd09272d83fbc98945ca26ca77bd8be9c9e3afca232795943: Status 404 returned error can't find the container with id ff3478a0863a6c6dd09272d83fbc98945ca26ca77bd8be9c9e3afca232795943
Jul 16 20:32:36.448601 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:36.443500    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 20:27:35 +0000 UTC" deadline="2028-01-22 18:32:35.616275762 +0000 UTC"
Jul 16 20:32:36.448601 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:36.443541    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13317h59m59.172737836s"
Jul 16 20:32:36.701956 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:36.684112    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:36.701956 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:36.684228    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:32:36.701956 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:36.695300    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9jr72" event={"ID":"6df4aad3-be34-4b4a-bd20-4b46e9d53737","Type":"ContainerStarted","Data":"ff3478a0863a6c6dd09272d83fbc98945ca26ca77bd8be9c9e3afca232795943"}
Jul 16 20:32:36.701956 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:36.697769    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v4qls" event={"ID":"c77ecce4-9890-4cef-bf49-ebac541c6d6d","Type":"ContainerStarted","Data":"80eb46985d1f0eb89985c5b1e93a66ba1f3c8c21ba00ab75d55df481af2ba2d9"}
Jul 16 20:32:36.701956 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:36.700522    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r5zjc" event={"ID":"91bc89fb-502f-4c3e-8aef-1c2f07998932","Type":"ContainerStarted","Data":"60e138743eef7ad5941a2152d3a6365309f19bb663fe9f820730d68af5b0e3a5"}
Jul 16 20:32:36.704716 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:36.703910    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w9582" event={"ID":"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c","Type":"ContainerStarted","Data":"edcd255e057d5eed8146325b327f49b70ec39a8ddb8190779f0fe5bc05fd109b"}
Jul 16 20:32:36.706748 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:36.705607    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-245.ec2.internal" event={"ID":"5861fc40cf4c0bb1ecbf0801c1a2d731","Type":"ContainerStarted","Data":"211bd6a3309b1bab26cd68a0d70893914ed96ae3b35b127e3bdf3aeac25316c4"}
Jul 16 20:32:36.708312 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:36.707021    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nwjsv" event={"ID":"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f","Type":"ContainerStarted","Data":"e873fd71d9e709063fbdc631560c042cbf44be70d502f77017d0500255adb16c"}
Jul 16 20:32:36.709445 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:36.709067    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6vjn8" event={"ID":"3b501cab-446b-4e3d-b9f8-3e2958d42293","Type":"ContainerStarted","Data":"a4c57edc8b202c43f4e6fbb823ce7171285a0c5739992e25e923f19cf1a7bc6c"}
Jul 16 20:32:36.711181 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:36.710563    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc" event={"ID":"8902bd3a-2b67-43d5-ae89-2c0271731c9a","Type":"ContainerStarted","Data":"6ae41b9048753575fcef83056c006a0485d0e1ba5309437a4545dcd51c660c0c"}
Jul 16 20:32:36.789830 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:36.712614    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-67v9j" event={"ID":"53dad2b0-4e1e-4fee-8022-8aceabe32607","Type":"ContainerStarted","Data":"a932487ba02b44b1300898a9a314f5915ff073e9c882467665167f9552d148e9"}
Jul 16 20:32:36.879943 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:36.818705    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 20:32:37.085743 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:37.085350    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 20:32:37.139136 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:37.138131    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs\") pod \"network-metrics-daemon-pr8rp\" (UID: \"2d50105b-d05a-487b-bfa1-2b5167015795\") " pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:37.139136 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:37.138186    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcz7l\" (UniqueName: \"kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l\") pod \"network-check-target-xxh76\" (UID: \"30258292-80a3-4166-a0e5-1ad9bec7d240\") " pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:37.139136 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:37.138240    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret\") pod \"global-pull-secret-syncer-q77jc\" (UID: \"6ef282cd-38f3-4e0d-b862-b9d510e70226\") " pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:37.139136 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:37.138406    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 20:32:37.139136 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:37.138468    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret podName:6ef282cd-38f3-4e0d-b862-b9d510e70226 nodeName:}" failed. No retries permitted until 2026-07-16 20:32:39.138447216 +0000 UTC m=+5.104381434 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret") pod "global-pull-secret-syncer-q77jc" (UID: "6ef282cd-38f3-4e0d-b862-b9d510e70226") : object "kube-system"/"original-pull-secret" not registered
Jul 16 20:32:37.139136 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:37.138835    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:32:37.139136 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:37.138882    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs podName:2d50105b-d05a-487b-bfa1-2b5167015795 nodeName:}" failed. No retries permitted until 2026-07-16 20:32:39.138866165 +0000 UTC m=+5.104800385 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs") pod "network-metrics-daemon-pr8rp" (UID: "2d50105b-d05a-487b-bfa1-2b5167015795") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:32:37.139136 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:37.138997    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 20:32:37.139136 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:37.139017    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 20:32:37.139136 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:37.139030    2576 projected.go:196] Error preparing data for projected volume kube-api-access-kcz7l for pod openshift-network-diagnostics/network-check-target-xxh76: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:32:37.139136 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:37.139069    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l podName:30258292-80a3-4166-a0e5-1ad9bec7d240 nodeName:}" failed. No retries permitted until 2026-07-16 20:32:39.139056739 +0000 UTC m=+5.104990959 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kcz7l" (UniqueName: "kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l") pod "network-check-target-xxh76" (UID: "30258292-80a3-4166-a0e5-1ad9bec7d240") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:32:37.444432 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:37.444285    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 20:27:35 +0000 UTC" deadline="2028-02-21 22:43:47.640914032 +0000 UTC"
Jul 16 20:32:37.444432 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:37.444322    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14042h11m10.196596471s"
Jul 16 20:32:37.683665 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:37.683633    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:37.684266 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:37.683774    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:32:37.684266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:37.684158    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:37.684266 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:37.684245    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:32:37.955297 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:37.955074    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 16 20:32:38.685849 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:38.685723    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:38.686295 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:38.685847    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:32:39.161157 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:39.161119    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret\") pod \"global-pull-secret-syncer-q77jc\" (UID: \"6ef282cd-38f3-4e0d-b862-b9d510e70226\") " pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:39.161313 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:39.161180    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs\") pod \"network-metrics-daemon-pr8rp\" (UID: \"2d50105b-d05a-487b-bfa1-2b5167015795\") " pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:39.161313 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:39.161215    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcz7l\" (UniqueName: \"kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l\") pod \"network-check-target-xxh76\" (UID: \"30258292-80a3-4166-a0e5-1ad9bec7d240\") " pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:39.161473 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:39.161328    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:32:39.161473 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:39.161332    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 20:32:39.161473 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:39.161413    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs podName:2d50105b-d05a-487b-bfa1-2b5167015795 nodeName:}" failed. No retries permitted until 2026-07-16 20:32:43.161393049 +0000 UTC m=+9.127327282 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs") pod "network-metrics-daemon-pr8rp" (UID: "2d50105b-d05a-487b-bfa1-2b5167015795") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:32:39.161473 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:39.161448    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret podName:6ef282cd-38f3-4e0d-b862-b9d510e70226 nodeName:}" failed. No retries permitted until 2026-07-16 20:32:43.161431209 +0000 UTC m=+9.127365427 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret") pod "global-pull-secret-syncer-q77jc" (UID: "6ef282cd-38f3-4e0d-b862-b9d510e70226") : object "kube-system"/"original-pull-secret" not registered
Jul 16 20:32:39.161473 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:39.161335    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 20:32:39.161473 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:39.161472    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 20:32:39.162036 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:39.161484    2576 projected.go:196] Error preparing data for projected volume kube-api-access-kcz7l for pod openshift-network-diagnostics/network-check-target-xxh76: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:32:39.162036 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:39.161521    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l podName:30258292-80a3-4166-a0e5-1ad9bec7d240 nodeName:}" failed. No retries permitted until 2026-07-16 20:32:43.161511294 +0000 UTC m=+9.127445509 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kcz7l" (UniqueName: "kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l") pod "network-check-target-xxh76" (UID: "30258292-80a3-4166-a0e5-1ad9bec7d240") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:32:39.684272 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:39.684168    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:39.684272 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:39.684202    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:39.684572 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:39.684287    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:32:39.684572 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:39.684428    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:32:40.685951 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:40.685722    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:40.685951 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:40.685847    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:32:41.683277 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:41.683245    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:41.683461 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:41.683245    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:41.683461 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:41.683389    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:32:41.683620 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:41.683499    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:32:42.683758 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:42.683684    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:42.684197 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:42.683815    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:32:43.187642 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:43.187610    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret\") pod \"global-pull-secret-syncer-q77jc\" (UID: \"6ef282cd-38f3-4e0d-b862-b9d510e70226\") " pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:43.187774 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:43.187663    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs\") pod \"network-metrics-daemon-pr8rp\" (UID: \"2d50105b-d05a-487b-bfa1-2b5167015795\") " pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:43.187774 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:43.187695    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcz7l\" (UniqueName: \"kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l\") pod \"network-check-target-xxh76\" (UID: \"30258292-80a3-4166-a0e5-1ad9bec7d240\") " pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:43.187861 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:43.187812    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 20:32:43.187861 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:43.187829    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:32:43.187934 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:43.187888    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret podName:6ef282cd-38f3-4e0d-b862-b9d510e70226 nodeName:}" failed. No retries permitted until 2026-07-16 20:32:51.187868687 +0000 UTC m=+17.153802903 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret") pod "global-pull-secret-syncer-q77jc" (UID: "6ef282cd-38f3-4e0d-b862-b9d510e70226") : object "kube-system"/"original-pull-secret" not registered
Jul 16 20:32:43.187934 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:43.187907    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs podName:2d50105b-d05a-487b-bfa1-2b5167015795 nodeName:}" failed. No retries permitted until 2026-07-16 20:32:51.187897788 +0000 UTC m=+17.153832007 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs") pod "network-metrics-daemon-pr8rp" (UID: "2d50105b-d05a-487b-bfa1-2b5167015795") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:32:43.187934 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:43.187888    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 20:32:43.188091 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:43.187955    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 20:32:43.188091 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:43.187968    2576 projected.go:196] Error preparing data for projected volume kube-api-access-kcz7l for pod openshift-network-diagnostics/network-check-target-xxh76: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:32:43.188091 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:43.188013    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l podName:30258292-80a3-4166-a0e5-1ad9bec7d240 nodeName:}" failed. No retries permitted until 2026-07-16 20:32:51.187997009 +0000 UTC m=+17.153931237 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kcz7l" (UniqueName: "kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l") pod "network-check-target-xxh76" (UID: "30258292-80a3-4166-a0e5-1ad9bec7d240") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:32:43.684222 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:43.684186    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:43.684658 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:43.684295    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:32:43.684658 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:43.684345    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:43.684658 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:43.684481    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:32:44.684237 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:44.684203    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:44.684681 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:44.684303    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:32:45.684125 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:45.684093    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:45.684277 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:45.684129    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:45.684277 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:45.684219    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:32:45.684693 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:45.684390    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:32:46.684140 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:46.684107    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:46.684294 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:46.684240    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:32:47.683458 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:47.683413    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:47.683458 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:47.683430    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:47.683755 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:47.683543    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:32:47.683755 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:47.683665    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:32:48.684003 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:48.683975    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:48.684372 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:48.684097    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:32:49.065814 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:49.065775    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3d7cdf18b301318dd0bf2a5e9344c0d.slice/crio-6b55a943e7071799b7fb23d5a559a6fb271331d4f3b22ecb8ca05428de709279 WatchSource:0}: Error finding container 6b55a943e7071799b7fb23d5a559a6fb271331d4f3b22ecb8ca05428de709279: Status 404 returned error can't find the container with id 6b55a943e7071799b7fb23d5a559a6fb271331d4f3b22ecb8ca05428de709279
Jul 16 20:32:49.068873 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:32:49.068851    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod380d1ee9_2151_419f_a44f_35445c5abba7.slice/crio-516fa737af45bbd4decb2a57b6720429bd447877cbf279c3869670881e077dd8 WatchSource:0}: Error finding container 516fa737af45bbd4decb2a57b6720429bd447877cbf279c3869670881e077dd8: Status 404 returned error can't find the container with id 516fa737af45bbd4decb2a57b6720429bd447877cbf279c3869670881e077dd8
Jul 16 20:32:49.684042 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:49.683880    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:49.684541 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:49.683939    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:49.684541 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:49.684152    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:32:49.684541 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:49.684196    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:32:49.745895 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:49.745194    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nwjsv" event={"ID":"c6be9422-38e1-4a0d-b2e9-fd3f77cea68f","Type":"ContainerStarted","Data":"620368c196f8cf0639edbf1a11e906fb2fca4eac2bc569a365ae701151a8b9cc"}
Jul 16 20:32:49.751466 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:49.751439    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24" event={"ID":"380d1ee9-2151-419f-a44f-35445c5abba7","Type":"ContainerStarted","Data":"516fa737af45bbd4decb2a57b6720429bd447877cbf279c3869670881e077dd8"}
Jul 16 20:32:49.755888 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:49.755463    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w9582" event={"ID":"6c8a52d2-f4d2-45d4-ad2b-0b4fecb8bf7c","Type":"ContainerStarted","Data":"fc67c6ddf9629474b1cb19f775de990c310e86009ce87595dea0f5d6f8220423"}
Jul 16 20:32:49.760391 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:49.760323    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-245.ec2.internal" event={"ID":"5861fc40cf4c0bb1ecbf0801c1a2d731","Type":"ContainerStarted","Data":"0e51a61dc545a466e6c440de93341034676e6f15a537280da20dfd7d2b07bd28"}
Jul 16 20:32:49.764036 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:49.764011    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal" event={"ID":"d3d7cdf18b301318dd0bf2a5e9344c0d","Type":"ContainerStarted","Data":"6b55a943e7071799b7fb23d5a559a6fb271331d4f3b22ecb8ca05428de709279"}
Jul 16 20:32:49.766761 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:49.766714    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nwjsv" podStartSLOduration=2.873167121 podStartE2EDuration="15.766697502s" totalImagesPullingTime="12.893530381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:32:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:32:49.765749594 +0000 UTC m=+15.731683828" watchObservedRunningTime="2026-07-16 20:32:49.766697502 +0000 UTC m=+15.732631735"
Jul 16 20:32:49.807998 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:49.807950    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-245.ec2.internal" podStartSLOduration=14.807932035 podStartE2EDuration="14.807932035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:32:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:32:49.779390786 +0000 UTC m=+15.745325014" watchObservedRunningTime="2026-07-16 20:32:49.807932035 +0000 UTC m=+15.773866268"
Jul 16 20:32:49.808117 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:49.808061    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-w9582" podStartSLOduration=1.585130011 podStartE2EDuration="14.8080559s" totalImagesPullingTime="13.222925889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:32:49.807697967 +0000 UTC m=+15.773632215" watchObservedRunningTime="2026-07-16 20:32:49.8080559 +0000 UTC m=+15.773990135"
Jul 16 20:32:50.683708 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:50.683677    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:50.683863 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:50.683796    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:32:50.767594 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:50.767523    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc" event={"ID":"8902bd3a-2b67-43d5-ae89-2c0271731c9a","Type":"ContainerStarted","Data":"65a6f65d5da6e9b9bb435219227468f78da8d49f669e28614300574afbdb8317"}
Jul 16 20:32:50.769185 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:50.769153    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-67v9j" event={"ID":"53dad2b0-4e1e-4fee-8022-8aceabe32607","Type":"ContainerStarted","Data":"df154ae549a42227c79f4069701f2adf65819c4583cc412e57928c7d8c2aa687"}
Jul 16 20:32:50.770964 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:50.770937    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9jr72" event={"ID":"6df4aad3-be34-4b4a-bd20-4b46e9d53737","Type":"ContainerStarted","Data":"185c29bf2e4e42950826962de101f1073ba940aa764cda563799da395d185f4c"}
Jul 16 20:32:50.776487 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:50.776461    2576 generic.go:356] "Generic (PLEG): container finished" podID="c77ecce4-9890-4cef-bf49-ebac541c6d6d" containerID="730125f0062a8098544da25a557dda2d98135e33d1be86aae80ebde3a916ea06" exitCode=0
Jul 16 20:32:50.776601 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:50.776542    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v4qls" event={"ID":"c77ecce4-9890-4cef-bf49-ebac541c6d6d","Type":"ContainerDied","Data":"730125f0062a8098544da25a557dda2d98135e33d1be86aae80ebde3a916ea06"}
Jul 16 20:32:50.779204 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:50.779179    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r5zjc" event={"ID":"91bc89fb-502f-4c3e-8aef-1c2f07998932","Type":"ContainerStarted","Data":"18b7dd3cd09a94fcfe233e31c15761c73535c8331d552f84537c989fc7d17b77"}
Jul 16 20:32:50.781722 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:50.781651    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6vjn8" event={"ID":"3b501cab-446b-4e3d-b9f8-3e2958d42293","Type":"ContainerStarted","Data":"b501971f7e11b7cb9017a7faceccec2f0b088eb20caeb1c9de0bdb2deb83768c"}
Jul 16 20:32:50.783271 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:50.783236    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-67v9j" podStartSLOduration=3.968732316 podStartE2EDuration="16.783224043s" totalImagesPullingTime="12.814491727s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:32:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:32:50.782827319 +0000 UTC m=+16.748761553" watchObservedRunningTime="2026-07-16 20:32:50.783224043 +0000 UTC m=+16.749158276"
Jul 16 20:32:50.797991 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:50.797792    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-r5zjc" podStartSLOduration=7.570768751 podStartE2EDuration="15.797777553s" totalImagesPullingTime="8.227008802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:32:50.797396347 +0000 UTC m=+16.763330579" watchObservedRunningTime="2026-07-16 20:32:50.797777553 +0000 UTC m=+16.763711787"
Jul 16 20:32:50.834458 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:50.834419    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9jr72" podStartSLOduration=4.042192215 podStartE2EDuration="16.83440532s" totalImagesPullingTime="12.792213105s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:32:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:32:50.812324967 +0000 UTC m=+16.778259201" watchObservedRunningTime="2026-07-16 20:32:50.83440532 +0000 UTC m=+16.800339553"
Jul 16 20:32:50.848815 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:50.848767    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6vjn8" podStartSLOduration=3.8985199440000002 podStartE2EDuration="15.848751119s" totalImagesPullingTime="11.950231175s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:32:50.848622793 +0000 UTC m=+16.814557027" watchObservedRunningTime="2026-07-16 20:32:50.848751119 +0000 UTC m=+16.814685353"
Jul 16 20:32:51.250829 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:51.250790    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret\") pod \"global-pull-secret-syncer-q77jc\" (UID: \"6ef282cd-38f3-4e0d-b862-b9d510e70226\") " pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:51.250829 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:51.250841    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs\") pod \"network-metrics-daemon-pr8rp\" (UID: \"2d50105b-d05a-487b-bfa1-2b5167015795\") " pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:51.251165 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:51.250875    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcz7l\" (UniqueName: \"kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l\") pod \"network-check-target-xxh76\" (UID: \"30258292-80a3-4166-a0e5-1ad9bec7d240\") " pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:51.251165 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:51.251039    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 20:32:51.251165 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:51.251061    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 20:32:51.251165 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:51.251073    2576 projected.go:196] Error preparing data for projected volume kube-api-access-kcz7l for pod openshift-network-diagnostics/network-check-target-xxh76: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:32:51.251165 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:51.251126    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l podName:30258292-80a3-4166-a0e5-1ad9bec7d240 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:07.251109551 +0000 UTC m=+33.217043766 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kcz7l" (UniqueName: "kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l") pod "network-check-target-xxh76" (UID: "30258292-80a3-4166-a0e5-1ad9bec7d240") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:32:51.251660 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:51.251515    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 20:32:51.251660 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:51.251563    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:32:51.251660 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:51.251569    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret podName:6ef282cd-38f3-4e0d-b862-b9d510e70226 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:07.251554182 +0000 UTC m=+33.217488399 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret") pod "global-pull-secret-syncer-q77jc" (UID: "6ef282cd-38f3-4e0d-b862-b9d510e70226") : object "kube-system"/"original-pull-secret" not registered
Jul 16 20:32:51.251660 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:51.251640    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs podName:2d50105b-d05a-487b-bfa1-2b5167015795 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:07.251622807 +0000 UTC m=+33.217557020 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs") pod "network-metrics-daemon-pr8rp" (UID: "2d50105b-d05a-487b-bfa1-2b5167015795") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:32:51.683606 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:51.683562    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:51.683767 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:51.683623    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:51.683767 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:51.683704    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:32:51.683897 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:51.683870    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:32:52.009514 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:52.008968    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 16 20:32:52.487223 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:52.487106    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T20:32:52.0089815Z","UUID":"589b51a3-1337-4d6f-9a74-8a9143583943","Handler":null,"Name":"","Endpoint":""}
Jul 16 20:32:52.488941 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:52.488917    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 16 20:32:52.489080 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:52.488962    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 16 20:32:52.683881 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:52.683852    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:52.684041 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:52.683977    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:32:52.788733 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:52.788137    2576 generic.go:356] "Generic (PLEG): container finished" podID="d3d7cdf18b301318dd0bf2a5e9344c0d" containerID="3920f4e919e75c0c9869d21bda7a33848781bbf1d97d0b116e06bbc259fa94f8" exitCode=0
Jul 16 20:32:52.788733 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:52.788233    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal" event={"ID":"d3d7cdf18b301318dd0bf2a5e9344c0d","Type":"ContainerDied","Data":"3920f4e919e75c0c9869d21bda7a33848781bbf1d97d0b116e06bbc259fa94f8"}
Jul 16 20:32:52.791786 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:52.791177    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc" event={"ID":"8902bd3a-2b67-43d5-ae89-2c0271731c9a","Type":"ContainerStarted","Data":"88f2fe4e1118870d41b1a9d919e4bd9360a242876afcabb1929f8392fd939ceb"}
Jul 16 20:32:52.819641 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:52.819118    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-r5zjc"
Jul 16 20:32:52.820237 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:52.820179    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-r5zjc"
Jul 16 20:32:53.684376 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:53.684146    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:53.685007 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:53.684193    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:53.685007 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:53.684467    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:32:53.685007 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:53.684575    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:32:53.794982 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:53.794950    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal" event={"ID":"d3d7cdf18b301318dd0bf2a5e9344c0d","Type":"ContainerStarted","Data":"9077c5ce2da4e7a7b5e61c0e70d6b4eb64e040492afc3192bf9befe5a5a3c441"}
Jul 16 20:32:53.797353 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:53.797318    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc" event={"ID":"8902bd3a-2b67-43d5-ae89-2c0271731c9a","Type":"ContainerStarted","Data":"41b02dc6bc00fd7e503a96ec164d8fe9868311b72f2d9ca869186599dd6975d0"}
Jul 16 20:32:53.797911 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:53.797777    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-r5zjc"
Jul 16 20:32:53.798106 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:53.798077    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-r5zjc"
Jul 16 20:32:53.808571 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:53.808527    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-245.ec2.internal" podStartSLOduration=18.808510281 podStartE2EDuration="18.808510281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:32:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:32:53.807936534 +0000 UTC m=+19.773870768" watchObservedRunningTime="2026-07-16 20:32:53.808510281 +0000 UTC m=+19.774444515"
Jul 16 20:32:53.823768 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:53.823726    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-85qfc" podStartSLOduration=3.674005857 podStartE2EDuration="18.823710227s" totalImagesPullingTime="15.14970437s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:32:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:32:53.823324366 +0000 UTC m=+19.789258599" watchObservedRunningTime="2026-07-16 20:32:53.823710227 +0000 UTC m=+19.789644461"
Jul 16 20:32:54.684903 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:54.684873    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:54.685351 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:54.684982    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:32:55.684315 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:55.684285    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:55.684315 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:55.684307    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:55.684561 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:55.684426    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:32:55.684630 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:55.684555    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:32:56.684025 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:56.683999    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:56.684502 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:56.684113    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:32:56.805192 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:56.805161    2576 generic.go:356] "Generic (PLEG): container finished" podID="c77ecce4-9890-4cef-bf49-ebac541c6d6d" containerID="5769ee5f2f681786d4def92aa58013e5bd6273c72182e99bba9ab828acb6acd0" exitCode=0
Jul 16 20:32:56.805345 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:56.805214    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v4qls" event={"ID":"c77ecce4-9890-4cef-bf49-ebac541c6d6d","Type":"ContainerDied","Data":"5769ee5f2f681786d4def92aa58013e5bd6273c72182e99bba9ab828acb6acd0"}
Jul 16 20:32:57.683783 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:57.683754    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:57.683897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:57.683757    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:57.683897 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:57.683872    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:32:57.684005 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:57.683968    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:32:57.809847 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:57.809773    2576 generic.go:356] "Generic (PLEG): container finished" podID="c77ecce4-9890-4cef-bf49-ebac541c6d6d" containerID="1124b0e45c998889fae3e757b556f9dd181e77f181b270348590b79a52a3a523" exitCode=0
Jul 16 20:32:57.809847 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:57.809831    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v4qls" event={"ID":"c77ecce4-9890-4cef-bf49-ebac541c6d6d","Type":"ContainerDied","Data":"1124b0e45c998889fae3e757b556f9dd181e77f181b270348590b79a52a3a523"}
Jul 16 20:32:58.683680 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:58.683647    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:32:58.683944 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:58.683767    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:32:59.684012 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:59.683969    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:32:59.684530 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:59.684096    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:32:59.684530 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:32:59.684149    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:32:59.684530 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:32:59.684243    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:33:00.683964 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:00.683936    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:33:00.684140 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:00.684062    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:33:01.684162 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:01.684134    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:33:01.684591 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:01.684144    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:33:01.684591 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:01.684237    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:33:01.684591 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:01.684320    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:33:02.684611 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:02.684588    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:33:02.685088 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:02.684791    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:33:02.821161 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:02.821048    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24" event={"ID":"380d1ee9-2151-419f-a44f-35445c5abba7","Type":"ContainerStarted","Data":"3a9b7262cfc5bddf539c3d7aa1342c8fa88977454948eded20fdae6c118eb3d4"}
Jul 16 20:33:02.821264 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:02.821236    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24" event={"ID":"380d1ee9-2151-419f-a44f-35445c5abba7","Type":"ContainerStarted","Data":"0901e4ed68e296cddabf807265cb700ec313ea919fe177ab32c0cf9df96133ca"}
Jul 16 20:33:02.821303 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:02.821278    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24" event={"ID":"380d1ee9-2151-419f-a44f-35445c5abba7","Type":"ContainerStarted","Data":"a03a2bc86e69d1c644d3a4882d4f4d43f708069dfb7e8e70212db1f654e0df6f"}
Jul 16 20:33:03.684237 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:03.684214    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:33:03.684351 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:03.684221    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:33:03.684351 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:03.684315    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:33:03.684510 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:03.684429    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:33:03.826435 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:03.826410    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24" event={"ID":"380d1ee9-2151-419f-a44f-35445c5abba7","Type":"ContainerStarted","Data":"7ca94ae2e329eb419999d84c025d5b2fb45be9300aa94ab6905d2008a23863f8"}
Jul 16 20:33:03.826963 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:03.826473    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24" event={"ID":"380d1ee9-2151-419f-a44f-35445c5abba7","Type":"ContainerStarted","Data":"db4c30b767b567cf35e95c8bd33ca34125e57386936623d7b8bebcff22d5d823"}
Jul 16 20:33:03.826963 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:03.826517    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24" event={"ID":"380d1ee9-2151-419f-a44f-35445c5abba7","Type":"ContainerStarted","Data":"0545dddda968c8b92e40ad74f20f6fc0f72b5d84daa9ed96b45782324b381ec4"}
Jul 16 20:33:03.828983 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:03.828950    2576 generic.go:356] "Generic (PLEG): container finished" podID="c77ecce4-9890-4cef-bf49-ebac541c6d6d" containerID="ef802267b4cad406bf18a8f6ff0e9ff27d420a5bd056b9e15d2aebabea6700cf" exitCode=0
Jul 16 20:33:03.829082 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:03.828989    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v4qls" event={"ID":"c77ecce4-9890-4cef-bf49-ebac541c6d6d","Type":"ContainerDied","Data":"ef802267b4cad406bf18a8f6ff0e9ff27d420a5bd056b9e15d2aebabea6700cf"}
Jul 16 20:33:04.684415 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:04.684385    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:33:04.684571 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:04.684485    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:33:05.683842 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:05.683817    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:33:05.684342 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:05.683933    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:33:05.684342 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:05.683994    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:33:05.684342 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:05.684106    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:33:05.837587 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:05.837556    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24" event={"ID":"380d1ee9-2151-419f-a44f-35445c5abba7","Type":"ContainerStarted","Data":"c8250a506f66cffc7d3764815c5370d1b68e9625205297f6244a9cab07bee14e"}
Jul 16 20:33:06.684021 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:06.683991    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:33:06.684435 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:06.684121    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:33:07.275633 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:07.275598    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret\") pod \"global-pull-secret-syncer-q77jc\" (UID: \"6ef282cd-38f3-4e0d-b862-b9d510e70226\") " pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:33:07.275813 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:07.275653    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs\") pod \"network-metrics-daemon-pr8rp\" (UID: \"2d50105b-d05a-487b-bfa1-2b5167015795\") " pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:33:07.275813 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:07.275707    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcz7l\" (UniqueName: \"kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l\") pod \"network-check-target-xxh76\" (UID: \"30258292-80a3-4166-a0e5-1ad9bec7d240\") " pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:33:07.275813 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:07.275773    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:33:07.275987 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:07.275828    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs podName:2d50105b-d05a-487b-bfa1-2b5167015795 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:39.275809313 +0000 UTC m=+65.241743527 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs") pod "network-metrics-daemon-pr8rp" (UID: "2d50105b-d05a-487b-bfa1-2b5167015795") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 16 20:33:07.275987 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:07.275851    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 16 20:33:07.275987 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:07.275870    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 16 20:33:07.275987 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:07.275869    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 16 20:33:07.275987 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:07.275883    2576 projected.go:196] Error preparing data for projected volume kube-api-access-kcz7l for pod openshift-network-diagnostics/network-check-target-xxh76: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:33:07.275987 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:07.275933    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret podName:6ef282cd-38f3-4e0d-b862-b9d510e70226 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:39.275914538 +0000 UTC m=+65.241848755 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret") pod "global-pull-secret-syncer-q77jc" (UID: "6ef282cd-38f3-4e0d-b862-b9d510e70226") : object "kube-system"/"original-pull-secret" not registered
Jul 16 20:33:07.275987 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:07.275972    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l podName:30258292-80a3-4166-a0e5-1ad9bec7d240 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:39.275958281 +0000 UTC m=+65.241892497 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kcz7l" (UniqueName: "kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l") pod "network-check-target-xxh76" (UID: "30258292-80a3-4166-a0e5-1ad9bec7d240") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 16 20:33:07.683646 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:07.683609    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:33:07.683808 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:07.683623    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:33:07.683808 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:07.683729    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:33:07.683808 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:07.683801    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:33:08.683931 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:08.683782    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:33:08.684376 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:08.684065    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:33:08.846268 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:08.846238    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v4qls" event={"ID":"c77ecce4-9890-4cef-bf49-ebac541c6d6d","Type":"ContainerStarted","Data":"333a1983952b0199b80f41a868add74d6db25d6d2bffc7c602611c9532ed43f0"}
Jul 16 20:33:08.851774 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:08.851752    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24" event={"ID":"380d1ee9-2151-419f-a44f-35445c5abba7","Type":"ContainerStarted","Data":"9876762aa58b0aed688df8295bdfb77ebe732fd8cac20f1443045c5045154bd4"}
Jul 16 20:33:08.851973 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:08.851956    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:33:08.852141 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:08.852028    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:33:08.852141 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:08.852082    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:33:08.868796 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:08.868769    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:33:08.881454 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:08.881437    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:33:08.890634 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:08.890582    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24" podStartSLOduration=21.56008292 podStartE2EDuration="34.890568815s" totalImagesPullingTime="13.330485895s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:32:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:08.889643901 +0000 UTC m=+34.855578135" watchObservedRunningTime="2026-07-16 20:33:08.890568815 +0000 UTC m=+34.856503047"
Jul 16 20:33:09.683605 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:09.683579    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:33:09.683761 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:09.683614    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:33:09.683761 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:09.683695    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:33:09.683843 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:09.683773    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:33:09.856533 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:09.856503    2576 generic.go:356] "Generic (PLEG): container finished" podID="c77ecce4-9890-4cef-bf49-ebac541c6d6d" containerID="333a1983952b0199b80f41a868add74d6db25d6d2bffc7c602611c9532ed43f0" exitCode=0
Jul 16 20:33:09.856921 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:09.856593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v4qls" event={"ID":"c77ecce4-9890-4cef-bf49-ebac541c6d6d","Type":"ContainerDied","Data":"333a1983952b0199b80f41a868add74d6db25d6d2bffc7c602611c9532ed43f0"}
Jul 16 20:33:10.451014 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:10.450975    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q77jc"]
Jul 16 20:33:10.451161 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:10.451133    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:33:10.451239 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:10.451219    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:33:10.454476 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:10.454445    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xxh76"]
Jul 16 20:33:10.454628 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:10.454561    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:33:10.454699 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:10.454670    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:33:10.455091 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:10.455053    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pr8rp"]
Jul 16 20:33:10.455187 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:10.455172    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:33:10.455322 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:10.455298    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:33:10.862015 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:10.861933    2576 generic.go:356] "Generic (PLEG): container finished" podID="c77ecce4-9890-4cef-bf49-ebac541c6d6d" containerID="9322bccfd3f64aa0d51f78ecb93ddb334806cb7b4bd60440f3e4303be0e28799" exitCode=0
Jul 16 20:33:10.862505 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:10.862027    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v4qls" event={"ID":"c77ecce4-9890-4cef-bf49-ebac541c6d6d","Type":"ContainerDied","Data":"9322bccfd3f64aa0d51f78ecb93ddb334806cb7b4bd60440f3e4303be0e28799"}
Jul 16 20:33:11.867930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:11.867860    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-v4qls" event={"ID":"c77ecce4-9890-4cef-bf49-ebac541c6d6d","Type":"ContainerStarted","Data":"8a6c82e763175803a56542ad7e4896ab0f7372a7ee7ba10b601b2925760a9da8"}
Jul 16 20:33:11.889255 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:11.889209    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-v4qls" podStartSLOduration=9.605256039 podStartE2EDuration="37.889194579s" totalImagesPullingTime="28.28393854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:32:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:11.88867578 +0000 UTC m=+37.854610013" watchObservedRunningTime="2026-07-16 20:33:11.889194579 +0000 UTC m=+37.855128812"
Jul 16 20:33:12.684628 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:12.684413    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:33:12.684762 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:12.684413    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:33:12.684762 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:12.684721    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:33:12.684834 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:12.684452    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:33:12.684865 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:12.684837    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:33:12.684896 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:12.684870    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:33:14.684400 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:14.684347    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:33:14.684965 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:14.684453    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-q77jc" podUID="6ef282cd-38f3-4e0d-b862-b9d510e70226"
Jul 16 20:33:14.684965 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:14.684473    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:33:14.684965 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:14.684478    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:33:14.684965 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:14.684571    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pr8rp" podUID="2d50105b-d05a-487b-bfa1-2b5167015795"
Jul 16 20:33:14.684965 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:14.684623    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xxh76" podUID="30258292-80a3-4166-a0e5-1ad9bec7d240"
Jul 16 20:33:15.354397 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.354373    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-245.ec2.internal" event="NodeReady"
Jul 16 20:33:15.354523 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.354469    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 16 20:33:15.388683 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.388659    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf"]
Jul 16 20:33:15.391735 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.391716    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf"
Jul 16 20:33:15.393534 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.393514    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh"]
Jul 16 20:33:15.395041 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.394878    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 16 20:33:15.395041 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.394919    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-r4zxc\""
Jul 16 20:33:15.395041 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.394901    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 16 20:33:15.395041 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.395016    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 16 20:33:15.396328 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.396311    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-648f5697fb-f9xb5"]
Jul 16 20:33:15.396561 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.396442    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh"
Jul 16 20:33:15.399062 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.399046    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 16 20:33:15.399157 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.399078    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 16 20:33:15.399915 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.399898    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 16 20:33:15.400011 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.399955    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rtpxj\""
Jul 16 20:33:15.400011 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.399977    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-v65xd"]
Jul 16 20:33:15.400133 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.400108    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.400901 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.400882    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 16 20:33:15.402619 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.402605    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 16 20:33:15.403060 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.403045    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 16 20:33:15.403126 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.403061    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 16 20:33:15.403495 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.403483    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-68jbx\""
Jul 16 20:33:15.404000 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.403984    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-stbb4"]
Jul 16 20:33:15.404077 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.404067    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-v65xd"
Jul 16 20:33:15.406493 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.406466    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-5kdxg\""
Jul 16 20:33:15.406632 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.406616    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 16 20:33:15.406971 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.406942    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 16 20:33:15.407242 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.407225    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-tppbf"]
Jul 16 20:33:15.407304 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.407279    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:15.411000 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.410982    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8"]
Jul 16 20:33:15.411114 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.411097    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.411959 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.411939    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 16 20:33:15.412085 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.412062    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 16 20:33:15.412209 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.412193    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-vx6m8\""
Jul 16 20:33:15.412291 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.412195    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 16 20:33:15.412647 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.412631    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 16 20:33:15.414179 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.414161    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf"]
Jul 16 20:33:15.414265 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.414199    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc"]
Jul 16 20:33:15.414265 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.414259    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8"
Jul 16 20:33:15.414392 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.414373    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 16 20:33:15.414486 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.414471    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 16 20:33:15.414546 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.414476    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 16 20:33:15.416576 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.416556    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 16 20:33:15.418469 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.418451    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 16 20:33:15.418548 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.418535    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 16 20:33:15.418703 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.418689    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"]
Jul 16 20:33:15.418790 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.418777    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc"
Jul 16 20:33:15.420956 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.420752    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-2xrl8\""
Jul 16 20:33:15.421560 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.421273    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 16 20:33:15.426193 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.422205    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9cm6b\""
Jul 16 20:33:15.426193 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.423106    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 16 20:33:15.428969 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.427395    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 16 20:33:15.428969 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.427551    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-phw25\""
Jul 16 20:33:15.428969 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.427741    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 16 20:33:15.428969 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.427813    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 16 20:33:15.428969 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.428436    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 16 20:33:15.428969 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.428691    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-65f496d8cd-qchkt"]
Jul 16 20:33:15.428969 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.428805    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"
Jul 16 20:33:15.431931 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.431613    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hs78"]
Jul 16 20:33:15.432690 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.432623    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:15.434863 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.434762    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 16 20:33:15.434863 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.434777    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 16 20:33:15.435167 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.435112    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 16 20:33:15.435167 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.435124    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-pzfdd\""
Jul 16 20:33:15.435650 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.435632    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 16 20:33:15.435931 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.435901    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 16 20:33:15.436038 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.436010    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 16 20:33:15.436355 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.436334    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qd8ff"]
Jul 16 20:33:15.436477 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.436388    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 16 20:33:15.436477 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.436432    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 16 20:33:15.436582 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.436498    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hs78"
Jul 16 20:33:15.436677 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.436660    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-nqzhf\""
Jul 16 20:33:15.436801 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.436783    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 16 20:33:15.436856 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.436817    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 16 20:33:15.436985 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.436969    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 16 20:33:15.439053 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.439037    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-tppbf"]
Jul 16 20:33:15.439187 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.439062    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-89j4k"]
Jul 16 20:33:15.439187 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.439145    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qd8ff"
Jul 16 20:33:15.442552 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.442323    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-v65xd"]
Jul 16 20:33:15.442552 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.442379    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh"]
Jul 16 20:33:15.442552 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.442405    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-stbb4"]
Jul 16 20:33:15.442552 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.442425    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-648f5697fb-f9xb5"]
Jul 16 20:33:15.442552 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.442442    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc"]
Jul 16 20:33:15.442552 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.442461    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"]
Jul 16 20:33:15.442552 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.442479    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hs78"]
Jul 16 20:33:15.442552 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.442497    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-65f496d8cd-qchkt"]
Jul 16 20:33:15.442552 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.442519    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qd8ff"]
Jul 16 20:33:15.442552 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.442405    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:15.443282 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.443258    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-qlkkd\""
Jul 16 20:33:15.443347 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.443306    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dt299\""
Jul 16 20:33:15.443427 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.443413    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 16 20:33:15.443554 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.443537    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 16 20:33:15.443616 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.443559    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 16 20:33:15.444074 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.444055    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 16 20:33:15.444582 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.444560    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8"]
Jul 16 20:33:15.444725 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.444711    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 16 20:33:15.445623 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.445601    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 16 20:33:15.445714 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.445647    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 16 20:33:15.445714 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.445693    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xlvk5\""
Jul 16 20:33:15.448864 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.448311    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-89j4k"]
Jul 16 20:33:15.537979 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.537952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz2tp\" (UniqueName: \"kubernetes.io/projected/44befb08-b09e-471f-a0bc-59d503cd5c93-kube-api-access-vz2tp\") pod \"console-operator-565b7b569d-stbb4\" (UID: \"44befb08-b09e-471f-a0bc-59d503cd5c93\") " pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:15.538067 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.537988    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3f4bd827-12b2-48f5-b027-c3ba1761b83c-serving-cert\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.538067 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538013    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/373a5e3f-70fc-4a11-bcd9-8c6a136f83b9-serving-cert\") pod \"service-ca-operator-5bcbc94c78-8rkfc\" (UID: \"373a5e3f-70fc-4a11-bcd9-8c6a136f83b9\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc"
Jul 16 20:33:15.538148 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538067    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m6nh\" (UniqueName: \"kubernetes.io/projected/3f4bd827-12b2-48f5-b027-c3ba1761b83c-kube-api-access-9m6nh\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.538148 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538098    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75phv\" (UniqueName: \"kubernetes.io/projected/deea2dc0-618a-48aa-833f-bae435b4cdaa-kube-api-access-75phv\") pod \"volume-data-source-validator-78cf4c6f6-v65xd\" (UID: \"deea2dc0-618a-48aa-833f-bae435b4cdaa\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-v65xd"
Jul 16 20:33:15.538148 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538118    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/be2be990-3e76-486b-bf39-976ddac0225d-image-registry-private-configuration\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.538148 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538135    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/be2be990-3e76-486b-bf39-976ddac0225d-ca-trust-extracted\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.538298 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538154    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/373a5e3f-70fc-4a11-bcd9-8c6a136f83b9-config\") pod \"service-ca-operator-5bcbc94c78-8rkfc\" (UID: \"373a5e3f-70fc-4a11-bcd9-8c6a136f83b9\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc"
Jul 16 20:33:15.538298 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538178    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f4bd827-12b2-48f5-b027-c3ba1761b83c-tmp\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.538298 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538193    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f4bd827-12b2-48f5-b027-c3ba1761b83c-service-ca-bundle\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.538298 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538208    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97z66\" (UniqueName: \"kubernetes.io/projected/41a9e89f-641d-4700-8c75-aefe2296b1f1-kube-api-access-97z66\") pod \"cluster-monitoring-operator-76cf8966f8-mq2hs\" (UID: \"41a9e89f-641d-4700-8c75-aefe2296b1f1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"
Jul 16 20:33:15.538298 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538222    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a3b3a919-d977-4f1e-9a34-e7c7a1368d35-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-nm6rf\" (UID: \"a3b3a919-d977-4f1e-9a34-e7c7a1368d35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf"
Jul 16 20:33:15.538298 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538239    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a3b3a919-d977-4f1e-9a34-e7c7a1368d35-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-nm6rf\" (UID: \"a3b3a919-d977-4f1e-9a34-e7c7a1368d35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf"
Jul 16 20:33:15.538298 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538262    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww5hv\" (UniqueName: \"kubernetes.io/projected/a3b3a919-d977-4f1e-9a34-e7c7a1368d35-kube-api-access-ww5hv\") pod \"kube-storage-version-migrator-operator-7658d9b749-nm6rf\" (UID: \"a3b3a919-d977-4f1e-9a34-e7c7a1368d35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf"
Jul 16 20:33:15.538298 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538287    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-default-certificate\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:15.538551 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538316    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stgmp\" (UniqueName: \"kubernetes.io/projected/353ffdb1-2e24-43bd-abf1-3a907e8dd32f-kube-api-access-stgmp\") pod \"network-check-source-7bc5b6bf85-9hs78\" (UID: \"353ffdb1-2e24-43bd-abf1-3a907e8dd32f\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hs78"
Jul 16 20:33:15.538551 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538341    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/44befb08-b09e-471f-a0bc-59d503cd5c93-config\") pod \"console-operator-565b7b569d-stbb4\" (UID: \"44befb08-b09e-471f-a0bc-59d503cd5c93\") " pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:15.538551 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538379    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44befb08-b09e-471f-a0bc-59d503cd5c93-trusted-ca\") pod \"console-operator-565b7b569d-stbb4\" (UID: \"44befb08-b09e-471f-a0bc-59d503cd5c93\") " pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:15.538551 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538394    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/41a9e89f-641d-4700-8c75-aefe2296b1f1-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-mq2hs\" (UID: \"41a9e89f-641d-4700-8c75-aefe2296b1f1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"
Jul 16 20:33:15.538551 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538410    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:15.538551 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538434    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kzk9\" (UniqueName: \"kubernetes.io/projected/cbb37170-322b-4128-8731-ad11b8117cdc-kube-api-access-2kzk9\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:15.538551 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538451    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c48a73e2-e6f7-468e-b923-927c7ec97110-config-volume\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:15.538551 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538465    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/be2be990-3e76-486b-bf39-976ddac0225d-registry-certificates\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.538551 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538482    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg5bn\" (UniqueName: \"kubernetes.io/projected/373a5e3f-70fc-4a11-bcd9-8c6a136f83b9-kube-api-access-xg5bn\") pod \"service-ca-operator-5bcbc94c78-8rkfc\" (UID: \"373a5e3f-70fc-4a11-bcd9-8c6a136f83b9\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc"
Jul 16 20:33:15.538551 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538508    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:15.538551 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538529    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:15.538551 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538545    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/44befb08-b09e-471f-a0bc-59d503cd5c93-serving-cert\") pod \"console-operator-565b7b569d-stbb4\" (UID: \"44befb08-b09e-471f-a0bc-59d503cd5c93\") " pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:15.538884 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538562    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f4bd827-12b2-48f5-b027-c3ba1761b83c-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.538884 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538582    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/be2be990-3e76-486b-bf39-976ddac0225d-installation-pull-secrets\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.538884 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538601    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-bound-sa-token\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.538884 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538620    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rz9m8\" (UID: \"95d2f323-8204-4405-8320-05650383659f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8"
Jul 16 20:33:15.538884 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538635    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cksg6\" (UniqueName: \"kubernetes.io/projected/fc249a17-1357-464e-be27-57a3a9182c21-kube-api-access-cksg6\") pod \"ingress-canary-qd8ff\" (UID: \"fc249a17-1357-464e-be27-57a3a9182c21\") " pod="openshift-ingress-canary/ingress-canary-qd8ff"
Jul 16 20:33:15.538884 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538682    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c48a73e2-e6f7-468e-b923-927c7ec97110-tmp-dir\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:15.538884 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538702    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4kjf\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-kube-api-access-c4kjf\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.538884 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538728    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9v9kh\" (UID: \"85b78f5b-d6c3-4d9f-adcd-de59794fa938\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh"
Jul 16 20:33:15.538884 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538758    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3f4bd827-12b2-48f5-b027-c3ba1761b83c-snapshots\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.538884 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538775    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.538884 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538811    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be2be990-3e76-486b-bf39-976ddac0225d-trusted-ca\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.538884 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538829    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9ft2\" (UniqueName: \"kubernetes.io/projected/c48a73e2-e6f7-468e-b923-927c7ec97110-kube-api-access-p9ft2\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:15.538884 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538849    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc249a17-1357-464e-be27-57a3a9182c21-cert\") pod \"ingress-canary-qd8ff\" (UID: \"fc249a17-1357-464e-be27-57a3a9182c21\") " pod="openshift-ingress-canary/ingress-canary-qd8ff"
Jul 16 20:33:15.539244 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538862    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-stats-auth\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:15.539244 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mq2hs\" (UID: \"41a9e89f-641d-4700-8c75-aefe2296b1f1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"
Jul 16 20:33:15.539244 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538920    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/95d2f323-8204-4405-8320-05650383659f-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-rz9m8\" (UID: \"95d2f323-8204-4405-8320-05650383659f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8"
Jul 16 20:33:15.539244 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.538956    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcgbt\" (UniqueName: \"kubernetes.io/projected/85b78f5b-d6c3-4d9f-adcd-de59794fa938-kube-api-access-mcgbt\") pod \"cluster-samples-operator-79b7c55f87-9v9kh\" (UID: \"85b78f5b-d6c3-4d9f-adcd-de59794fa938\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh"
Jul 16 20:33:15.640135 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640110    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3f4bd827-12b2-48f5-b027-c3ba1761b83c-snapshots\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.640234 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640140    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.640234 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640166    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be2be990-3e76-486b-bf39-976ddac0225d-trusted-ca\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.640234 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640193    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9ft2\" (UniqueName: \"kubernetes.io/projected/c48a73e2-e6f7-468e-b923-927c7ec97110-kube-api-access-p9ft2\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:15.640425 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:15.640255    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 20:33:15.640425 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:15.640272    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-648f5697fb-f9xb5: secret "image-registry-tls" not found
Jul 16 20:33:15.640425 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640312    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc249a17-1357-464e-be27-57a3a9182c21-cert\") pod \"ingress-canary-qd8ff\" (UID: \"fc249a17-1357-464e-be27-57a3a9182c21\") " pod="openshift-ingress-canary/ingress-canary-qd8ff"
Jul 16 20:33:15.640425 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:15.640318    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls podName:be2be990-3e76-486b-bf39-976ddac0225d nodeName:}" failed. No retries permitted until 2026-07-16 20:33:16.140301833 +0000 UTC m=+42.106236060 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls") pod "image-registry-648f5697fb-f9xb5" (UID: "be2be990-3e76-486b-bf39-976ddac0225d") : secret "image-registry-tls" not found
Jul 16 20:33:15.640425 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640344    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-stats-auth\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:15.640425 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640385    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mq2hs\" (UID: \"41a9e89f-641d-4700-8c75-aefe2296b1f1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"
Jul 16 20:33:15.640663 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:15.640552    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 20:33:15.640663 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640595    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/95d2f323-8204-4405-8320-05650383659f-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-rz9m8\" (UID: \"95d2f323-8204-4405-8320-05650383659f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8"
Jul 16 20:33:15.640663 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:15.640611    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls podName:41a9e89f-641d-4700-8c75-aefe2296b1f1 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:16.140596397 +0000 UTC m=+42.106530629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mq2hs" (UID: "41a9e89f-641d-4700-8c75-aefe2296b1f1") : secret "cluster-monitoring-operator-tls" not found
Jul 16 20:33:15.640663 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640640    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcgbt\" (UniqueName: \"kubernetes.io/projected/85b78f5b-d6c3-4d9f-adcd-de59794fa938-kube-api-access-mcgbt\") pod \"cluster-samples-operator-79b7c55f87-9v9kh\" (UID: \"85b78f5b-d6c3-4d9f-adcd-de59794fa938\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh"
Jul 16 20:33:15.640860 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640672    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz2tp\" (UniqueName: \"kubernetes.io/projected/44befb08-b09e-471f-a0bc-59d503cd5c93-kube-api-access-vz2tp\") pod \"console-operator-565b7b569d-stbb4\" (UID: \"44befb08-b09e-471f-a0bc-59d503cd5c93\") " pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:15.640860 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640692    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3f4bd827-12b2-48f5-b027-c3ba1761b83c-serving-cert\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.640860 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640711    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/373a5e3f-70fc-4a11-bcd9-8c6a136f83b9-serving-cert\") pod \"service-ca-operator-5bcbc94c78-8rkfc\" (UID: \"373a5e3f-70fc-4a11-bcd9-8c6a136f83b9\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc"
Jul 16 20:33:15.640860 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m6nh\" (UniqueName: \"kubernetes.io/projected/3f4bd827-12b2-48f5-b027-c3ba1761b83c-kube-api-access-9m6nh\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.640860 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640755    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75phv\" (UniqueName: \"kubernetes.io/projected/deea2dc0-618a-48aa-833f-bae435b4cdaa-kube-api-access-75phv\") pod \"volume-data-source-validator-78cf4c6f6-v65xd\" (UID: \"deea2dc0-618a-48aa-833f-bae435b4cdaa\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-v65xd"
Jul 16 20:33:15.640860 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640778    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/be2be990-3e76-486b-bf39-976ddac0225d-image-registry-private-configuration\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.640860 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640794    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3f4bd827-12b2-48f5-b027-c3ba1761b83c-snapshots\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.640860 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640799    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/be2be990-3e76-486b-bf39-976ddac0225d-ca-trust-extracted\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.640860 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640823    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/373a5e3f-70fc-4a11-bcd9-8c6a136f83b9-config\") pod \"service-ca-operator-5bcbc94c78-8rkfc\" (UID: \"373a5e3f-70fc-4a11-bcd9-8c6a136f83b9\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc"
Jul 16 20:33:15.640860 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640847    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f4bd827-12b2-48f5-b027-c3ba1761b83c-tmp\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.641404 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640867    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f4bd827-12b2-48f5-b027-c3ba1761b83c-service-ca-bundle\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.641404 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640893    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97z66\" (UniqueName: \"kubernetes.io/projected/41a9e89f-641d-4700-8c75-aefe2296b1f1-kube-api-access-97z66\") pod \"cluster-monitoring-operator-76cf8966f8-mq2hs\" (UID: \"41a9e89f-641d-4700-8c75-aefe2296b1f1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"
Jul 16 20:33:15.641404 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640919    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a3b3a919-d977-4f1e-9a34-e7c7a1368d35-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-nm6rf\" (UID: \"a3b3a919-d977-4f1e-9a34-e7c7a1368d35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf"
Jul 16 20:33:15.641404 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640944    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a3b3a919-d977-4f1e-9a34-e7c7a1368d35-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-nm6rf\" (UID: \"a3b3a919-d977-4f1e-9a34-e7c7a1368d35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf"
Jul 16 20:33:15.641404 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.640974    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww5hv\" (UniqueName: \"kubernetes.io/projected/a3b3a919-d977-4f1e-9a34-e7c7a1368d35-kube-api-access-ww5hv\") pod \"kube-storage-version-migrator-operator-7658d9b749-nm6rf\" (UID: \"a3b3a919-d977-4f1e-9a34-e7c7a1368d35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf"
Jul 16 20:33:15.641404 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641001    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-default-certificate\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:15.641404 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641025    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stgmp\" (UniqueName: \"kubernetes.io/projected/353ffdb1-2e24-43bd-abf1-3a907e8dd32f-kube-api-access-stgmp\") pod \"network-check-source-7bc5b6bf85-9hs78\" (UID: \"353ffdb1-2e24-43bd-abf1-3a907e8dd32f\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hs78"
Jul 16 20:33:15.641404 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641047    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/44befb08-b09e-471f-a0bc-59d503cd5c93-config\") pod \"console-operator-565b7b569d-stbb4\" (UID: \"44befb08-b09e-471f-a0bc-59d503cd5c93\") " pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:15.641404 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641067    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44befb08-b09e-471f-a0bc-59d503cd5c93-trusted-ca\") pod \"console-operator-565b7b569d-stbb4\" (UID: \"44befb08-b09e-471f-a0bc-59d503cd5c93\") " pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:15.641404 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641087    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/41a9e89f-641d-4700-8c75-aefe2296b1f1-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-mq2hs\" (UID: \"41a9e89f-641d-4700-8c75-aefe2296b1f1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"
Jul 16 20:33:15.641404 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641113    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:15.641404 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641139    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kzk9\" (UniqueName: \"kubernetes.io/projected/cbb37170-322b-4128-8731-ad11b8117cdc-kube-api-access-2kzk9\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:15.641404 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641164    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c48a73e2-e6f7-468e-b923-927c7ec97110-config-volume\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:15.642025 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641181    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be2be990-3e76-486b-bf39-976ddac0225d-trusted-ca\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.642025 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641191    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/be2be990-3e76-486b-bf39-976ddac0225d-registry-certificates\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.642025 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641189    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/95d2f323-8204-4405-8320-05650383659f-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-rz9m8\" (UID: \"95d2f323-8204-4405-8320-05650383659f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8"
Jul 16 20:33:15.642025 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641223    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg5bn\" (UniqueName: \"kubernetes.io/projected/373a5e3f-70fc-4a11-bcd9-8c6a136f83b9-kube-api-access-xg5bn\") pod \"service-ca-operator-5bcbc94c78-8rkfc\" (UID: \"373a5e3f-70fc-4a11-bcd9-8c6a136f83b9\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc"
Jul 16 20:33:15.642025 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641262    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:15.642025 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641290    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:15.642025 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641317    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/44befb08-b09e-471f-a0bc-59d503cd5c93-serving-cert\") pod \"console-operator-565b7b569d-stbb4\" (UID: \"44befb08-b09e-471f-a0bc-59d503cd5c93\") " pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:15.642025 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641376    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f4bd827-12b2-48f5-b027-c3ba1761b83c-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.642025 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641403    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/be2be990-3e76-486b-bf39-976ddac0225d-installation-pull-secrets\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.642025 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641432    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-bound-sa-token\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.642025 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641466    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rz9m8\" (UID: \"95d2f323-8204-4405-8320-05650383659f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8"
Jul 16 20:33:15.642025 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641495    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cksg6\" (UniqueName: \"kubernetes.io/projected/fc249a17-1357-464e-be27-57a3a9182c21-kube-api-access-cksg6\") pod \"ingress-canary-qd8ff\" (UID: \"fc249a17-1357-464e-be27-57a3a9182c21\") " pod="openshift-ingress-canary/ingress-canary-qd8ff"
Jul 16 20:33:15.642025 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641547    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c48a73e2-e6f7-468e-b923-927c7ec97110-tmp-dir\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:15.642638 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641555    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/373a5e3f-70fc-4a11-bcd9-8c6a136f83b9-config\") pod \"service-ca-operator-5bcbc94c78-8rkfc\" (UID: \"373a5e3f-70fc-4a11-bcd9-8c6a136f83b9\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc"
Jul 16 20:33:15.642638 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641574    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4kjf\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-kube-api-access-c4kjf\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.642638 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.641602    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9v9kh\" (UID: \"85b78f5b-d6c3-4d9f-adcd-de59794fa938\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh"
Jul 16 20:33:15.642638 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:15.641709    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 20:33:15.642638 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:15.641757    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls podName:85b78f5b-d6c3-4d9f-adcd-de59794fa938 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:16.141744725 +0000 UTC m=+42.107678939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-9v9kh" (UID: "85b78f5b-d6c3-4d9f-adcd-de59794fa938") : secret "samples-operator-tls" not found
Jul 16 20:33:15.642638 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:15.642162    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 20:33:15.642638 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:15.642207    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert podName:95d2f323-8204-4405-8320-05650383659f nodeName:}" failed. No retries permitted until 2026-07-16 20:33:16.142193095 +0000 UTC m=+42.108127306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-rz9m8" (UID: "95d2f323-8204-4405-8320-05650383659f") : secret "networking-console-plugin-cert" not found
Jul 16 20:33:15.642638 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.642440    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c48a73e2-e6f7-468e-b923-927c7ec97110-tmp-dir\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:15.643038 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.642865    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/44befb08-b09e-471f-a0bc-59d503cd5c93-config\") pod \"console-operator-565b7b569d-stbb4\" (UID: \"44befb08-b09e-471f-a0bc-59d503cd5c93\") " pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:15.643038 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.642888    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f4bd827-12b2-48f5-b027-c3ba1761b83c-tmp\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.643038 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:15.642979    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle podName:cbb37170-322b-4128-8731-ad11b8117cdc nodeName:}" failed. No retries permitted until 2026-07-16 20:33:16.142962629 +0000 UTC m=+42.108896860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle") pod "router-default-65f496d8cd-qchkt" (UID: "cbb37170-322b-4128-8731-ad11b8117cdc") : configmap references non-existent config key: service-ca.crt
Jul 16 20:33:15.643180 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:15.643040    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 20:33:15.643180 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:15.643076    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs podName:cbb37170-322b-4128-8731-ad11b8117cdc nodeName:}" failed. No retries permitted until 2026-07-16 20:33:16.143064141 +0000 UTC m=+42.108998567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs") pod "router-default-65f496d8cd-qchkt" (UID: "cbb37170-322b-4128-8731-ad11b8117cdc") : secret "router-metrics-certs-default" not found
Jul 16 20:33:15.643490 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.643466    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f4bd827-12b2-48f5-b027-c3ba1761b83c-service-ca-bundle\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.643561 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.643494    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a3b3a919-d977-4f1e-9a34-e7c7a1368d35-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-nm6rf\" (UID: \"a3b3a919-d977-4f1e-9a34-e7c7a1368d35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf"
Jul 16 20:33:15.644226 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.644204    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f4bd827-12b2-48f5-b027-c3ba1761b83c-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.644384 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.644345    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/41a9e89f-641d-4700-8c75-aefe2296b1f1-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-mq2hs\" (UID: \"41a9e89f-641d-4700-8c75-aefe2296b1f1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"
Jul 16 20:33:15.644469 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.644446    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44befb08-b09e-471f-a0bc-59d503cd5c93-trusted-ca\") pod \"console-operator-565b7b569d-stbb4\" (UID: \"44befb08-b09e-471f-a0bc-59d503cd5c93\") " pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:15.644529 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:15.644453    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 20:33:15.644582 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:15.644529    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls podName:c48a73e2-e6f7-468e-b923-927c7ec97110 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:16.144513169 +0000 UTC m=+42.110447395 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls") pod "dns-default-89j4k" (UID: "c48a73e2-e6f7-468e-b923-927c7ec97110") : secret "dns-default-metrics-tls" not found
Jul 16 20:33:15.644864 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.644845    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/be2be990-3e76-486b-bf39-976ddac0225d-ca-trust-extracted\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.644941 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.644918    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/be2be990-3e76-486b-bf39-976ddac0225d-registry-certificates\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.645279 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.645255    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c48a73e2-e6f7-468e-b923-927c7ec97110-config-volume\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:15.646431 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.646034    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/373a5e3f-70fc-4a11-bcd9-8c6a136f83b9-serving-cert\") pod \"service-ca-operator-5bcbc94c78-8rkfc\" (UID: \"373a5e3f-70fc-4a11-bcd9-8c6a136f83b9\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc"
Jul 16 20:33:15.646431 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.646063    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3f4bd827-12b2-48f5-b027-c3ba1761b83c-serving-cert\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.646431 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.646313    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc249a17-1357-464e-be27-57a3a9182c21-cert\") pod \"ingress-canary-qd8ff\" (UID: \"fc249a17-1357-464e-be27-57a3a9182c21\") " pod="openshift-ingress-canary/ingress-canary-qd8ff"
Jul 16 20:33:15.646431 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.646389    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/44befb08-b09e-471f-a0bc-59d503cd5c93-serving-cert\") pod \"console-operator-565b7b569d-stbb4\" (UID: \"44befb08-b09e-471f-a0bc-59d503cd5c93\") " pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:15.646431 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.646424    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-default-certificate\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:15.646704 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.646637    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a3b3a919-d977-4f1e-9a34-e7c7a1368d35-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-nm6rf\" (UID: \"a3b3a919-d977-4f1e-9a34-e7c7a1368d35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf"
Jul 16 20:33:15.646824 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.646802    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-stats-auth\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:15.647536 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.647513    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/be2be990-3e76-486b-bf39-976ddac0225d-installation-pull-secrets\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.647816 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.647800    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/be2be990-3e76-486b-bf39-976ddac0225d-image-registry-private-configuration\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.650940 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.650893    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz2tp\" (UniqueName: \"kubernetes.io/projected/44befb08-b09e-471f-a0bc-59d503cd5c93-kube-api-access-vz2tp\") pod \"console-operator-565b7b569d-stbb4\" (UID: \"44befb08-b09e-471f-a0bc-59d503cd5c93\") " pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:15.652075 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.651562    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcgbt\" (UniqueName: \"kubernetes.io/projected/85b78f5b-d6c3-4d9f-adcd-de59794fa938-kube-api-access-mcgbt\") pod \"cluster-samples-operator-79b7c55f87-9v9kh\" (UID: \"85b78f5b-d6c3-4d9f-adcd-de59794fa938\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh"
Jul 16 20:33:15.653128 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.653084    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9ft2\" (UniqueName: \"kubernetes.io/projected/c48a73e2-e6f7-468e-b923-927c7ec97110-kube-api-access-p9ft2\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:15.653224 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.653140    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cksg6\" (UniqueName: \"kubernetes.io/projected/fc249a17-1357-464e-be27-57a3a9182c21-kube-api-access-cksg6\") pod \"ingress-canary-qd8ff\" (UID: \"fc249a17-1357-464e-be27-57a3a9182c21\") " pod="openshift-ingress-canary/ingress-canary-qd8ff"
Jul 16 20:33:15.653451 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.653402    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stgmp\" (UniqueName: \"kubernetes.io/projected/353ffdb1-2e24-43bd-abf1-3a907e8dd32f-kube-api-access-stgmp\") pod \"network-check-source-7bc5b6bf85-9hs78\" (UID: \"353ffdb1-2e24-43bd-abf1-3a907e8dd32f\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hs78"
Jul 16 20:33:15.654467 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.654428    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4kjf\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-kube-api-access-c4kjf\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.654559 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.654470    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww5hv\" (UniqueName: \"kubernetes.io/projected/a3b3a919-d977-4f1e-9a34-e7c7a1368d35-kube-api-access-ww5hv\") pod \"kube-storage-version-migrator-operator-7658d9b749-nm6rf\" (UID: \"a3b3a919-d977-4f1e-9a34-e7c7a1368d35\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf"
Jul 16 20:33:15.654822 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.654783    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg5bn\" (UniqueName: \"kubernetes.io/projected/373a5e3f-70fc-4a11-bcd9-8c6a136f83b9-kube-api-access-xg5bn\") pod \"service-ca-operator-5bcbc94c78-8rkfc\" (UID: \"373a5e3f-70fc-4a11-bcd9-8c6a136f83b9\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc"
Jul 16 20:33:15.655070 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.655026    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kzk9\" (UniqueName: \"kubernetes.io/projected/cbb37170-322b-4128-8731-ad11b8117cdc-kube-api-access-2kzk9\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:15.655929 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.655909    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-bound-sa-token\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:15.655929 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.655918    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97z66\" (UniqueName: \"kubernetes.io/projected/41a9e89f-641d-4700-8c75-aefe2296b1f1-kube-api-access-97z66\") pod \"cluster-monitoring-operator-76cf8966f8-mq2hs\" (UID: \"41a9e89f-641d-4700-8c75-aefe2296b1f1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"
Jul 16 20:33:15.656067 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.656051    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m6nh\" (UniqueName: \"kubernetes.io/projected/3f4bd827-12b2-48f5-b027-c3ba1761b83c-kube-api-access-9m6nh\") pod \"insights-operator-555d9d7885-tppbf\" (UID: \"3f4bd827-12b2-48f5-b027-c3ba1761b83c\") " pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.656683 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.656661    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75phv\" (UniqueName: \"kubernetes.io/projected/deea2dc0-618a-48aa-833f-bae435b4cdaa-kube-api-access-75phv\") pod \"volume-data-source-validator-78cf4c6f6-v65xd\" (UID: \"deea2dc0-618a-48aa-833f-bae435b4cdaa\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-v65xd"
Jul 16 20:33:15.704795 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.704751    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf"
Jul 16 20:33:15.736587 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.736572    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-v65xd"
Jul 16 20:33:15.744884 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.744866    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:15.753022 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.753006    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-tppbf"
Jul 16 20:33:15.764023 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.764003    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc"
Jul 16 20:33:15.783419 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.783399    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hs78"
Jul 16 20:33:15.789854 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.789827    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qd8ff"
Jul 16 20:33:15.922199 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.921714    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-v65xd"]
Jul 16 20:33:15.927089 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.926953    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf"]
Jul 16 20:33:15.987007 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:15.986739    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-stbb4"]
Jul 16 20:33:15.987595 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:33:15.987558    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44befb08_b09e_471f_a0bc_59d503cd5c93.slice/crio-7e0a955fa59d99880c8c0c5d57b82057ebabf14fcb9930ff404d0c53cece712c WatchSource:0}: Error finding container 7e0a955fa59d99880c8c0c5d57b82057ebabf14fcb9930ff404d0c53cece712c: Status 404 returned error can't find the container with id 7e0a955fa59d99880c8c0c5d57b82057ebabf14fcb9930ff404d0c53cece712c
Jul 16 20:33:16.009992 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.009625    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc"]
Jul 16 20:33:16.014184 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:33:16.013724    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod373a5e3f_70fc_4a11_bcd9_8c6a136f83b9.slice/crio-5f690d3409731442e79d44a4fcd50dddcf230fed925569564a063c40c55257f1 WatchSource:0}: Error finding container 5f690d3409731442e79d44a4fcd50dddcf230fed925569564a063c40c55257f1: Status 404 returned error can't find the container with id 5f690d3409731442e79d44a4fcd50dddcf230fed925569564a063c40c55257f1
Jul 16 20:33:16.059688 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.059659    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-tppbf"]
Jul 16 20:33:16.066332 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.066308    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qd8ff"]
Jul 16 20:33:16.068707 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:33:16.068686    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f4bd827_12b2_48f5_b027_c3ba1761b83c.slice/crio-a105697fc5615ca9d07e6de7ef247acc11d24d9366b1721e9a0b1675f9715436 WatchSource:0}: Error finding container a105697fc5615ca9d07e6de7ef247acc11d24d9366b1721e9a0b1675f9715436: Status 404 returned error can't find the container with id a105697fc5615ca9d07e6de7ef247acc11d24d9366b1721e9a0b1675f9715436
Jul 16 20:33:16.069278 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:33:16.069230    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc249a17_1357_464e_be27_57a3a9182c21.slice/crio-2e6ec67c4a15ee087b8d2f148deedfb78febd868f767a7189b56344307cbc953 WatchSource:0}: Error finding container 2e6ec67c4a15ee087b8d2f148deedfb78febd868f767a7189b56344307cbc953: Status 404 returned error can't find the container with id 2e6ec67c4a15ee087b8d2f148deedfb78febd868f767a7189b56344307cbc953
Jul 16 20:33:16.072194 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.072098    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hs78"]
Jul 16 20:33:16.075329 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:33:16.075309    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod353ffdb1_2e24_43bd_abf1_3a907e8dd32f.slice/crio-76eeca45f27c3026a0aa28ad925e235a0cc056294ff8d38e3d183d205b4dc34a WatchSource:0}: Error finding container 76eeca45f27c3026a0aa28ad925e235a0cc056294ff8d38e3d183d205b4dc34a: Status 404 returned error can't find the container with id 76eeca45f27c3026a0aa28ad925e235a0cc056294ff8d38e3d183d205b4dc34a
Jul 16 20:33:16.149654 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.149628    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rz9m8\" (UID: \"95d2f323-8204-4405-8320-05650383659f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8"
Jul 16 20:33:16.149735 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.149667    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9v9kh\" (UID: \"85b78f5b-d6c3-4d9f-adcd-de59794fa938\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh"
Jul 16 20:33:16.149735 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.149690    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:16.149735 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.149731    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mq2hs\" (UID: \"41a9e89f-641d-4700-8c75-aefe2296b1f1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"
Jul 16 20:33:16.149852 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:16.149767    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 20:33:16.149852 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:16.149774    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 20:33:16.149852 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.149776    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:16.149852 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:16.149828    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert podName:95d2f323-8204-4405-8320-05650383659f nodeName:}" failed. No retries permitted until 2026-07-16 20:33:17.149807019 +0000 UTC m=+43.115741245 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-rz9m8" (UID: "95d2f323-8204-4405-8320-05650383659f") : secret "networking-console-plugin-cert" not found
Jul 16 20:33:16.149852 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:16.149835    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 20:33:16.149852 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:16.149848    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 20:33:16.150131 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:16.149861    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-648f5697fb-f9xb5: secret "image-registry-tls" not found
Jul 16 20:33:16.150131 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:16.149879    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls podName:85b78f5b-d6c3-4d9f-adcd-de59794fa938 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:17.149866881 +0000 UTC m=+43.115801092 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-9v9kh" (UID: "85b78f5b-d6c3-4d9f-adcd-de59794fa938") : secret "samples-operator-tls" not found
Jul 16 20:33:16.150131 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:16.149893    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls podName:be2be990-3e76-486b-bf39-976ddac0225d nodeName:}" failed. No retries permitted until 2026-07-16 20:33:17.149887655 +0000 UTC m=+43.115821865 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls") pod "image-registry-648f5697fb-f9xb5" (UID: "be2be990-3e76-486b-bf39-976ddac0225d") : secret "image-registry-tls" not found
Jul 16 20:33:16.150131 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:16.149907    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls podName:41a9e89f-641d-4700-8c75-aefe2296b1f1 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:17.149902154 +0000 UTC m=+43.115836365 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mq2hs" (UID: "41a9e89f-641d-4700-8c75-aefe2296b1f1") : secret "cluster-monitoring-operator-tls" not found
Jul 16 20:33:16.150131 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:16.149946    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 20:33:16.150131 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:16.149983    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 20:33:16.150131 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:16.149997    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls podName:c48a73e2-e6f7-468e-b923-927c7ec97110 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:17.149981839 +0000 UTC m=+43.115916065 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls") pod "dns-default-89j4k" (UID: "c48a73e2-e6f7-468e-b923-927c7ec97110") : secret "dns-default-metrics-tls" not found
Jul 16 20:33:16.150131 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.149947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:16.150131 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:16.150018    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs podName:cbb37170-322b-4128-8731-ad11b8117cdc nodeName:}" failed. No retries permitted until 2026-07-16 20:33:17.150007341 +0000 UTC m=+43.115941552 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs") pod "router-default-65f496d8cd-qchkt" (UID: "cbb37170-322b-4128-8731-ad11b8117cdc") : secret "router-metrics-certs-default" not found
Jul 16 20:33:16.150131 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.150064    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:16.150494 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:16.150179    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle podName:cbb37170-322b-4128-8731-ad11b8117cdc nodeName:}" failed. No retries permitted until 2026-07-16 20:33:17.150166872 +0000 UTC m=+43.116101083 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle") pod "router-default-65f496d8cd-qchkt" (UID: "cbb37170-322b-4128-8731-ad11b8117cdc") : configmap references non-existent config key: service-ca.crt
Jul 16 20:33:16.686913 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.685478    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:33:16.686913 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.686235    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:33:16.686913 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.686601    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:33:16.688521 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.688497    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 20:33:16.689948 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.689420    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ql74g\""
Jul 16 20:33:16.689948 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.689654    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pjwwv\""
Jul 16 20:33:16.689948 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.689823    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 20:33:16.883512 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.883439    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc" event={"ID":"373a5e3f-70fc-4a11-bcd9-8c6a136f83b9","Type":"ContainerStarted","Data":"5f690d3409731442e79d44a4fcd50dddcf230fed925569564a063c40c55257f1"}
Jul 16 20:33:16.886733 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.886681    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf" event={"ID":"a3b3a919-d977-4f1e-9a34-e7c7a1368d35","Type":"ContainerStarted","Data":"6b15fef41d7a17f6da47291dc3dd983974338b50cd431ce28b41df91ed707816"}
Jul 16 20:33:16.888812 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.888654    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-stbb4" event={"ID":"44befb08-b09e-471f-a0bc-59d503cd5c93","Type":"ContainerStarted","Data":"7e0a955fa59d99880c8c0c5d57b82057ebabf14fcb9930ff404d0c53cece712c"}
Jul 16 20:33:16.890275 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.890250    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hs78" event={"ID":"353ffdb1-2e24-43bd-abf1-3a907e8dd32f","Type":"ContainerStarted","Data":"76eeca45f27c3026a0aa28ad925e235a0cc056294ff8d38e3d183d205b4dc34a"}
Jul 16 20:33:16.892085 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.892027    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-tppbf" event={"ID":"3f4bd827-12b2-48f5-b027-c3ba1761b83c","Type":"ContainerStarted","Data":"a105697fc5615ca9d07e6de7ef247acc11d24d9366b1721e9a0b1675f9715436"}
Jul 16 20:33:16.894678 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.894602    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qd8ff" event={"ID":"fc249a17-1357-464e-be27-57a3a9182c21","Type":"ContainerStarted","Data":"2e6ec67c4a15ee087b8d2f148deedfb78febd868f767a7189b56344307cbc953"}
Jul 16 20:33:16.897430 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:16.897406    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-v65xd" event={"ID":"deea2dc0-618a-48aa-833f-bae435b4cdaa","Type":"ContainerStarted","Data":"ea1c028e1c5ab10d04e91c9a40d2ccfbe6d1176eb29106139b509fb2cc6bf776"}
Jul 16 20:33:17.159547 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:17.158130    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:17.159547 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:17.158203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mq2hs\" (UID: \"41a9e89f-641d-4700-8c75-aefe2296b1f1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"
Jul 16 20:33:17.159547 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:17.158284    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:17.159547 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:17.158321    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:17.159547 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:17.158346    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:17.159547 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:17.158398    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rz9m8\" (UID: \"95d2f323-8204-4405-8320-05650383659f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8"
Jul 16 20:33:17.159547 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:17.158444    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9v9kh\" (UID: \"85b78f5b-d6c3-4d9f-adcd-de59794fa938\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh"
Jul 16 20:33:17.159547 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:17.158565    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 20:33:17.159547 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:17.158625    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls podName:85b78f5b-d6c3-4d9f-adcd-de59794fa938 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:19.158607273 +0000 UTC m=+45.124541490 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-9v9kh" (UID: "85b78f5b-d6c3-4d9f-adcd-de59794fa938") : secret "samples-operator-tls" not found
Jul 16 20:33:17.159547 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:17.159028    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 20:33:17.159547 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:17.159045    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-648f5697fb-f9xb5: secret "image-registry-tls" not found
Jul 16 20:33:17.159547 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:17.159095    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls podName:be2be990-3e76-486b-bf39-976ddac0225d nodeName:}" failed. No retries permitted until 2026-07-16 20:33:19.159077549 +0000 UTC m=+45.125011796 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls") pod "image-registry-648f5697fb-f9xb5" (UID: "be2be990-3e76-486b-bf39-976ddac0225d") : secret "image-registry-tls" not found
Jul 16 20:33:17.159547 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:17.159139    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 20:33:17.160273 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:17.159174    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls podName:41a9e89f-641d-4700-8c75-aefe2296b1f1 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:19.159162575 +0000 UTC m=+45.125096786 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mq2hs" (UID: "41a9e89f-641d-4700-8c75-aefe2296b1f1") : secret "cluster-monitoring-operator-tls" not found
Jul 16 20:33:17.160273 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:17.159228    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 20:33:17.160273 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:17.159263    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls podName:c48a73e2-e6f7-468e-b923-927c7ec97110 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:19.159249996 +0000 UTC m=+45.125184210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls") pod "dns-default-89j4k" (UID: "c48a73e2-e6f7-468e-b923-927c7ec97110") : secret "dns-default-metrics-tls" not found
Jul 16 20:33:17.160273 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:17.159311    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 20:33:17.160273 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:17.159343    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs podName:cbb37170-322b-4128-8731-ad11b8117cdc nodeName:}" failed. No retries permitted until 2026-07-16 20:33:19.159332271 +0000 UTC m=+45.125266487 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs") pod "router-default-65f496d8cd-qchkt" (UID: "cbb37170-322b-4128-8731-ad11b8117cdc") : secret "router-metrics-certs-default" not found
Jul 16 20:33:17.160273 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:17.159421    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle podName:cbb37170-322b-4128-8731-ad11b8117cdc nodeName:}" failed. No retries permitted until 2026-07-16 20:33:19.159410641 +0000 UTC m=+45.125344854 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle") pod "router-default-65f496d8cd-qchkt" (UID: "cbb37170-322b-4128-8731-ad11b8117cdc") : configmap references non-existent config key: service-ca.crt
Jul 16 20:33:17.160273 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:17.159477    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 20:33:17.160273 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:17.159507    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert podName:95d2f323-8204-4405-8320-05650383659f nodeName:}" failed. No retries permitted until 2026-07-16 20:33:19.159496267 +0000 UTC m=+45.125430484 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-rz9m8" (UID: "95d2f323-8204-4405-8320-05650383659f") : secret "networking-console-plugin-cert" not found
Jul 16 20:33:19.180340 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:19.180311    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9v9kh\" (UID: \"85b78f5b-d6c3-4d9f-adcd-de59794fa938\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh"
Jul 16 20:33:19.180857 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:19.180354    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:19.180857 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:19.180421    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mq2hs\" (UID: \"41a9e89f-641d-4700-8c75-aefe2296b1f1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"
Jul 16 20:33:19.180857 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:19.180465    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 20:33:19.180857 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:19.180499    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:19.180857 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:19.180518    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 20:33:19.180857 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:19.180537    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-648f5697fb-f9xb5: secret "image-registry-tls" not found
Jul 16 20:33:19.180857 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:19.180543    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls podName:85b78f5b-d6c3-4d9f-adcd-de59794fa938 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:23.18052237 +0000 UTC m=+49.146456582 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-9v9kh" (UID: "85b78f5b-d6c3-4d9f-adcd-de59794fa938") : secret "samples-operator-tls" not found
Jul 16 20:33:19.180857 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:19.180568    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:19.180857 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:19.180579    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls podName:be2be990-3e76-486b-bf39-976ddac0225d nodeName:}" failed. No retries permitted until 2026-07-16 20:33:23.180566184 +0000 UTC m=+49.146500395 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls") pod "image-registry-648f5697fb-f9xb5" (UID: "be2be990-3e76-486b-bf39-976ddac0225d") : secret "image-registry-tls" not found
Jul 16 20:33:19.180857 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:19.180591    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 20:33:19.180857 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:19.180605    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:19.180857 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:19.180626    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 20:33:19.180857 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:19.180633    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rz9m8\" (UID: \"95d2f323-8204-4405-8320-05650383659f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8"
Jul 16 20:33:19.180857 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:19.180648    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls podName:41a9e89f-641d-4700-8c75-aefe2296b1f1 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:23.180635211 +0000 UTC m=+49.146569442 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mq2hs" (UID: "41a9e89f-641d-4700-8c75-aefe2296b1f1") : secret "cluster-monitoring-operator-tls" not found
Jul 16 20:33:19.181266 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:19.180592    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 20:33:19.181266 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:19.180673    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs podName:cbb37170-322b-4128-8731-ad11b8117cdc nodeName:}" failed. No retries permitted until 2026-07-16 20:33:23.180662403 +0000 UTC m=+49.146596619 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs") pod "router-default-65f496d8cd-qchkt" (UID: "cbb37170-322b-4128-8731-ad11b8117cdc") : secret "router-metrics-certs-default" not found
Jul 16 20:33:19.181266 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:19.180682    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 20:33:19.181266 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:19.180698    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls podName:c48a73e2-e6f7-468e-b923-927c7ec97110 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:23.180691271 +0000 UTC m=+49.146625482 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls") pod "dns-default-89j4k" (UID: "c48a73e2-e6f7-468e-b923-927c7ec97110") : secret "dns-default-metrics-tls" not found
Jul 16 20:33:19.181266 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:19.180734    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert podName:95d2f323-8204-4405-8320-05650383659f nodeName:}" failed. No retries permitted until 2026-07-16 20:33:23.180727417 +0000 UTC m=+49.146661628 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-rz9m8" (UID: "95d2f323-8204-4405-8320-05650383659f") : secret "networking-console-plugin-cert" not found
Jul 16 20:33:19.181266 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:19.180760    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle podName:cbb37170-322b-4128-8731-ad11b8117cdc nodeName:}" failed. No retries permitted until 2026-07-16 20:33:23.180746014 +0000 UTC m=+49.146680240 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle") pod "router-default-65f496d8cd-qchkt" (UID: "cbb37170-322b-4128-8731-ad11b8117cdc") : configmap references non-existent config key: service-ca.crt
Jul 16 20:33:23.220932 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.219181    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mq2hs\" (UID: \"41a9e89f-641d-4700-8c75-aefe2296b1f1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"
Jul 16 20:33:23.220932 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.219540    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:23.220932 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.219592    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:23.220932 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.219622    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:23.220932 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.219661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rz9m8\" (UID: \"95d2f323-8204-4405-8320-05650383659f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8"
Jul 16 20:33:23.220932 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.219706    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9v9kh\" (UID: \"85b78f5b-d6c3-4d9f-adcd-de59794fa938\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh"
Jul 16 20:33:23.220932 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.219738    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:23.220932 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:23.219894    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 20:33:23.220932 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:23.219909    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-648f5697fb-f9xb5: secret "image-registry-tls" not found
Jul 16 20:33:23.220932 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:23.219966    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls podName:be2be990-3e76-486b-bf39-976ddac0225d nodeName:}" failed. No retries permitted until 2026-07-16 20:33:31.219946058 +0000 UTC m=+57.185880269 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls") pod "image-registry-648f5697fb-f9xb5" (UID: "be2be990-3e76-486b-bf39-976ddac0225d") : secret "image-registry-tls" not found
Jul 16 20:33:23.220932 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:23.220396    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 20:33:23.220932 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:23.220449    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls podName:c48a73e2-e6f7-468e-b923-927c7ec97110 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:31.220432378 +0000 UTC m=+57.186366592 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls") pod "dns-default-89j4k" (UID: "c48a73e2-e6f7-468e-b923-927c7ec97110") : secret "dns-default-metrics-tls" not found
Jul 16 20:33:23.220932 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:23.220508    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 20:33:23.220932 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:23.220538    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs podName:cbb37170-322b-4128-8731-ad11b8117cdc nodeName:}" failed. No retries permitted until 2026-07-16 20:33:31.220527512 +0000 UTC m=+57.186461738 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs") pod "router-default-65f496d8cd-qchkt" (UID: "cbb37170-322b-4128-8731-ad11b8117cdc") : secret "router-metrics-certs-default" not found
Jul 16 20:33:23.222326 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:23.220602    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle podName:cbb37170-322b-4128-8731-ad11b8117cdc nodeName:}" failed. No retries permitted until 2026-07-16 20:33:31.22059243 +0000 UTC m=+57.186526656 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle") pod "router-default-65f496d8cd-qchkt" (UID: "cbb37170-322b-4128-8731-ad11b8117cdc") : configmap references non-existent config key: service-ca.crt
Jul 16 20:33:23.222326 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:23.220659    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 20:33:23.222326 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:23.220690    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert podName:95d2f323-8204-4405-8320-05650383659f nodeName:}" failed. No retries permitted until 2026-07-16 20:33:31.220680181 +0000 UTC m=+57.186614394 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-rz9m8" (UID: "95d2f323-8204-4405-8320-05650383659f") : secret "networking-console-plugin-cert" not found
Jul 16 20:33:23.222326 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:23.220742    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 20:33:23.222326 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:23.220769    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls podName:85b78f5b-d6c3-4d9f-adcd-de59794fa938 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:31.220761507 +0000 UTC m=+57.186695723 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-9v9kh" (UID: "85b78f5b-d6c3-4d9f-adcd-de59794fa938") : secret "samples-operator-tls" not found
Jul 16 20:33:23.222326 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:23.220827    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 20:33:23.222326 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:23.220858    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls podName:41a9e89f-641d-4700-8c75-aefe2296b1f1 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:31.220849211 +0000 UTC m=+57.186783425 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mq2hs" (UID: "41a9e89f-641d-4700-8c75-aefe2296b1f1") : secret "cluster-monitoring-operator-tls" not found
Jul 16 20:33:23.919166 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.919135    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/0.log"
Jul 16 20:33:23.919456 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.919182    2576 generic.go:356] "Generic (PLEG): container finished" podID="44befb08-b09e-471f-a0bc-59d503cd5c93" containerID="1822e09561647a359f52d096e4c5bc9602d7eb88bbcbebba76325b1e32661fc3" exitCode=255
Jul 16 20:33:23.919456 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.919280    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-stbb4" event={"ID":"44befb08-b09e-471f-a0bc-59d503cd5c93","Type":"ContainerDied","Data":"1822e09561647a359f52d096e4c5bc9602d7eb88bbcbebba76325b1e32661fc3"}
Jul 16 20:33:23.919596 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.919553    2576 scope.go:117] "RemoveContainer" containerID="1822e09561647a359f52d096e4c5bc9602d7eb88bbcbebba76325b1e32661fc3"
Jul 16 20:33:23.921319 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.921297    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hs78" event={"ID":"353ffdb1-2e24-43bd-abf1-3a907e8dd32f","Type":"ContainerStarted","Data":"6e3d31bc27e9b540e016a1555185daacaa5faec51598c88e0ca7895da8c3cdf8"}
Jul 16 20:33:23.923470 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.922736    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-tppbf" event={"ID":"3f4bd827-12b2-48f5-b027-c3ba1761b83c","Type":"ContainerStarted","Data":"2815f0e21a7ef163516acd1eaabb8efd38ec18a904f74d0f2f1f96409700c072"}
Jul 16 20:33:23.924556 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.924533    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qd8ff" event={"ID":"fc249a17-1357-464e-be27-57a3a9182c21","Type":"ContainerStarted","Data":"2aae95b23b5ac4e06d6b2abc8c6ed9994baa3f1646bdf49fcd078e17e37ba2ab"}
Jul 16 20:33:23.926313 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.926292    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-v65xd" event={"ID":"deea2dc0-618a-48aa-833f-bae435b4cdaa","Type":"ContainerStarted","Data":"c4c5da032d4ebf5d6dd09f4dec939cfbd6baaa14fa1538adec5823deeea7b024"}
Jul 16 20:33:23.928025 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.928001    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc" event={"ID":"373a5e3f-70fc-4a11-bcd9-8c6a136f83b9","Type":"ContainerStarted","Data":"ba31643f98f7927ab2bec30b807670c8251888457565df260f37353cfdd64ab6"}
Jul 16 20:33:23.929924 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.929894    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf" event={"ID":"a3b3a919-d977-4f1e-9a34-e7c7a1368d35","Type":"ContainerStarted","Data":"c65d1668483104144f6b50152bd865fe20b225bc5ea5cfd86da961fe8272ebbc"}
Jul 16 20:33:23.954412 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.954355    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hs78" podStartSLOduration=28.027228077 podStartE2EDuration="34.954342821s" totalImagesPullingTime="6.927114744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:32:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:23.95392373 +0000 UTC m=+49.919857965" watchObservedRunningTime="2026-07-16 20:33:23.954342821 +0000 UTC m=+49.920277100"
Jul 16 20:33:23.971484 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:23.971439    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-tppbf" podStartSLOduration=28.054824503 podStartE2EDuration="34.971422878s" totalImagesPullingTime="6.916598375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:32:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:23.970236708 +0000 UTC m=+49.936170942" watchObservedRunningTime="2026-07-16 20:33:23.971422878 +0000 UTC m=+49.937357093"
Jul 16 20:33:24.004111 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.002688    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qd8ff" podStartSLOduration=2.081080162 podStartE2EDuration="9.002669205s" totalImagesPullingTime="6.921589043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:24.002290111 +0000 UTC m=+49.968224338" watchObservedRunningTime="2026-07-16 20:33:24.002669205 +0000 UTC m=+49.968603440"
Jul 16 20:33:24.004111 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.003559    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc" podStartSLOduration=28.031922402 podStartE2EDuration="35.003544037s" totalImagesPullingTime="6.971621635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:32:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:23.9855474 +0000 UTC m=+49.951481634" watchObservedRunningTime="2026-07-16 20:33:24.003544037 +0000 UTC m=+49.969478269"
Jul 16 20:33:24.020225 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.020179    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-v65xd" podStartSLOduration=27.976724466 podStartE2EDuration="35.020160129s" totalImagesPullingTime="7.043435663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:32:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:24.019245066 +0000 UTC m=+49.985179300" watchObservedRunningTime="2026-07-16 20:33:24.020160129 +0000 UTC m=+49.986094362"
Jul 16 20:33:24.034394 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.034338    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf" podStartSLOduration=27.989702645 podStartE2EDuration="35.034323013s" totalImagesPullingTime="7.044620368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:32:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:24.033653302 +0000 UTC m=+49.999587534" watchObservedRunningTime="2026-07-16 20:33:24.034323013 +0000 UTC m=+50.000257247"
Jul 16 20:33:24.798055 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.798024    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-w8wzk"]
Jul 16 20:33:24.800317 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.800295    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:24.802942 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.802915    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-khzsm\""
Jul 16 20:33:24.803328 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.803311    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 16 20:33:24.803564 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.803545    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 16 20:33:24.813123 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.813091    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w8wzk"]
Jul 16 20:33:24.934989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.934969    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 20:33:24.935372 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.935343    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/0.log"
Jul 16 20:33:24.935435 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.935397    2576 generic.go:356] "Generic (PLEG): container finished" podID="44befb08-b09e-471f-a0bc-59d503cd5c93" containerID="c7d55420bb4d44a4fb9431c326c5dd784222b3fc2947058b04b0986ab8b5a8d8" exitCode=255
Jul 16 20:33:24.935507 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.935486    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-stbb4" event={"ID":"44befb08-b09e-471f-a0bc-59d503cd5c93","Type":"ContainerDied","Data":"c7d55420bb4d44a4fb9431c326c5dd784222b3fc2947058b04b0986ab8b5a8d8"}
Jul 16 20:33:24.935551 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.935535    2576 scope.go:117] "RemoveContainer" containerID="1822e09561647a359f52d096e4c5bc9602d7eb88bbcbebba76325b1e32661fc3"
Jul 16 20:33:24.935660 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.935639    2576 scope.go:117] "RemoveContainer" containerID="c7d55420bb4d44a4fb9431c326c5dd784222b3fc2947058b04b0986ab8b5a8d8"
Jul 16 20:33:24.935868 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:24.935828    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-stbb4_openshift-console-operator(44befb08-b09e-471f-a0bc-59d503cd5c93)\"" pod="openshift-console-operator/console-operator-565b7b569d-stbb4" podUID="44befb08-b09e-471f-a0bc-59d503cd5c93"
Jul 16 20:33:24.937854 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.937837    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff8aa4a9-dff1-4881-9148-c18582ee7864-crio-socket\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:24.937948 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.937931    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx4pf\" (UniqueName: \"kubernetes.io/projected/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-api-access-xx4pf\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:24.938002 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.937976    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff8aa4a9-dff1-4881-9148-c18582ee7864-data-volume\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:24.938042 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.938020    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:24.938116 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:24.938098    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:25.038956 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:25.038933    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff8aa4a9-dff1-4881-9148-c18582ee7864-data-volume\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:25.039070 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:25.038989    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:25.039132 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:25.039113    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 20:33:25.039205 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:25.039184    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls podName:ff8aa4a9-dff1-4881-9148-c18582ee7864 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:25.539164467 +0000 UTC m=+51.505098699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w8wzk" (UID: "ff8aa4a9-dff1-4881-9148-c18582ee7864") : secret "insights-runtime-extractor-tls" not found
Jul 16 20:33:25.039303 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:25.039282    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff8aa4a9-dff1-4881-9148-c18582ee7864-data-volume\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:25.039440 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:25.039420    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:25.039602 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:25.039584    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 20:33:25.039692 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:25.039669    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff8aa4a9-dff1-4881-9148-c18582ee7864-crio-socket\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:25.039763 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:25.039722    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm podName:ff8aa4a9-dff1-4881-9148-c18582ee7864 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:25.539704161 +0000 UTC m=+51.505638606 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w8wzk" (UID: "ff8aa4a9-dff1-4881-9148-c18582ee7864") : configmap "kube-rbac-proxy" not found
Jul 16 20:33:25.039940 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:25.039916    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ff8aa4a9-dff1-4881-9148-c18582ee7864-crio-socket\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:25.040022 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:25.039997    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx4pf\" (UniqueName: \"kubernetes.io/projected/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-api-access-xx4pf\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:25.049844 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:25.049794    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx4pf\" (UniqueName: \"kubernetes.io/projected/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-api-access-xx4pf\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:25.543475 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:25.543447    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:25.543581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:25.543495    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:25.543581 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:25.543562    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 20:33:25.543656 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:25.543588    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 20:33:25.543656 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:25.543628    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm podName:ff8aa4a9-dff1-4881-9148-c18582ee7864 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:26.543613149 +0000 UTC m=+52.509547360 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w8wzk" (UID: "ff8aa4a9-dff1-4881-9148-c18582ee7864") : configmap "kube-rbac-proxy" not found
Jul 16 20:33:25.543656 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:25.543644    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls podName:ff8aa4a9-dff1-4881-9148-c18582ee7864 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:26.543637249 +0000 UTC m=+52.509571459 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w8wzk" (UID: "ff8aa4a9-dff1-4881-9148-c18582ee7864") : secret "insights-runtime-extractor-tls" not found
Jul 16 20:33:25.745096 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:25.745067    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:25.745210 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:25.745115    2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:25.941413 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:25.941392    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 20:33:25.941740 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:25.941723    2576 scope.go:117] "RemoveContainer" containerID="c7d55420bb4d44a4fb9431c326c5dd784222b3fc2947058b04b0986ab8b5a8d8"
Jul 16 20:33:25.941902 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:25.941875    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-stbb4_openshift-console-operator(44befb08-b09e-471f-a0bc-59d503cd5c93)\"" pod="openshift-console-operator/console-operator-565b7b569d-stbb4" podUID="44befb08-b09e-471f-a0bc-59d503cd5c93"
Jul 16 20:33:26.552680 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:26.552652    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:26.552812 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:26.552698    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:26.552812 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:26.552800    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 20:33:26.552903 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:26.552836    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 20:33:26.552903 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:26.552850    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls podName:ff8aa4a9-dff1-4881-9148-c18582ee7864 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:28.552837469 +0000 UTC m=+54.518771679 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w8wzk" (UID: "ff8aa4a9-dff1-4881-9148-c18582ee7864") : secret "insights-runtime-extractor-tls" not found
Jul 16 20:33:26.552903 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:26.552874    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm podName:ff8aa4a9-dff1-4881-9148-c18582ee7864 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:28.552863724 +0000 UTC m=+54.518797936 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w8wzk" (UID: "ff8aa4a9-dff1-4881-9148-c18582ee7864") : configmap "kube-rbac-proxy" not found
Jul 16 20:33:26.945919 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:26.945885    2576 scope.go:117] "RemoveContainer" containerID="c7d55420bb4d44a4fb9431c326c5dd784222b3fc2947058b04b0986ab8b5a8d8"
Jul 16 20:33:26.946383 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:26.946173    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-stbb4_openshift-console-operator(44befb08-b09e-471f-a0bc-59d503cd5c93)\"" pod="openshift-console-operator/console-operator-565b7b569d-stbb4" podUID="44befb08-b09e-471f-a0bc-59d503cd5c93"
Jul 16 20:33:27.129313 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:27.129292    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9jr72_6df4aad3-be34-4b4a-bd20-4b46e9d53737/dns-node-resolver/0.log"
Jul 16 20:33:27.930299 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:27.930270    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6vjn8_3b501cab-446b-4e3d-b9f8-3e2958d42293/node-ca/0.log"
Jul 16 20:33:28.569962 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:28.569921    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:28.570407 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:28.569985    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:28.570407 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:28.570090    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 20:33:28.570407 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:28.570118    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 20:33:28.570407 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:28.570172    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls podName:ff8aa4a9-dff1-4881-9148-c18582ee7864 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:32.570153583 +0000 UTC m=+58.536087795 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w8wzk" (UID: "ff8aa4a9-dff1-4881-9148-c18582ee7864") : secret "insights-runtime-extractor-tls" not found
Jul 16 20:33:28.570407 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:28.570195    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm podName:ff8aa4a9-dff1-4881-9148-c18582ee7864 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:32.570179904 +0000 UTC m=+58.536114118 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w8wzk" (UID: "ff8aa4a9-dff1-4881-9148-c18582ee7864") : configmap "kube-rbac-proxy" not found
Jul 16 20:33:28.929649 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:28.929624    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qd8ff_fc249a17-1357-464e-be27-57a3a9182c21/serve-healthcheck-canary/0.log"
Jul 16 20:33:29.532263 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:29.532230    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-nm6rf_a3b3a919-d977-4f1e-9a34-e7c7a1368d35/kube-storage-version-migrator-operator/0.log"
Jul 16 20:33:31.294604 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:31.294572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:31.294604 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:31.294606    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:31.295075 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:31.294629    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rz9m8\" (UID: \"95d2f323-8204-4405-8320-05650383659f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8"
Jul 16 20:33:31.295075 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:31.294672    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9v9kh\" (UID: \"85b78f5b-d6c3-4d9f-adcd-de59794fa938\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh"
Jul 16 20:33:31.295075 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:31.294703    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:31.295075 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:31.294738    2576 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 16 20:33:31.295075 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:31.294768    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle podName:cbb37170-322b-4128-8731-ad11b8117cdc nodeName:}" failed. No retries permitted until 2026-07-16 20:33:47.294748478 +0000 UTC m=+73.260682709 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle") pod "router-default-65f496d8cd-qchkt" (UID: "cbb37170-322b-4128-8731-ad11b8117cdc") : configmap references non-existent config key: service-ca.crt
Jul 16 20:33:31.295075 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:31.294748    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mq2hs\" (UID: \"41a9e89f-641d-4700-8c75-aefe2296b1f1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"
Jul 16 20:33:31.295075 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:31.294806    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs podName:cbb37170-322b-4128-8731-ad11b8117cdc nodeName:}" failed. No retries permitted until 2026-07-16 20:33:47.294788651 +0000 UTC m=+73.260722863 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs") pod "router-default-65f496d8cd-qchkt" (UID: "cbb37170-322b-4128-8731-ad11b8117cdc") : secret "router-metrics-certs-default" not found
Jul 16 20:33:31.295075 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:31.294808    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 16 20:33:31.295075 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:31.294830    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-648f5697fb-f9xb5: secret "image-registry-tls" not found
Jul 16 20:33:31.295075 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:31.294830    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 16 20:33:31.295075 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:31.294749    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 16 20:33:31.295075 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:31.294890    2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 16 20:33:31.295075 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:31.294893    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls podName:be2be990-3e76-486b-bf39-976ddac0225d nodeName:}" failed. No retries permitted until 2026-07-16 20:33:47.294877748 +0000 UTC m=+73.260811962 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls") pod "image-registry-648f5697fb-f9xb5" (UID: "be2be990-3e76-486b-bf39-976ddac0225d") : secret "image-registry-tls" not found
Jul 16 20:33:31.295075 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:31.294954    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls podName:85b78f5b-d6c3-4d9f-adcd-de59794fa938 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:47.294941874 +0000 UTC m=+73.260876091 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-9v9kh" (UID: "85b78f5b-d6c3-4d9f-adcd-de59794fa938") : secret "samples-operator-tls" not found
Jul 16 20:33:31.295547 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:31.294967    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert podName:95d2f323-8204-4405-8320-05650383659f nodeName:}" failed. No retries permitted until 2026-07-16 20:33:47.294961871 +0000 UTC m=+73.260896082 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-rz9m8" (UID: "95d2f323-8204-4405-8320-05650383659f") : secret "networking-console-plugin-cert" not found
Jul 16 20:33:31.295547 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:31.294981    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls podName:41a9e89f-641d-4700-8c75-aefe2296b1f1 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:47.294975481 +0000 UTC m=+73.260909692 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-mq2hs" (UID: "41a9e89f-641d-4700-8c75-aefe2296b1f1") : secret "cluster-monitoring-operator-tls" not found
Jul 16 20:33:31.295547 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:31.295006    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:31.295547 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:31.295068    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 16 20:33:31.295547 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:31.295099    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls podName:c48a73e2-e6f7-468e-b923-927c7ec97110 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:47.295082464 +0000 UTC m=+73.261026889 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls") pod "dns-default-89j4k" (UID: "c48a73e2-e6f7-468e-b923-927c7ec97110") : secret "dns-default-metrics-tls" not found
Jul 16 20:33:32.605006 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:32.604971    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:32.605541 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:32.605108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:32.605541 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:32.605120    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 16 20:33:32.605541 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:32.605194    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm podName:ff8aa4a9-dff1-4881-9148-c18582ee7864 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:40.605175007 +0000 UTC m=+66.571109219 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm") pod "insights-runtime-extractor-w8wzk" (UID: "ff8aa4a9-dff1-4881-9148-c18582ee7864") : configmap "kube-rbac-proxy" not found
Jul 16 20:33:32.605541 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:32.605197    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 20:33:32.605541 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:32.605242    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls podName:ff8aa4a9-dff1-4881-9148-c18582ee7864 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:40.605229862 +0000 UTC m=+66.571164073 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w8wzk" (UID: "ff8aa4a9-dff1-4881-9148-c18582ee7864") : secret "insights-runtime-extractor-tls" not found
Jul 16 20:33:39.360010 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.359924    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret\") pod \"global-pull-secret-syncer-q77jc\" (UID: \"6ef282cd-38f3-4e0d-b862-b9d510e70226\") " pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:33:39.360010 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.359984    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs\") pod \"network-metrics-daemon-pr8rp\" (UID: \"2d50105b-d05a-487b-bfa1-2b5167015795\") " pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:33:39.360403 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.360084    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcz7l\" (UniqueName: \"kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l\") pod \"network-check-target-xxh76\" (UID: \"30258292-80a3-4166-a0e5-1ad9bec7d240\") " pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:33:39.362470 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.362449    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcz7l\" (UniqueName: \"kubernetes.io/projected/30258292-80a3-4166-a0e5-1ad9bec7d240-kube-api-access-kcz7l\") pod \"network-check-target-xxh76\" (UID: \"30258292-80a3-4166-a0e5-1ad9bec7d240\") " pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:33:39.362618 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.362604    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 16 20:33:39.362719 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.362699    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 16 20:33:39.372651 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.372626    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d50105b-d05a-487b-bfa1-2b5167015795-metrics-certs\") pod \"network-metrics-daemon-pr8rp\" (UID: \"2d50105b-d05a-487b-bfa1-2b5167015795\") " pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:33:39.372651 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.372643    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6ef282cd-38f3-4e0d-b862-b9d510e70226-original-pull-secret\") pod \"global-pull-secret-syncer-q77jc\" (UID: \"6ef282cd-38f3-4e0d-b862-b9d510e70226\") " pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:33:39.505214 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.505190    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ql74g\""
Jul 16 20:33:39.513341 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.513323    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pr8rp"
Jul 16 20:33:39.515510 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.515494    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pjwwv\""
Jul 16 20:33:39.520420 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.520404    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-q77jc"
Jul 16 20:33:39.523553 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.523538    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:33:39.690246 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.690215    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pr8rp"]
Jul 16 20:33:39.693896 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:33:39.693869    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d50105b_d05a_487b_bfa1_2b5167015795.slice/crio-fc17178e3d16b52ab3fdef84ddfd34f1f30c53f7d735f5aa15f6dae28a53cc8b WatchSource:0}: Error finding container fc17178e3d16b52ab3fdef84ddfd34f1f30c53f7d735f5aa15f6dae28a53cc8b: Status 404 returned error can't find the container with id fc17178e3d16b52ab3fdef84ddfd34f1f30c53f7d735f5aa15f6dae28a53cc8b
Jul 16 20:33:39.704393 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.704352    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-q77jc"]
Jul 16 20:33:39.707256 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:33:39.707226    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ef282cd_38f3_4e0d_b862_b9d510e70226.slice/crio-55948208ac655dae4228be1fc58c524050f09db525b92767b2b0324b0cddafd3 WatchSource:0}: Error finding container 55948208ac655dae4228be1fc58c524050f09db525b92767b2b0324b0cddafd3: Status 404 returned error can't find the container with id 55948208ac655dae4228be1fc58c524050f09db525b92767b2b0324b0cddafd3
Jul 16 20:33:39.725027 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.725003    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xxh76"]
Jul 16 20:33:39.727595 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:33:39.727574    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30258292_80a3_4166_a0e5_1ad9bec7d240.slice/crio-4cafdc5c38052606adca141fee2f851da688d8dd701cb72e10f5544c5af5a212 WatchSource:0}: Error finding container 4cafdc5c38052606adca141fee2f851da688d8dd701cb72e10f5544c5af5a212: Status 404 returned error can't find the container with id 4cafdc5c38052606adca141fee2f851da688d8dd701cb72e10f5544c5af5a212
Jul 16 20:33:39.988080 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.988047    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xxh76" event={"ID":"30258292-80a3-4166-a0e5-1ad9bec7d240","Type":"ContainerStarted","Data":"9fe5232e259937b33f8f0dc78f34c0f9ab3c824580ff32baed1aa47c1d5d010b"}
Jul 16 20:33:39.988290 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.988094    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xxh76" event={"ID":"30258292-80a3-4166-a0e5-1ad9bec7d240","Type":"ContainerStarted","Data":"4cafdc5c38052606adca141fee2f851da688d8dd701cb72e10f5544c5af5a212"}
Jul 16 20:33:39.988290 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.988118    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:33:39.989174 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.989154    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q77jc" event={"ID":"6ef282cd-38f3-4e0d-b862-b9d510e70226","Type":"ContainerStarted","Data":"55948208ac655dae4228be1fc58c524050f09db525b92767b2b0324b0cddafd3"}
Jul 16 20:33:39.990259 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:39.990237    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pr8rp" event={"ID":"2d50105b-d05a-487b-bfa1-2b5167015795","Type":"ContainerStarted","Data":"fc17178e3d16b52ab3fdef84ddfd34f1f30c53f7d735f5aa15f6dae28a53cc8b"}
Jul 16 20:33:40.006945 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:40.006894    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xxh76" podStartSLOduration=66.006880662 podStartE2EDuration="1m6.006880662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:32:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:40.005901305 +0000 UTC m=+65.971835537" watchObservedRunningTime="2026-07-16 20:33:40.006880662 +0000 UTC m=+65.972814927"
Jul 16 20:33:40.670610 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:40.670577    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:40.670993 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:40.670620    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:40.670993 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:40.670716    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 16 20:33:40.670993 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:33:40.670778    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls podName:ff8aa4a9-dff1-4881-9148-c18582ee7864 nodeName:}" failed. No retries permitted until 2026-07-16 20:33:56.670763918 +0000 UTC m=+82.636698129 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls") pod "insights-runtime-extractor-w8wzk" (UID: "ff8aa4a9-dff1-4881-9148-c18582ee7864") : secret "insights-runtime-extractor-tls" not found
Jul 16 20:33:40.671117 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:40.671002    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ff8aa4a9-dff1-4881-9148-c18582ee7864-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:40.877028 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:40.877000    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9lz24"
Jul 16 20:33:41.683953 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:41.683921    2576 scope.go:117] "RemoveContainer" containerID="c7d55420bb4d44a4fb9431c326c5dd784222b3fc2947058b04b0986ab8b5a8d8"
Jul 16 20:33:42.003049 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:42.003016    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pr8rp" event={"ID":"2d50105b-d05a-487b-bfa1-2b5167015795","Type":"ContainerStarted","Data":"a1933ece050daee6011a6398a087799424f82d98554d35bfc9c52b3e5795d071"}
Jul 16 20:33:42.003182 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:42.003105    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pr8rp" event={"ID":"2d50105b-d05a-487b-bfa1-2b5167015795","Type":"ContainerStarted","Data":"9e330a50e9cc8c1ac9b36cb1d4709e5716e8a3619190ef856d01df530d0e0c4c"}
Jul 16 20:33:42.005290 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:42.005257    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 20:33:42.005435 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:42.005313    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-stbb4" event={"ID":"44befb08-b09e-471f-a0bc-59d503cd5c93","Type":"ContainerStarted","Data":"e11093a4cb45c10e4c0a05c410e5a3b188f9f968bf768b7cf989c08f330d761d"}
Jul 16 20:33:42.005656 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:42.005634    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:42.018448 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:42.018348    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pr8rp" podStartSLOduration=65.508875314 podStartE2EDuration="1m7.018333569s" totalImagesPullingTime="1.509458255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:42.01740809 +0000 UTC m=+67.983342326" watchObservedRunningTime="2026-07-16 20:33:42.018333569 +0000 UTC m=+67.984267820"
Jul 16 20:33:42.033760 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:42.033707    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-stbb4" podStartSLOduration=46.041179743 podStartE2EDuration="53.033689919s" totalImagesPullingTime="6.992510176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:32:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:42.031757786 +0000 UTC m=+67.997692048" watchObservedRunningTime="2026-07-16 20:33:42.033689919 +0000 UTC m=+67.999624154"
Jul 16 20:33:42.303079 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:42.303006    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-stbb4"
Jul 16 20:33:45.018145 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:45.018100    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-q77jc" event={"ID":"6ef282cd-38f3-4e0d-b862-b9d510e70226","Type":"ContainerStarted","Data":"113e0ee15cedac0333d068333ff963cc4950cb6b6515e4bfd2830a6a4dcf2492"}
Jul 16 20:33:45.035646 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:45.035592    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-q77jc" podStartSLOduration=65.346267395 podStartE2EDuration="1m10.035576267s" totalImagesPullingTime="4.689308872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:45.035060068 +0000 UTC m=+71.000994319" watchObservedRunningTime="2026-07-16 20:33:45.035576267 +0000 UTC m=+71.001510575"
Jul 16 20:33:46.051794 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:46.051764    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-wd874"]
Jul 16 20:33:46.053884 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:46.053864    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-wd874"
Jul 16 20:33:46.056241 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:46.056224    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 16 20:33:46.056490 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:46.056475    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-h7bt7\""
Jul 16 20:33:46.056579 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:46.056520    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 16 20:33:46.067316 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:46.067299    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-wd874"]
Jul 16 20:33:46.115708 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:46.115689    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npdbs\" (UniqueName: \"kubernetes.io/projected/8e0056a5-3a26-4a65-a72b-70eaba00fc54-kube-api-access-npdbs\") pod \"downloads-5bc67dcd46-wd874\" (UID: \"8e0056a5-3a26-4a65-a72b-70eaba00fc54\") " pod="openshift-console/downloads-5bc67dcd46-wd874"
Jul 16 20:33:46.216559 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:46.216537    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npdbs\" (UniqueName: \"kubernetes.io/projected/8e0056a5-3a26-4a65-a72b-70eaba00fc54-kube-api-access-npdbs\") pod \"downloads-5bc67dcd46-wd874\" (UID: \"8e0056a5-3a26-4a65-a72b-70eaba00fc54\") " pod="openshift-console/downloads-5bc67dcd46-wd874"
Jul 16 20:33:46.226567 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:46.226548    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npdbs\" (UniqueName: \"kubernetes.io/projected/8e0056a5-3a26-4a65-a72b-70eaba00fc54-kube-api-access-npdbs\") pod \"downloads-5bc67dcd46-wd874\" (UID: \"8e0056a5-3a26-4a65-a72b-70eaba00fc54\") " pod="openshift-console/downloads-5bc67dcd46-wd874"
Jul 16 20:33:46.363910 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:46.363858    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-wd874"
Jul 16 20:33:46.484481 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:46.484454    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-wd874"]
Jul 16 20:33:46.487188 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:33:46.487165    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e0056a5_3a26_4a65_a72b_70eaba00fc54.slice/crio-15de384759b7a0b83c221fa08b91986137684b46c6e020e2aac7302f806943f6 WatchSource:0}: Error finding container 15de384759b7a0b83c221fa08b91986137684b46c6e020e2aac7302f806943f6: Status 404 returned error can't find the container with id 15de384759b7a0b83c221fa08b91986137684b46c6e020e2aac7302f806943f6
Jul 16 20:33:47.026188 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.026143    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-wd874" event={"ID":"8e0056a5-3a26-4a65-a72b-70eaba00fc54","Type":"ContainerStarted","Data":"15de384759b7a0b83c221fa08b91986137684b46c6e020e2aac7302f806943f6"}
Jul 16 20:33:47.325839 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.325766    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rz9m8\" (UID: \"95d2f323-8204-4405-8320-05650383659f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8"
Jul 16 20:33:47.325839 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.325814    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9v9kh\" (UID: \"85b78f5b-d6c3-4d9f-adcd-de59794fa938\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh"
Jul 16 20:33:47.326315 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.326211    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:47.326315 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.326263    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mq2hs\" (UID: \"41a9e89f-641d-4700-8c75-aefe2296b1f1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"
Jul 16 20:33:47.326450 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.326336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:47.326450 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.326403    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:47.326450 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.326428    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:47.327173 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.327124    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbb37170-322b-4128-8731-ad11b8117cdc-service-ca-bundle\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:47.329180 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.329137    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/95d2f323-8204-4405-8320-05650383659f-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rz9m8\" (UID: \"95d2f323-8204-4405-8320-05650383659f\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8"
Jul 16 20:33:47.329279 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.329191    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/85b78f5b-d6c3-4d9f-adcd-de59794fa938-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-9v9kh\" (UID: \"85b78f5b-d6c3-4d9f-adcd-de59794fa938\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh"
Jul 16 20:33:47.329279 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.329207    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls\") pod \"image-registry-648f5697fb-f9xb5\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") " pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:47.329500 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.329474    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbb37170-322b-4128-8731-ad11b8117cdc-metrics-certs\") pod \"router-default-65f496d8cd-qchkt\" (UID: \"cbb37170-322b-4128-8731-ad11b8117cdc\") " pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:47.329620 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.329496    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/41a9e89f-641d-4700-8c75-aefe2296b1f1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-mq2hs\" (UID: \"41a9e89f-641d-4700-8c75-aefe2296b1f1\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"
Jul 16 20:33:47.330394 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.330353    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c48a73e2-e6f7-468e-b923-927c7ec97110-metrics-tls\") pod \"dns-default-89j4k\" (UID: \"c48a73e2-e6f7-468e-b923-927c7ec97110\") " pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:47.516910 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.516882    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rtpxj\""
Jul 16 20:33:47.525696 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.525661    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh"
Jul 16 20:33:47.527959 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.527935    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-68jbx\""
Jul 16 20:33:47.536571 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.536552    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:47.561121 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.561099    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9cm6b\""
Jul 16 20:33:47.569547 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.569525    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8"
Jul 16 20:33:47.571995 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.571959    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-pzfdd\""
Jul 16 20:33:47.578387 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.578314    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-nqzhf\""
Jul 16 20:33:47.579879 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.579853    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"
Jul 16 20:33:47.587826 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.587767    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:47.596759 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.596521    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xlvk5\""
Jul 16 20:33:47.604651 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.604383    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:47.710168 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.710107    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh"]
Jul 16 20:33:47.791424 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.791167    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-648f5697fb-f9xb5"]
Jul 16 20:33:47.800962 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:33:47.800922    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe2be990_3e76_486b_bf39_976ddac0225d.slice/crio-67631133d42f597e03b7e34273e11c44f03e8ec2478fad688ab1ace66471351c WatchSource:0}: Error finding container 67631133d42f597e03b7e34273e11c44f03e8ec2478fad688ab1ace66471351c: Status 404 returned error can't find the container with id 67631133d42f597e03b7e34273e11c44f03e8ec2478fad688ab1ace66471351c
Jul 16 20:33:47.821259 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.821192    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8"]
Jul 16 20:33:47.827828 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.827771    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs"]
Jul 16 20:33:47.831842 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:33:47.831343    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41a9e89f_641d_4700_8c75_aefe2296b1f1.slice/crio-fc9f7c2e49c3579beebc9e3b4810a73031ad5521fc94e05d13e1909291aa489d WatchSource:0}: Error finding container fc9f7c2e49c3579beebc9e3b4810a73031ad5521fc94e05d13e1909291aa489d: Status 404 returned error can't find the container with id fc9f7c2e49c3579beebc9e3b4810a73031ad5521fc94e05d13e1909291aa489d
Jul 16 20:33:47.834283 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:33:47.834239    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95d2f323_8204_4405_8320_05650383659f.slice/crio-1824b5ebb7d8a60be440a502a54bf92b5d1525094e6c75c8be12c39802d69697 WatchSource:0}: Error finding container 1824b5ebb7d8a60be440a502a54bf92b5d1525094e6c75c8be12c39802d69697: Status 404 returned error can't find the container with id 1824b5ebb7d8a60be440a502a54bf92b5d1525094e6c75c8be12c39802d69697
Jul 16 20:33:47.851642 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.851617    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-65f496d8cd-qchkt"]
Jul 16 20:33:47.856354 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:33:47.856329    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbb37170_322b_4128_8731_ad11b8117cdc.slice/crio-8bea953962fe2f405a3f1b01a8dfe21a599edb94780432fb7ba68a5d8b6843cd WatchSource:0}: Error finding container 8bea953962fe2f405a3f1b01a8dfe21a599edb94780432fb7ba68a5d8b6843cd: Status 404 returned error can't find the container with id 8bea953962fe2f405a3f1b01a8dfe21a599edb94780432fb7ba68a5d8b6843cd
Jul 16 20:33:47.870420 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:47.870324    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-89j4k"]
Jul 16 20:33:47.872815 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:33:47.872792    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc48a73e2_e6f7_468e_b923_927c7ec97110.slice/crio-0977ec6141d98e873f4ee9a9dfefdc822a18057ee30788459d2addd8e0330afb WatchSource:0}: Error finding container 0977ec6141d98e873f4ee9a9dfefdc822a18057ee30788459d2addd8e0330afb: Status 404 returned error can't find the container with id 0977ec6141d98e873f4ee9a9dfefdc822a18057ee30788459d2addd8e0330afb
Jul 16 20:33:48.032623 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:48.032536    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8" event={"ID":"95d2f323-8204-4405-8320-05650383659f","Type":"ContainerStarted","Data":"1824b5ebb7d8a60be440a502a54bf92b5d1525094e6c75c8be12c39802d69697"}
Jul 16 20:33:48.034274 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:48.034229    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-65f496d8cd-qchkt" event={"ID":"cbb37170-322b-4128-8731-ad11b8117cdc","Type":"ContainerStarted","Data":"5f9b982398a5dff4950aa462a74668c022b0625a35ffd919f16c8edd682b86b4"}
Jul 16 20:33:48.034274 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:48.034272    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-65f496d8cd-qchkt" event={"ID":"cbb37170-322b-4128-8731-ad11b8117cdc","Type":"ContainerStarted","Data":"8bea953962fe2f405a3f1b01a8dfe21a599edb94780432fb7ba68a5d8b6843cd"}
Jul 16 20:33:48.035989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:48.035962    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs" event={"ID":"41a9e89f-641d-4700-8c75-aefe2296b1f1","Type":"ContainerStarted","Data":"fc9f7c2e49c3579beebc9e3b4810a73031ad5521fc94e05d13e1909291aa489d"}
Jul 16 20:33:48.037702 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:48.037674    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-648f5697fb-f9xb5" event={"ID":"be2be990-3e76-486b-bf39-976ddac0225d","Type":"ContainerStarted","Data":"ea5f5913766221d12dd29b932de93408db4eca31435fdb98df998c2b38c77fa0"}
Jul 16 20:33:48.037812 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:48.037717    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-648f5697fb-f9xb5" event={"ID":"be2be990-3e76-486b-bf39-976ddac0225d","Type":"ContainerStarted","Data":"67631133d42f597e03b7e34273e11c44f03e8ec2478fad688ab1ace66471351c"}
Jul 16 20:33:48.037871 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:48.037806    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:33:48.039548 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:48.039519    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh" event={"ID":"85b78f5b-d6c3-4d9f-adcd-de59794fa938","Type":"ContainerStarted","Data":"f0c7c81cb25377302fbc29a347b645cadbc85689fdde72cdba9c6aa0c6d20402"}
Jul 16 20:33:48.040703 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:48.040680    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-89j4k" event={"ID":"c48a73e2-e6f7-468e-b923-927c7ec97110","Type":"ContainerStarted","Data":"0977ec6141d98e873f4ee9a9dfefdc822a18057ee30788459d2addd8e0330afb"}
Jul 16 20:33:48.052807 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:48.052758    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-65f496d8cd-qchkt" podStartSLOduration=59.052745413 podStartE2EDuration="59.052745413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:32:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:48.051337743 +0000 UTC m=+74.017271997" watchObservedRunningTime="2026-07-16 20:33:48.052745413 +0000 UTC m=+74.018679623"
Jul 16 20:33:48.071541 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:48.071489    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-648f5697fb-f9xb5" podStartSLOduration=73.071472893 podStartE2EDuration="1m13.071472893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:32:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:48.070346629 +0000 UTC m=+74.036280855" watchObservedRunningTime="2026-07-16 20:33:48.071472893 +0000 UTC m=+74.037407127"
Jul 16 20:33:48.588371 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:48.588337    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:48.590886 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:48.590860    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:49.046468 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:49.046436    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:49.047764 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:49.047737    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-65f496d8cd-qchkt"
Jul 16 20:33:50.052821 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:50.052736    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh" event={"ID":"85b78f5b-d6c3-4d9f-adcd-de59794fa938","Type":"ContainerStarted","Data":"5edcc1a217a08d81d694c4d144efb21748c1cb53250486246f859725395fed6d"}
Jul 16 20:33:50.052821 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:50.052804    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh" event={"ID":"85b78f5b-d6c3-4d9f-adcd-de59794fa938","Type":"ContainerStarted","Data":"d167dc88e3fa192d04c8ec9694a51e7a6112b4ceb74afa7287b93f23a4e52e4f"}
Jul 16 20:33:50.072426 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:50.072349    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-9v9kh" podStartSLOduration=59.172170261 podStartE2EDuration="1m1.072329465s" totalImagesPullingTime="1.900159204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:32:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:50.071343788 +0000 UTC m=+76.037278021" watchObservedRunningTime="2026-07-16 20:33:50.072329465 +0000 UTC m=+76.038263699"
Jul 16 20:33:53.075516 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:53.075430    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs" event={"ID":"41a9e89f-641d-4700-8c75-aefe2296b1f1","Type":"ContainerStarted","Data":"2f80d9a4695434aa5a921308ca0e2fd7df182fdf5680aed1006db8af6deff852"}
Jul 16 20:33:53.078108 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:53.078080    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-89j4k" event={"ID":"c48a73e2-e6f7-468e-b923-927c7ec97110","Type":"ContainerStarted","Data":"b784ed3b94d281ff1943ac14eef2c9394eeb40042a27ba8fca1395894cb4f42e"}
Jul 16 20:33:53.078251 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:53.078133    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-89j4k" event={"ID":"c48a73e2-e6f7-468e-b923-927c7ec97110","Type":"ContainerStarted","Data":"e408f8893d95f353f8b6d06a543302df96318fa10341b63d7eeb508e3fafb714"}
Jul 16 20:33:53.078251 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:53.078166    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-89j4k"
Jul 16 20:33:53.082058 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:53.081527    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8" event={"ID":"95d2f323-8204-4405-8320-05650383659f","Type":"ContainerStarted","Data":"eff97301c195a4ff7645a73ee0f413ed01d30b08fbab1e4736c3f7547c6a2e73"}
Jul 16 20:33:53.098973 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:53.098923    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-mq2hs" podStartSLOduration=59.161378606 podStartE2EDuration="1m4.098907239s" totalImagesPullingTime="4.937528633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:32:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:53.097752504 +0000 UTC m=+79.063686739" watchObservedRunningTime="2026-07-16 20:33:53.098907239 +0000 UTC m=+79.064841472"
Jul 16 20:33:53.116447 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:53.116394    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-89j4k" podStartSLOduration=33.221621798 podStartE2EDuration="38.116355958s" totalImagesPullingTime="4.89473416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:53.115179308 +0000 UTC m=+79.081113542" watchObservedRunningTime="2026-07-16 20:33:53.116355958 +0000 UTC m=+79.082290191"
Jul 16 20:33:53.136052 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:53.136009    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rz9m8" podStartSLOduration=59.203873498 podStartE2EDuration="1m4.135991684s" totalImagesPullingTime="4.932118186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:32:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:33:53.134106271 +0000 UTC m=+79.100040505" watchObservedRunningTime="2026-07-16 20:33:53.135991684 +0000 UTC m=+79.101925919"
Jul 16 20:33:56.708334 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:56.708298    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:56.711069 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:56.711041    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ff8aa4a9-dff1-4881-9148-c18582ee7864-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-w8wzk\" (UID: \"ff8aa4a9-dff1-4881-9148-c18582ee7864\") " pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:56.912818 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:56.912787    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-khzsm\""
Jul 16 20:33:56.921099 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:56.921073    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-w8wzk"
Jul 16 20:33:57.100402 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:57.100345    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-w8wzk"]
Jul 16 20:33:57.104005 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:33:57.103976    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff8aa4a9_dff1_4881_9148_c18582ee7864.slice/crio-84581c993884469b76f0820194617ddfc20d181331699e26bb67ef5743213c7f WatchSource:0}: Error finding container 84581c993884469b76f0820194617ddfc20d181331699e26bb67ef5743213c7f: Status 404 returned error can't find the container with id 84581c993884469b76f0820194617ddfc20d181331699e26bb67ef5743213c7f
Jul 16 20:33:58.105173 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:33:58.105068    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w8wzk" event={"ID":"ff8aa4a9-dff1-4881-9148-c18582ee7864","Type":"ContainerStarted","Data":"84581c993884469b76f0820194617ddfc20d181331699e26bb67ef5743213c7f"}
Jul 16 20:34:03.088783 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:03.088756    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-89j4k"
Jul 16 20:34:03.832255 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:03.832224    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-t24lg"]
Jul 16 20:34:03.835424 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:03.835397    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:03.837866 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:03.837796    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 16 20:34:03.837866 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:03.837838    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 16 20:34:03.838030 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:03.837804    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 16 20:34:03.838701 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:03.838647    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 16 20:34:03.838701 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:03.838661    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zmp64\""
Jul 16 20:34:03.967872 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:03.967837    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c62609d-708e-4e2a-b535-35a48ff7fddf-metrics-client-ca\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:03.968020 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:03.967889    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7c62609d-708e-4e2a-b535-35a48ff7fddf-node-exporter-textfile\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:03.968020 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:03.967928    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7b4d\" (UniqueName: \"kubernetes.io/projected/7c62609d-708e-4e2a-b535-35a48ff7fddf-kube-api-access-w7b4d\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:03.968020 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:03.967958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7c62609d-708e-4e2a-b535-35a48ff7fddf-root\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:03.968020 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:03.967985    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7c62609d-708e-4e2a-b535-35a48ff7fddf-node-exporter-wtmp\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:03.968211 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:03.968038    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c62609d-708e-4e2a-b535-35a48ff7fddf-sys\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:03.968211 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:03.968066    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7c62609d-708e-4e2a-b535-35a48ff7fddf-node-exporter-tls\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:03.968211 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:03.968097    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c62609d-708e-4e2a-b535-35a48ff7fddf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:03.968211 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:03.968129    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7c62609d-708e-4e2a-b535-35a48ff7fddf-node-exporter-accelerators-collector-config\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.068588 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.068555    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7c62609d-708e-4e2a-b535-35a48ff7fddf-node-exporter-textfile\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.068745 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.068618    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7b4d\" (UniqueName: \"kubernetes.io/projected/7c62609d-708e-4e2a-b535-35a48ff7fddf-kube-api-access-w7b4d\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.068745 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.068661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7c62609d-708e-4e2a-b535-35a48ff7fddf-root\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.068745 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.068689    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7c62609d-708e-4e2a-b535-35a48ff7fddf-node-exporter-wtmp\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.068745 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.068725    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c62609d-708e-4e2a-b535-35a48ff7fddf-sys\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.068957 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.068750    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7c62609d-708e-4e2a-b535-35a48ff7fddf-node-exporter-tls\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.068957 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.068773    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7c62609d-708e-4e2a-b535-35a48ff7fddf-root\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.068957 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.068816    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c62609d-708e-4e2a-b535-35a48ff7fddf-sys\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.068957 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.068859    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7c62609d-708e-4e2a-b535-35a48ff7fddf-node-exporter-wtmp\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.068957 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.068859    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c62609d-708e-4e2a-b535-35a48ff7fddf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.068957 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.068944    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7c62609d-708e-4e2a-b535-35a48ff7fddf-node-exporter-textfile\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.069239 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.068988    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7c62609d-708e-4e2a-b535-35a48ff7fddf-node-exporter-accelerators-collector-config\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.069239 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.069060    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c62609d-708e-4e2a-b535-35a48ff7fddf-metrics-client-ca\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.069670 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.069624    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7c62609d-708e-4e2a-b535-35a48ff7fddf-node-exporter-accelerators-collector-config\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.069786 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.069680    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7c62609d-708e-4e2a-b535-35a48ff7fddf-metrics-client-ca\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.071464 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.071433    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7c62609d-708e-4e2a-b535-35a48ff7fddf-node-exporter-tls\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.071740 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.071718    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7c62609d-708e-4e2a-b535-35a48ff7fddf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.081828 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.081805    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7b4d\" (UniqueName: \"kubernetes.io/projected/7c62609d-708e-4e2a-b535-35a48ff7fddf-kube-api-access-w7b4d\") pod \"node-exporter-t24lg\" (UID: \"7c62609d-708e-4e2a-b535-35a48ff7fddf\") " pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:04.147846 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:04.147823    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t24lg"
Jul 16 20:34:07.904768 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:07.904723    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-648f5697fb-f9xb5"]
Jul 16 20:34:07.910120 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:07.910096    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:34:07.974083 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:34:07.974055    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c62609d_708e_4e2a_b535_35a48ff7fddf.slice/crio-3db6c23167ad72559f09cd3e4cd6fc8f6a7e6e5400b7ebc3d9c33453d979aa3e WatchSource:0}: Error finding container 3db6c23167ad72559f09cd3e4cd6fc8f6a7e6e5400b7ebc3d9c33453d979aa3e: Status 404 returned error can't find the container with id 3db6c23167ad72559f09cd3e4cd6fc8f6a7e6e5400b7ebc3d9c33453d979aa3e
Jul 16 20:34:08.151731 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:08.151666    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w8wzk" event={"ID":"ff8aa4a9-dff1-4881-9148-c18582ee7864","Type":"ContainerStarted","Data":"1fef307846996aff3289747428d365a5ce3bbd6d6f06b8afda92efe01c987571"}
Jul 16 20:34:08.153125 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:08.153087    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t24lg" event={"ID":"7c62609d-708e-4e2a-b535-35a48ff7fddf","Type":"ContainerStarted","Data":"3db6c23167ad72559f09cd3e4cd6fc8f6a7e6e5400b7ebc3d9c33453d979aa3e"}
Jul 16 20:34:08.154840 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:08.154788    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-wd874" event={"ID":"8e0056a5-3a26-4a65-a72b-70eaba00fc54","Type":"ContainerStarted","Data":"e0ce3a5c818ffb489654e5eb11c9348fb4d0d36be9b69c38a7e0d059d851080c"}
Jul 16 20:34:08.155072 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:08.155051    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-wd874"
Jul 16 20:34:08.156574 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:08.156539    2576 patch_prober.go:28] interesting pod/downloads-5bc67dcd46-wd874 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.20:8080/\": dial tcp 10.132.0.20:8080: connect: connection refused" start-of-body=
Jul 16 20:34:08.156660 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:08.156603    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-5bc67dcd46-wd874" podUID="8e0056a5-3a26-4a65-a72b-70eaba00fc54" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.20:8080/\": dial tcp 10.132.0.20:8080: connect: connection refused"
Jul 16 20:34:09.174403 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:09.174021    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-wd874"
Jul 16 20:34:09.194611 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:09.194557    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-wd874" podStartSLOduration=1.712650879 podStartE2EDuration="23.194533827s" totalImagesPullingTime="21.481882948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:33:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:08.185423712 +0000 UTC m=+94.151357945" watchObservedRunningTime="2026-07-16 20:34:09.194533827 +0000 UTC m=+95.160468064"
Jul 16 20:34:10.152130 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.152109    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 20:34:10.180201 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.178737    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 20:34:10.180201 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.178916    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.181607 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.181587    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 16 20:34:10.182110 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.182088    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 16 20:34:10.182264 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.181923    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 16 20:34:10.182264 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.181962    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 16 20:34:10.182403 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.181998    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 16 20:34:10.182464 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.182004    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 16 20:34:10.182518 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.181840    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jzq74\""
Jul 16 20:34:10.182877 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.182855    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 16 20:34:10.183005 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.182910    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 16 20:34:10.183513 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.183101    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 16 20:34:10.183513 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.183304    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 16 20:34:10.183696 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.183523    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 16 20:34:10.183896 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.183875    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 16 20:34:10.184098 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.184078    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 16 20:34:10.184279 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.184261    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 16 20:34:10.184816 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.184629    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6ktouqipmkpmh\""
Jul 16 20:34:10.194926 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.194902    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 16 20:34:10.326579 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325389    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.326579 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325443    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.326579 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325467    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.326579 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325496    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.326579 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325525    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.326579 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325559    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.326579 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kglwt\" (UniqueName: \"kubernetes.io/projected/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-kube-api-access-kglwt\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.326579 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325646    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.326579 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325675    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.326579 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325717    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-web-config\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.326579 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325753    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.326579 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325779    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.326579 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325805    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-config\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.327221 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325829    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-config-out\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.327221 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325865    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.327221 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325898    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.327221 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325924    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.327221 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325949    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.327221 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.325977    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.327221 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.326004    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.426943 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.426908    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427075 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.426960    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427075 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.426989    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427075 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.427022    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427075 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.427050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427254 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.427082    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427254 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.427107    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kglwt\" (UniqueName: \"kubernetes.io/projected/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-kube-api-access-kglwt\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427254 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.427143    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427254 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.427171    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427254 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.427208    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-web-config\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427254 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.427240    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427516 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.427264    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427516 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.427292    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-config\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427516 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.427315    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-config-out\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427516 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.427352    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427516 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.427414    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427516 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.427449    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427516 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.427475    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427516 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.427505    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.427854 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.427536    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.428966 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.428613    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.429974 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.429629    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.429974 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.429708    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.431714 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.430429    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.431714 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.430906    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.431714 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.431209    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.432815 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.432447    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.436014 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.433339    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.436014 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.434660    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.437173 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.437150    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-web-config\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.438455 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.438412    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-config-out\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.438723 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.438678    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.439280 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.439242    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.439389 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.439293    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-config\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.439938 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.439898    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.441869 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.441836    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kglwt\" (UniqueName: \"kubernetes.io/projected/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-kube-api-access-kglwt\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.442857 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.442214    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.442857 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.442547    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.442857 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.442824    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.444863 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.444820    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:10.500735 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:10.500629    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:11.001103 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:11.001063    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xxh76"
Jul 16 20:34:11.171941 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:11.171855    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t24lg" event={"ID":"7c62609d-708e-4e2a-b535-35a48ff7fddf","Type":"ContainerStarted","Data":"cc4644666fc7c82ed5e6cf2ba90b4a72fb74192e66392503c9bfa181f5b6db8b"}
Jul 16 20:34:12.183688 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:12.183649    2576 generic.go:356] "Generic (PLEG): container finished" podID="7c62609d-708e-4e2a-b535-35a48ff7fddf" containerID="cc4644666fc7c82ed5e6cf2ba90b4a72fb74192e66392503c9bfa181f5b6db8b" exitCode=0
Jul 16 20:34:12.184230 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:12.183773    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t24lg" event={"ID":"7c62609d-708e-4e2a-b535-35a48ff7fddf","Type":"ContainerDied","Data":"cc4644666fc7c82ed5e6cf2ba90b4a72fb74192e66392503c9bfa181f5b6db8b"}
Jul 16 20:34:13.106409 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:13.106379    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 20:34:13.108332 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:34:13.108300    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa483f9b_53c4_4c4d_a0b3_17ab25e61685.slice/crio-e9d77b26ecb361143306f62e1aee17184a038442c5391d8d5eb594429b21ad62 WatchSource:0}: Error finding container e9d77b26ecb361143306f62e1aee17184a038442c5391d8d5eb594429b21ad62: Status 404 returned error can't find the container with id e9d77b26ecb361143306f62e1aee17184a038442c5391d8d5eb594429b21ad62
Jul 16 20:34:13.189593 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:13.189561    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa483f9b-53c4-4c4d-a0b3-17ab25e61685","Type":"ContainerStarted","Data":"e9d77b26ecb361143306f62e1aee17184a038442c5391d8d5eb594429b21ad62"}
Jul 16 20:34:13.191851 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:13.191823    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t24lg" event={"ID":"7c62609d-708e-4e2a-b535-35a48ff7fddf","Type":"ContainerStarted","Data":"16c7a8634756ce65fdc30fcad6c113b3f82e1d208e56b8a686e1d79a1ffa9ccb"}
Jul 16 20:34:14.199833 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:14.199772    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w8wzk" event={"ID":"ff8aa4a9-dff1-4881-9148-c18582ee7864","Type":"ContainerStarted","Data":"ddde02eef75ae704ba54361f31007dd695866c2555f556ecfb583bc02c671afd"}
Jul 16 20:34:14.202723 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:14.202696    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t24lg" event={"ID":"7c62609d-708e-4e2a-b535-35a48ff7fddf","Type":"ContainerStarted","Data":"589c287955a759376b86b28eb769d769f96771d10c565dc1b7d0cb85e8383499"}
Jul 16 20:34:14.223151 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:14.223095    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-t24lg" podStartSLOduration=8.99958367 podStartE2EDuration="11.223077258s" totalImagesPullingTime="2.123818587s" totalInitContainerRuntime="99.675001ms" isStatefulPod=true podCreationTimestamp="2026-07-16 20:34:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:14.222171622 +0000 UTC m=+100.188105857" watchObservedRunningTime="2026-07-16 20:34:14.223077258 +0000 UTC m=+100.189011493"
Jul 16 20:34:15.209251 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:15.209220    2576 generic.go:356] "Generic (PLEG): container finished" podID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerID="130a60efac8663191d17a398bb45aad694ca0d185a8c6f50e52e2ab97214e48b" exitCode=0
Jul 16 20:34:15.209737 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:15.209321    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa483f9b-53c4-4c4d-a0b3-17ab25e61685","Type":"ContainerDied","Data":"130a60efac8663191d17a398bb45aad694ca0d185a8c6f50e52e2ab97214e48b"}
Jul 16 20:34:21.239342 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:21.239313    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-w8wzk" event={"ID":"ff8aa4a9-dff1-4881-9148-c18582ee7864","Type":"ContainerStarted","Data":"d0deca972a6b72e2e78f415b160a3c01aa2c77c9ec85fd7e0ef8a18a260b019c"}
Jul 16 20:34:21.241371 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:21.241334    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa483f9b-53c4-4c4d-a0b3-17ab25e61685","Type":"ContainerStarted","Data":"ec1aebfc5adf8cb292baf350d998aa2a76ade4968da8df85eef8bbc2aa2c8723"}
Jul 16 20:34:21.258876 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:21.258835    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-w8wzk" podStartSLOduration=33.488071308 podStartE2EDuration="57.258819659s" totalImagesPullingTime="23.770748351s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 20:33:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:21.258034221 +0000 UTC m=+107.223968454" watchObservedRunningTime="2026-07-16 20:34:21.258819659 +0000 UTC m=+107.224753892"
Jul 16 20:34:22.247859 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:22.247800    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa483f9b-53c4-4c4d-a0b3-17ab25e61685","Type":"ContainerStarted","Data":"34530f821e92d1999fde5e9207c3e18ba801370400bc33d996fa73f1b9edb856"}
Jul 16 20:34:24.258765 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:24.258741    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa483f9b-53c4-4c4d-a0b3-17ab25e61685","Type":"ContainerStarted","Data":"36a8b44700711e0d3cb9cccc123286033826d321ebeb17ef4f88b7c88d3524ab"}
Jul 16 20:34:24.259100 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:24.258803    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa483f9b-53c4-4c4d-a0b3-17ab25e61685","Type":"ContainerStarted","Data":"69ac7057a00c65188904540681811bfed94040361ae1e4737d92cfaa6bb6bddd"}
Jul 16 20:34:24.259100 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:24.258846    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa483f9b-53c4-4c4d-a0b3-17ab25e61685","Type":"ContainerStarted","Data":"0cfa33703a629ba59e0252d88aaec72519e8746b85617e27653ee116492cd904"}
Jul 16 20:34:24.259100 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:24.258878    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa483f9b-53c4-4c4d-a0b3-17ab25e61685","Type":"ContainerStarted","Data":"8616557b03554bd06c16825c911df0ca0a1ba8782193220d9619f664593d2e87"}
Jul 16 20:34:24.291272 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:24.291216    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.292472326 podStartE2EDuration="14.29119505s" totalImagesPullingTime="9.978269198s" totalInitContainerRuntime="20.453526ms" isStatefulPod=false podCreationTimestamp="2026-07-16 20:34:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:34:24.289166063 +0000 UTC m=+110.255100334" watchObservedRunningTime="2026-07-16 20:34:24.29119505 +0000 UTC m=+110.257129292"
Jul 16 20:34:25.501143 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:25.501110    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:34:32.929836 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:32.929770    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-648f5697fb-f9xb5" podUID="be2be990-3e76-486b-bf39-976ddac0225d" containerName="registry" containerID="cri-o://ea5f5913766221d12dd29b932de93408db4eca31435fdb98df998c2b38c77fa0" gracePeriod=30
Jul 16 20:34:33.195025 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.194978    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:34:33.217197 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.217174    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4kjf\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-kube-api-access-c4kjf\") pod \"be2be990-3e76-486b-bf39-976ddac0225d\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") "
Jul 16 20:34:33.217314 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.217228    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/be2be990-3e76-486b-bf39-976ddac0225d-ca-trust-extracted\") pod \"be2be990-3e76-486b-bf39-976ddac0225d\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") "
Jul 16 20:34:33.217314 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.217248    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/be2be990-3e76-486b-bf39-976ddac0225d-registry-certificates\") pod \"be2be990-3e76-486b-bf39-976ddac0225d\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") "
Jul 16 20:34:33.217314 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.217286    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/be2be990-3e76-486b-bf39-976ddac0225d-image-registry-private-configuration\") pod \"be2be990-3e76-486b-bf39-976ddac0225d\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") "
Jul 16 20:34:33.217522 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.217314    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-bound-sa-token\") pod \"be2be990-3e76-486b-bf39-976ddac0225d\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") "
Jul 16 20:34:33.217522 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.217342    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls\") pod \"be2be990-3e76-486b-bf39-976ddac0225d\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") "
Jul 16 20:34:33.217522 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.217386    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/be2be990-3e76-486b-bf39-976ddac0225d-installation-pull-secrets\") pod \"be2be990-3e76-486b-bf39-976ddac0225d\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") "
Jul 16 20:34:33.217522 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.217448    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be2be990-3e76-486b-bf39-976ddac0225d-trusted-ca\") pod \"be2be990-3e76-486b-bf39-976ddac0225d\" (UID: \"be2be990-3e76-486b-bf39-976ddac0225d\") "
Jul 16 20:34:33.217999 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.217773    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be2be990-3e76-486b-bf39-976ddac0225d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "be2be990-3e76-486b-bf39-976ddac0225d" (UID: "be2be990-3e76-486b-bf39-976ddac0225d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:34:33.218114 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.218072    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be2be990-3e76-486b-bf39-976ddac0225d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "be2be990-3e76-486b-bf39-976ddac0225d" (UID: "be2be990-3e76-486b-bf39-976ddac0225d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:34:33.220671 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.220559    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-kube-api-access-c4kjf" (OuterVolumeSpecName: "kube-api-access-c4kjf") pod "be2be990-3e76-486b-bf39-976ddac0225d" (UID: "be2be990-3e76-486b-bf39-976ddac0225d"). InnerVolumeSpecName "kube-api-access-c4kjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:34:33.220671 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.220638    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be2be990-3e76-486b-bf39-976ddac0225d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "be2be990-3e76-486b-bf39-976ddac0225d" (UID: "be2be990-3e76-486b-bf39-976ddac0225d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:34:33.220849 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.220801    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "be2be990-3e76-486b-bf39-976ddac0225d" (UID: "be2be990-3e76-486b-bf39-976ddac0225d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:34:33.220950 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.220928    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be2be990-3e76-486b-bf39-976ddac0225d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "be2be990-3e76-486b-bf39-976ddac0225d" (UID: "be2be990-3e76-486b-bf39-976ddac0225d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:34:33.221003 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.220936    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "be2be990-3e76-486b-bf39-976ddac0225d" (UID: "be2be990-3e76-486b-bf39-976ddac0225d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:34:33.229062 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.229033    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be2be990-3e76-486b-bf39-976ddac0225d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "be2be990-3e76-486b-bf39-976ddac0225d" (UID: "be2be990-3e76-486b-bf39-976ddac0225d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:34:33.294812 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.294789    2576 generic.go:356] "Generic (PLEG): container finished" podID="be2be990-3e76-486b-bf39-976ddac0225d" containerID="ea5f5913766221d12dd29b932de93408db4eca31435fdb98df998c2b38c77fa0" exitCode=0
Jul 16 20:34:33.294899 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.294834    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-648f5697fb-f9xb5"
Jul 16 20:34:33.294899 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.294873    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-648f5697fb-f9xb5" event={"ID":"be2be990-3e76-486b-bf39-976ddac0225d","Type":"ContainerDied","Data":"ea5f5913766221d12dd29b932de93408db4eca31435fdb98df998c2b38c77fa0"}
Jul 16 20:34:33.294966 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.294959    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-648f5697fb-f9xb5" event={"ID":"be2be990-3e76-486b-bf39-976ddac0225d","Type":"ContainerDied","Data":"67631133d42f597e03b7e34273e11c44f03e8ec2478fad688ab1ace66471351c"}
Jul 16 20:34:33.294998 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.294984    2576 scope.go:117] "RemoveContainer" containerID="ea5f5913766221d12dd29b932de93408db4eca31435fdb98df998c2b38c77fa0"
Jul 16 20:34:33.302758 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.302737    2576 scope.go:117] "RemoveContainer" containerID="ea5f5913766221d12dd29b932de93408db4eca31435fdb98df998c2b38c77fa0"
Jul 16 20:34:33.303033 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:34:33.303008    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea5f5913766221d12dd29b932de93408db4eca31435fdb98df998c2b38c77fa0\": container with ID starting with ea5f5913766221d12dd29b932de93408db4eca31435fdb98df998c2b38c77fa0 not found: ID does not exist" containerID="ea5f5913766221d12dd29b932de93408db4eca31435fdb98df998c2b38c77fa0"
Jul 16 20:34:33.303103 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.303044    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea5f5913766221d12dd29b932de93408db4eca31435fdb98df998c2b38c77fa0"} err="failed to get container status \"ea5f5913766221d12dd29b932de93408db4eca31435fdb98df998c2b38c77fa0\": rpc error: code = NotFound desc = could not find container \"ea5f5913766221d12dd29b932de93408db4eca31435fdb98df998c2b38c77fa0\": container with ID starting with ea5f5913766221d12dd29b932de93408db4eca31435fdb98df998c2b38c77fa0 not found: ID does not exist"
Jul 16 20:34:33.316583 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.316564    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-648f5697fb-f9xb5"]
Jul 16 20:34:33.318224 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.318200    2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/be2be990-3e76-486b-bf39-976ddac0225d-image-registry-private-configuration\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:34:33.318317 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.318226    2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-bound-sa-token\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:34:33.318317 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.318242    2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-registry-tls\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:34:33.318317 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.318250    2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/be2be990-3e76-486b-bf39-976ddac0225d-installation-pull-secrets\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:34:33.318317 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.318262    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be2be990-3e76-486b-bf39-976ddac0225d-trusted-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:34:33.318317 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.318274    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4kjf\" (UniqueName: \"kubernetes.io/projected/be2be990-3e76-486b-bf39-976ddac0225d-kube-api-access-c4kjf\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:34:33.318317 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.318282    2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/be2be990-3e76-486b-bf39-976ddac0225d-ca-trust-extracted\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:34:33.318317 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.318290    2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/be2be990-3e76-486b-bf39-976ddac0225d-registry-certificates\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:34:33.323909 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:33.323886    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-648f5697fb-f9xb5"]
Jul 16 20:34:34.688476 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:34.688409    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be2be990-3e76-486b-bf39-976ddac0225d" path="/var/lib/kubelet/pods/be2be990-3e76-486b-bf39-976ddac0225d/volumes"
Jul 16 20:34:44.342441 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:44.342354    2576 generic.go:356] "Generic (PLEG): container finished" podID="a3b3a919-d977-4f1e-9a34-e7c7a1368d35" containerID="c65d1668483104144f6b50152bd865fe20b225bc5ea5cfd86da961fe8272ebbc" exitCode=0
Jul 16 20:34:44.342743 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:44.342431    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf" event={"ID":"a3b3a919-d977-4f1e-9a34-e7c7a1368d35","Type":"ContainerDied","Data":"c65d1668483104144f6b50152bd865fe20b225bc5ea5cfd86da961fe8272ebbc"}
Jul 16 20:34:44.342789 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:44.342744    2576 scope.go:117] "RemoveContainer" containerID="c65d1668483104144f6b50152bd865fe20b225bc5ea5cfd86da961fe8272ebbc"
Jul 16 20:34:45.348372 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:45.348337    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-nm6rf" event={"ID":"a3b3a919-d977-4f1e-9a34-e7c7a1368d35","Type":"ContainerStarted","Data":"8ff86a0d629590310bde75723c50e2df92c588a8ab4c53a8de2589bfdc2b3507"}
Jul 16 20:34:49.365205 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:49.365172    2576 generic.go:356] "Generic (PLEG): container finished" podID="373a5e3f-70fc-4a11-bcd9-8c6a136f83b9" containerID="ba31643f98f7927ab2bec30b807670c8251888457565df260f37353cfdd64ab6" exitCode=0
Jul 16 20:34:49.365205 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:49.365208    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc" event={"ID":"373a5e3f-70fc-4a11-bcd9-8c6a136f83b9","Type":"ContainerDied","Data":"ba31643f98f7927ab2bec30b807670c8251888457565df260f37353cfdd64ab6"}
Jul 16 20:34:49.365625 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:49.365528    2576 scope.go:117] "RemoveContainer" containerID="ba31643f98f7927ab2bec30b807670c8251888457565df260f37353cfdd64ab6"
Jul 16 20:34:50.371042 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:50.371007    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8rkfc" event={"ID":"373a5e3f-70fc-4a11-bcd9-8c6a136f83b9","Type":"ContainerStarted","Data":"f011db75361eab1460037387b002625465ac385826ff0e5079316a8726e3921a"}
Jul 16 20:34:54.388899 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:54.388866    2576 generic.go:356] "Generic (PLEG): container finished" podID="3f4bd827-12b2-48f5-b027-c3ba1761b83c" containerID="2815f0e21a7ef163516acd1eaabb8efd38ec18a904f74d0f2f1f96409700c072" exitCode=0
Jul 16 20:34:54.389379 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:54.388987    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-tppbf" event={"ID":"3f4bd827-12b2-48f5-b027-c3ba1761b83c","Type":"ContainerDied","Data":"2815f0e21a7ef163516acd1eaabb8efd38ec18a904f74d0f2f1f96409700c072"}
Jul 16 20:34:54.389379 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:54.389337    2576 scope.go:117] "RemoveContainer" containerID="2815f0e21a7ef163516acd1eaabb8efd38ec18a904f74d0f2f1f96409700c072"
Jul 16 20:34:55.319994 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:55.319961    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65f496d8cd-qchkt_cbb37170-322b-4128-8731-ad11b8117cdc/router/0.log"
Jul 16 20:34:55.339617 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:55.339596    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qd8ff_fc249a17-1357-464e-be27-57a3a9182c21/serve-healthcheck-canary/0.log"
Jul 16 20:34:55.394435 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:34:55.394408    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-tppbf" event={"ID":"3f4bd827-12b2-48f5-b027-c3ba1761b83c","Type":"ContainerStarted","Data":"8e2197e0d1c505e7abd24a64844b09a99477d8612f172a0a69aa8800dfc17583"}
Jul 16 20:35:10.501692 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:10.501656    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:10.519444 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:10.519414    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:11.476401 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:11.476349    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:28.557493 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:28.557392    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 20:35:28.558010 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:28.557738    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="prometheus" containerID="cri-o://ec1aebfc5adf8cb292baf350d998aa2a76ade4968da8df85eef8bbc2aa2c8723" gracePeriod=600
Jul 16 20:35:28.558010 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:28.557805    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="thanos-sidecar" containerID="cri-o://8616557b03554bd06c16825c911df0ca0a1ba8782193220d9619f664593d2e87" gracePeriod=600
Jul 16 20:35:28.558010 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:28.557743    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="kube-rbac-proxy" containerID="cri-o://69ac7057a00c65188904540681811bfed94040361ae1e4737d92cfaa6bb6bddd" gracePeriod=600
Jul 16 20:35:28.558010 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:28.557760    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="kube-rbac-proxy-thanos" containerID="cri-o://36a8b44700711e0d3cb9cccc123286033826d321ebeb17ef4f88b7c88d3524ab" gracePeriod=600
Jul 16 20:35:28.558010 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:28.557810    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="config-reloader" containerID="cri-o://34530f821e92d1999fde5e9207c3e18ba801370400bc33d996fa73f1b9edb856" gracePeriod=600
Jul 16 20:35:28.558010 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:28.557810    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="kube-rbac-proxy-web" containerID="cri-o://0cfa33703a629ba59e0252d88aaec72519e8746b85617e27653ee116492cd904" gracePeriod=600
Jul 16 20:35:29.557964 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.557928    2576 generic.go:356] "Generic (PLEG): container finished" podID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerID="36a8b44700711e0d3cb9cccc123286033826d321ebeb17ef4f88b7c88d3524ab" exitCode=0
Jul 16 20:35:29.557964 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.557956    2576 generic.go:356] "Generic (PLEG): container finished" podID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerID="69ac7057a00c65188904540681811bfed94040361ae1e4737d92cfaa6bb6bddd" exitCode=0
Jul 16 20:35:29.557964 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.557965    2576 generic.go:356] "Generic (PLEG): container finished" podID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerID="8616557b03554bd06c16825c911df0ca0a1ba8782193220d9619f664593d2e87" exitCode=0
Jul 16 20:35:29.558541 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.557984    2576 generic.go:356] "Generic (PLEG): container finished" podID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerID="34530f821e92d1999fde5e9207c3e18ba801370400bc33d996fa73f1b9edb856" exitCode=0
Jul 16 20:35:29.558541 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.557996    2576 generic.go:356] "Generic (PLEG): container finished" podID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerID="ec1aebfc5adf8cb292baf350d998aa2a76ade4968da8df85eef8bbc2aa2c8723" exitCode=0
Jul 16 20:35:29.558541 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.558011    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa483f9b-53c4-4c4d-a0b3-17ab25e61685","Type":"ContainerDied","Data":"36a8b44700711e0d3cb9cccc123286033826d321ebeb17ef4f88b7c88d3524ab"}
Jul 16 20:35:29.558541 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.558116    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa483f9b-53c4-4c4d-a0b3-17ab25e61685","Type":"ContainerDied","Data":"69ac7057a00c65188904540681811bfed94040361ae1e4737d92cfaa6bb6bddd"}
Jul 16 20:35:29.558541 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.558160    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa483f9b-53c4-4c4d-a0b3-17ab25e61685","Type":"ContainerDied","Data":"8616557b03554bd06c16825c911df0ca0a1ba8782193220d9619f664593d2e87"}
Jul 16 20:35:29.558541 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.558206    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa483f9b-53c4-4c4d-a0b3-17ab25e61685","Type":"ContainerDied","Data":"34530f821e92d1999fde5e9207c3e18ba801370400bc33d996fa73f1b9edb856"}
Jul 16 20:35:29.558541 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.558246    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa483f9b-53c4-4c4d-a0b3-17ab25e61685","Type":"ContainerDied","Data":"ec1aebfc5adf8cb292baf350d998aa2a76ade4968da8df85eef8bbc2aa2c8723"}
Jul 16 20:35:29.802152 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.802129    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:29.923518 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.923498    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-kube-rbac-proxy\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.923658 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.923531    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-0\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.923658 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.923550    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-2\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.923658 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.923578    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-serving-certs-ca-bundle\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.923824 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.923660    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-config\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.923824 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.923706    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-metrics-client-certs\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.923824 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.923735    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-tls\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.923824 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.923771    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-metrics-client-ca\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.923824 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.923798    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-tls-assets\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.924069 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.923836    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-thanos-prometheus-http-client-file\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.924069 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.923867    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-kubelet-serving-ca-bundle\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.924069 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.923897    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-config-out\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.924069 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.923926    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-db\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.924069 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.923936    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:35:29.924069 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.923979    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-web-config\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.924069 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.923988    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:35:29.924069 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.924017    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-trusted-ca-bundle\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.924069 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.924052    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.925180 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.924079    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-grpc-tls\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.925180 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.924131    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kglwt\" (UniqueName: \"kubernetes.io/projected/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-kube-api-access-kglwt\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.925180 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.924158    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-1\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.925180 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.924197    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\" (UID: \"aa483f9b-53c4-4c4d-a0b3-17ab25e61685\") "
Jul 16 20:35:29.925180 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.924455    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:29.925180 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.924475    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:29.925180 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.924741    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:35:29.925180 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.925119    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:35:29.925912 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.925736    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:35:29.926323 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.926284    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:35:29.926441 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.926403    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:35:29.926441 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.926426    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:35:29.926567 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.926506    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:35:29.927185 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.927156    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:35:29.927281 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.927232    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-config" (OuterVolumeSpecName: "config") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:35:29.928220 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.927771    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:35:29.928220 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.927974    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-config-out" (OuterVolumeSpecName: "config-out") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:35:29.928462 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.928426    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:35:29.928608 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.928583    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 20:35:29.928826 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.928807    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:35:29.929231 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.929198    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-kube-api-access-kglwt" (OuterVolumeSpecName: "kube-api-access-kglwt") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "kube-api-access-kglwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:35:29.929515 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.929491    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:35:29.929683 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.929667    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:35:29.939242 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:29.939220    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-web-config" (OuterVolumeSpecName: "web-config") pod "aa483f9b-53c4-4c4d-a0b3-17ab25e61685" (UID: "aa483f9b-53c4-4c4d-a0b3-17ab25e61685"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 20:35:30.024839 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.024817    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.024924 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.024846    2576 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-kube-rbac-proxy\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.024924 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.024863    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.024924 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.024878    2576 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-config\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.024924 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.024892    2576 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-metrics-client-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.024924 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.024907    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.025072 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.024925    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-metrics-client-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.025072 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.024940    2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-tls-assets\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.025072 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.024955    2576 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.025072 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.024969    2576 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.025072 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.024982    2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-config-out\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.025072 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.024998    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-db\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.025072 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.025011    2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-web-config\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.025072 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.025027    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.025072 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.025041    2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.025072 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.025055    2576 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-secret-grpc-tls\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.025072 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.025069    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kglwt\" (UniqueName: \"kubernetes.io/projected/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-kube-api-access-kglwt\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.025406 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.025083    2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aa483f9b-53c4-4c4d-a0b3-17ab25e61685-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:35:30.565788 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.565758    2576 generic.go:356] "Generic (PLEG): container finished" podID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerID="0cfa33703a629ba59e0252d88aaec72519e8746b85617e27653ee116492cd904" exitCode=0
Jul 16 20:35:30.566155 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.565852    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.566155 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.565844    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa483f9b-53c4-4c4d-a0b3-17ab25e61685","Type":"ContainerDied","Data":"0cfa33703a629ba59e0252d88aaec72519e8746b85617e27653ee116492cd904"}
Jul 16 20:35:30.566155 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.566037    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aa483f9b-53c4-4c4d-a0b3-17ab25e61685","Type":"ContainerDied","Data":"e9d77b26ecb361143306f62e1aee17184a038442c5391d8d5eb594429b21ad62"}
Jul 16 20:35:30.566155 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.566098    2576 scope.go:117] "RemoveContainer" containerID="36a8b44700711e0d3cb9cccc123286033826d321ebeb17ef4f88b7c88d3524ab"
Jul 16 20:35:30.574668 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.574646    2576 scope.go:117] "RemoveContainer" containerID="69ac7057a00c65188904540681811bfed94040361ae1e4737d92cfaa6bb6bddd"
Jul 16 20:35:30.584206 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.584186    2576 scope.go:117] "RemoveContainer" containerID="0cfa33703a629ba59e0252d88aaec72519e8746b85617e27653ee116492cd904"
Jul 16 20:35:30.591978 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.591952    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 20:35:30.592400 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.592378    2576 scope.go:117] "RemoveContainer" containerID="8616557b03554bd06c16825c911df0ca0a1ba8782193220d9619f664593d2e87"
Jul 16 20:35:30.596354 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.596331    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 20:35:30.599872 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.599853    2576 scope.go:117] "RemoveContainer" containerID="34530f821e92d1999fde5e9207c3e18ba801370400bc33d996fa73f1b9edb856"
Jul 16 20:35:30.606528 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.606511    2576 scope.go:117] "RemoveContainer" containerID="ec1aebfc5adf8cb292baf350d998aa2a76ade4968da8df85eef8bbc2aa2c8723"
Jul 16 20:35:30.613762 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.613747    2576 scope.go:117] "RemoveContainer" containerID="130a60efac8663191d17a398bb45aad694ca0d185a8c6f50e52e2ab97214e48b"
Jul 16 20:35:30.620894 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.620874    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 20:35:30.621148 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621133    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="prometheus"
Jul 16 20:35:30.621193 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621149    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="prometheus"
Jul 16 20:35:30.621193 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621160    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="config-reloader"
Jul 16 20:35:30.621193 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621166    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="config-reloader"
Jul 16 20:35:30.621193 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621172    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="thanos-sidecar"
Jul 16 20:35:30.621193 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621178    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="thanos-sidecar"
Jul 16 20:35:30.621193 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621190    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="kube-rbac-proxy-thanos"
Jul 16 20:35:30.621193 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621195    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="kube-rbac-proxy-thanos"
Jul 16 20:35:30.621434 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621203    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="kube-rbac-proxy-web"
Jul 16 20:35:30.621434 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621208    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="kube-rbac-proxy-web"
Jul 16 20:35:30.621434 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621214    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be2be990-3e76-486b-bf39-976ddac0225d" containerName="registry"
Jul 16 20:35:30.621434 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621219    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="be2be990-3e76-486b-bf39-976ddac0225d" containerName="registry"
Jul 16 20:35:30.621434 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621229    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="init-config-reloader"
Jul 16 20:35:30.621434 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621234    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="init-config-reloader"
Jul 16 20:35:30.621434 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621240    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="kube-rbac-proxy"
Jul 16 20:35:30.621434 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621245    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="kube-rbac-proxy"
Jul 16 20:35:30.621434 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621289    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="be2be990-3e76-486b-bf39-976ddac0225d" containerName="registry"
Jul 16 20:35:30.621434 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621296    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="kube-rbac-proxy-web"
Jul 16 20:35:30.621434 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621301    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="kube-rbac-proxy"
Jul 16 20:35:30.621434 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621307    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="kube-rbac-proxy-thanos"
Jul 16 20:35:30.621434 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621312    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="config-reloader"
Jul 16 20:35:30.621434 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621320    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="prometheus"
Jul 16 20:35:30.621434 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.621325    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" containerName="thanos-sidecar"
Jul 16 20:35:30.623539 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.623523    2576 scope.go:117] "RemoveContainer" containerID="36a8b44700711e0d3cb9cccc123286033826d321ebeb17ef4f88b7c88d3524ab"
Jul 16 20:35:30.623794 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:35:30.623776    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36a8b44700711e0d3cb9cccc123286033826d321ebeb17ef4f88b7c88d3524ab\": container with ID starting with 36a8b44700711e0d3cb9cccc123286033826d321ebeb17ef4f88b7c88d3524ab not found: ID does not exist" containerID="36a8b44700711e0d3cb9cccc123286033826d321ebeb17ef4f88b7c88d3524ab"
Jul 16 20:35:30.623834 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.623800    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36a8b44700711e0d3cb9cccc123286033826d321ebeb17ef4f88b7c88d3524ab"} err="failed to get container status \"36a8b44700711e0d3cb9cccc123286033826d321ebeb17ef4f88b7c88d3524ab\": rpc error: code = NotFound desc = could not find container \"36a8b44700711e0d3cb9cccc123286033826d321ebeb17ef4f88b7c88d3524ab\": container with ID starting with 36a8b44700711e0d3cb9cccc123286033826d321ebeb17ef4f88b7c88d3524ab not found: ID does not exist"
Jul 16 20:35:30.623834 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.623816    2576 scope.go:117] "RemoveContainer" containerID="69ac7057a00c65188904540681811bfed94040361ae1e4737d92cfaa6bb6bddd"
Jul 16 20:35:30.624070 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:35:30.624049    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69ac7057a00c65188904540681811bfed94040361ae1e4737d92cfaa6bb6bddd\": container with ID starting with 69ac7057a00c65188904540681811bfed94040361ae1e4737d92cfaa6bb6bddd not found: ID does not exist" containerID="69ac7057a00c65188904540681811bfed94040361ae1e4737d92cfaa6bb6bddd"
Jul 16 20:35:30.624116 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.624076    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69ac7057a00c65188904540681811bfed94040361ae1e4737d92cfaa6bb6bddd"} err="failed to get container status \"69ac7057a00c65188904540681811bfed94040361ae1e4737d92cfaa6bb6bddd\": rpc error: code = NotFound desc = could not find container \"69ac7057a00c65188904540681811bfed94040361ae1e4737d92cfaa6bb6bddd\": container with ID starting with 69ac7057a00c65188904540681811bfed94040361ae1e4737d92cfaa6bb6bddd not found: ID does not exist"
Jul 16 20:35:30.624116 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.624093    2576 scope.go:117] "RemoveContainer" containerID="0cfa33703a629ba59e0252d88aaec72519e8746b85617e27653ee116492cd904"
Jul 16 20:35:30.624332 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:35:30.624316    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cfa33703a629ba59e0252d88aaec72519e8746b85617e27653ee116492cd904\": container with ID starting with 0cfa33703a629ba59e0252d88aaec72519e8746b85617e27653ee116492cd904 not found: ID does not exist" containerID="0cfa33703a629ba59e0252d88aaec72519e8746b85617e27653ee116492cd904"
Jul 16 20:35:30.624389 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.624334    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cfa33703a629ba59e0252d88aaec72519e8746b85617e27653ee116492cd904"} err="failed to get container status \"0cfa33703a629ba59e0252d88aaec72519e8746b85617e27653ee116492cd904\": rpc error: code = NotFound desc = could not find container \"0cfa33703a629ba59e0252d88aaec72519e8746b85617e27653ee116492cd904\": container with ID starting with 0cfa33703a629ba59e0252d88aaec72519e8746b85617e27653ee116492cd904 not found: ID does not exist"
Jul 16 20:35:30.624389 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.624347    2576 scope.go:117] "RemoveContainer" containerID="8616557b03554bd06c16825c911df0ca0a1ba8782193220d9619f664593d2e87"
Jul 16 20:35:30.624610 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:35:30.624590    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8616557b03554bd06c16825c911df0ca0a1ba8782193220d9619f664593d2e87\": container with ID starting with 8616557b03554bd06c16825c911df0ca0a1ba8782193220d9619f664593d2e87 not found: ID does not exist" containerID="8616557b03554bd06c16825c911df0ca0a1ba8782193220d9619f664593d2e87"
Jul 16 20:35:30.624659 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.624613    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8616557b03554bd06c16825c911df0ca0a1ba8782193220d9619f664593d2e87"} err="failed to get container status \"8616557b03554bd06c16825c911df0ca0a1ba8782193220d9619f664593d2e87\": rpc error: code = NotFound desc = could not find container \"8616557b03554bd06c16825c911df0ca0a1ba8782193220d9619f664593d2e87\": container with ID starting with 8616557b03554bd06c16825c911df0ca0a1ba8782193220d9619f664593d2e87 not found: ID does not exist"
Jul 16 20:35:30.624659 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.624626    2576 scope.go:117] "RemoveContainer" containerID="34530f821e92d1999fde5e9207c3e18ba801370400bc33d996fa73f1b9edb856"
Jul 16 20:35:30.624862 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:35:30.624845    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34530f821e92d1999fde5e9207c3e18ba801370400bc33d996fa73f1b9edb856\": container with ID starting with 34530f821e92d1999fde5e9207c3e18ba801370400bc33d996fa73f1b9edb856 not found: ID does not exist" containerID="34530f821e92d1999fde5e9207c3e18ba801370400bc33d996fa73f1b9edb856"
Jul 16 20:35:30.624904 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.624867    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34530f821e92d1999fde5e9207c3e18ba801370400bc33d996fa73f1b9edb856"} err="failed to get container status \"34530f821e92d1999fde5e9207c3e18ba801370400bc33d996fa73f1b9edb856\": rpc error: code = NotFound desc = could not find container \"34530f821e92d1999fde5e9207c3e18ba801370400bc33d996fa73f1b9edb856\": container with ID starting with 34530f821e92d1999fde5e9207c3e18ba801370400bc33d996fa73f1b9edb856 not found: ID does not exist"
Jul 16 20:35:30.624904 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.624881    2576 scope.go:117] "RemoveContainer" containerID="ec1aebfc5adf8cb292baf350d998aa2a76ade4968da8df85eef8bbc2aa2c8723"
Jul 16 20:35:30.625128 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:35:30.625112    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec1aebfc5adf8cb292baf350d998aa2a76ade4968da8df85eef8bbc2aa2c8723\": container with ID starting with ec1aebfc5adf8cb292baf350d998aa2a76ade4968da8df85eef8bbc2aa2c8723 not found: ID does not exist" containerID="ec1aebfc5adf8cb292baf350d998aa2a76ade4968da8df85eef8bbc2aa2c8723"
Jul 16 20:35:30.625181 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.625124    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.625181 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.625131    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec1aebfc5adf8cb292baf350d998aa2a76ade4968da8df85eef8bbc2aa2c8723"} err="failed to get container status \"ec1aebfc5adf8cb292baf350d998aa2a76ade4968da8df85eef8bbc2aa2c8723\": rpc error: code = NotFound desc = could not find container \"ec1aebfc5adf8cb292baf350d998aa2a76ade4968da8df85eef8bbc2aa2c8723\": container with ID starting with ec1aebfc5adf8cb292baf350d998aa2a76ade4968da8df85eef8bbc2aa2c8723 not found: ID does not exist"
Jul 16 20:35:30.625181 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.625144    2576 scope.go:117] "RemoveContainer" containerID="130a60efac8663191d17a398bb45aad694ca0d185a8c6f50e52e2ab97214e48b"
Jul 16 20:35:30.625590 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:35:30.625419    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"130a60efac8663191d17a398bb45aad694ca0d185a8c6f50e52e2ab97214e48b\": container with ID starting with 130a60efac8663191d17a398bb45aad694ca0d185a8c6f50e52e2ab97214e48b not found: ID does not exist" containerID="130a60efac8663191d17a398bb45aad694ca0d185a8c6f50e52e2ab97214e48b"
Jul 16 20:35:30.625590 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.625453    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"130a60efac8663191d17a398bb45aad694ca0d185a8c6f50e52e2ab97214e48b"} err="failed to get container status \"130a60efac8663191d17a398bb45aad694ca0d185a8c6f50e52e2ab97214e48b\": rpc error: code = NotFound desc = could not find container \"130a60efac8663191d17a398bb45aad694ca0d185a8c6f50e52e2ab97214e48b\": container with ID starting with 130a60efac8663191d17a398bb45aad694ca0d185a8c6f50e52e2ab97214e48b not found: ID does not exist"
Jul 16 20:35:30.627682 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.627662    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 16 20:35:30.628206 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.628187    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 16 20:35:30.628315 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.628289    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 16 20:35:30.628492 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.628474    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 16 20:35:30.628590 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.628502    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 16 20:35:30.628590 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.628540    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 16 20:35:30.628590 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.628546    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6ktouqipmkpmh\""
Jul 16 20:35:30.628590 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.628580    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 16 20:35:30.628860 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.628579    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 16 20:35:30.628860 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.628646    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 16 20:35:30.628860 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.628726    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 16 20:35:30.628860 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.628775    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 16 20:35:30.628860 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.628783    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 16 20:35:30.629103 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.628890    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 16 20:35:30.629103 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.629086    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jzq74\""
Jul 16 20:35:30.636068 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.635744    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 16 20:35:30.638344 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.638322    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 16 20:35:30.646128 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.646109    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 20:35:30.688507 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.688486    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa483f9b-53c4-4c4d-a0b3-17ab25e61685" path="/var/lib/kubelet/pods/aa483f9b-53c4-4c4d-a0b3-17ab25e61685/volumes"
Jul 16 20:35:30.728749 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.728728    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-config\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.728836 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.728757    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.728836 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.728775    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae0f683a-3a21-40ec-b0c3-832212333b3f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.728836 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.728806    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.728936 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.728852    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.728936 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.728904    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.728936 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.728921    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.729033 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.728944    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.729033 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.728968    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.729033 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.728988    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.729033 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.729006    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.729162 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.729033    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.729162 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.729061    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.729162 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.729082    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.729162 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.729098    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ae0f683a-3a21-40ec-b0c3-832212333b3f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.729162 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.729143    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.729322 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.729171    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-web-config\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.729322 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.729186    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.729322 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.729206    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9p2s\" (UniqueName: \"kubernetes.io/projected/ae0f683a-3a21-40ec-b0c3-832212333b3f-kube-api-access-c9p2s\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.729322 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.729230    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae0f683a-3a21-40ec-b0c3-832212333b3f-config-out\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830035 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.829982    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae0f683a-3a21-40ec-b0c3-832212333b3f-config-out\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830035 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830008    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-config\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830035 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830221 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830043    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae0f683a-3a21-40ec-b0c3-832212333b3f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830221 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830071    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830221 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830094    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830221 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830123    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830221 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830146    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830221 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830169    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830503 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830419    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830503 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830457    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830503 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830493    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830644 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830542    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830644 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830567    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830644 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830594    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830644 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ae0f683a-3a21-40ec-b0c3-832212333b3f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830835 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830647    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830835 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830682    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-web-config\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830835 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830707    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.830835 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830750    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9p2s\" (UniqueName: \"kubernetes.io/projected/ae0f683a-3a21-40ec-b0c3-832212333b3f-kube-api-access-c9p2s\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.831016 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830929    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.831016 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.830999    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.831615 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.831273    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ae0f683a-3a21-40ec-b0c3-832212333b3f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.832129 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.832098    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.832668 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.832640    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.833092 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.833069    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.833599 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.833573    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.833887 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.833829    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-config\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.834390 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.834229    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.834390 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.834345    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0f683a-3a21-40ec-b0c3-832212333b3f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.834552 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.834465    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.834615 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.834585    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae0f683a-3a21-40ec-b0c3-832212333b3f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.835306 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.835279    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.835636 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.835613    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae0f683a-3a21-40ec-b0c3-832212333b3f-config-out\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.835804 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.835788    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.835879 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.835859    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.836093 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.836077    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.836745 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.836720    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-web-config\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.836904 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.836884    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae0f683a-3a21-40ec-b0c3-832212333b3f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.839259 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.839232    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9p2s\" (UniqueName: \"kubernetes.io/projected/ae0f683a-3a21-40ec-b0c3-832212333b3f-kube-api-access-c9p2s\") pod \"prometheus-k8s-0\" (UID: \"ae0f683a-3a21-40ec-b0c3-832212333b3f\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:30.938394 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:30.938373    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:35:31.076008 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:31.075978    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 16 20:35:31.079881 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:35:31.079855    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae0f683a_3a21_40ec_b0c3_832212333b3f.slice/crio-b29428ee29ce4ebe1cc96b5b4b3fdaac9bd61b8c98a998f99613949824eecc82 WatchSource:0}: Error finding container b29428ee29ce4ebe1cc96b5b4b3fdaac9bd61b8c98a998f99613949824eecc82: Status 404 returned error can't find the container with id b29428ee29ce4ebe1cc96b5b4b3fdaac9bd61b8c98a998f99613949824eecc82
Jul 16 20:35:31.572419 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:31.572384    2576 generic.go:356] "Generic (PLEG): container finished" podID="ae0f683a-3a21-40ec-b0c3-832212333b3f" containerID="faea96662a158c2bfd8c6eb8de2a40c5d51ceafc9163897b7bb52bc9c675db4a" exitCode=0
Jul 16 20:35:31.572774 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:31.572424    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae0f683a-3a21-40ec-b0c3-832212333b3f","Type":"ContainerDied","Data":"faea96662a158c2bfd8c6eb8de2a40c5d51ceafc9163897b7bb52bc9c675db4a"}
Jul 16 20:35:31.572774 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:31.572479    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae0f683a-3a21-40ec-b0c3-832212333b3f","Type":"ContainerStarted","Data":"b29428ee29ce4ebe1cc96b5b4b3fdaac9bd61b8c98a998f99613949824eecc82"}
Jul 16 20:35:32.580351 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:32.580305    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae0f683a-3a21-40ec-b0c3-832212333b3f","Type":"ContainerStarted","Data":"753bec56d05c25391a27b7dbff6a8a2273902b95df65661464910cf1a988bfdb"}
Jul 16 20:35:32.580740 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:32.580420    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae0f683a-3a21-40ec-b0c3-832212333b3f","Type":"ContainerStarted","Data":"235e27e6049d76f1b579c5e093b0a432024fdff6490e1fa775fe770ea956a1c8"}
Jul 16 20:35:32.580740 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:32.580453    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae0f683a-3a21-40ec-b0c3-832212333b3f","Type":"ContainerStarted","Data":"c1751be961f899f6ebc5c8568d1b19e17bbfeb60058d4489a0c3e72a19c7b500"}
Jul 16 20:35:32.580740 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:32.580485    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae0f683a-3a21-40ec-b0c3-832212333b3f","Type":"ContainerStarted","Data":"3433ad8796396e896605c7a680a9581537a390e32aad588901ed5d55a944b659"}
Jul 16 20:35:32.580740 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:32.580512    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae0f683a-3a21-40ec-b0c3-832212333b3f","Type":"ContainerStarted","Data":"8cdbc18c31c6f45947eaf0c4b822679daf58591c66107e0891ca968179ff6057"}
Jul 16 20:35:32.580740 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:32.580536    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ae0f683a-3a21-40ec-b0c3-832212333b3f","Type":"ContainerStarted","Data":"46be80a85865016456de54a1fe0803f888bf5a72acd241a69a36498bb2e11986"}
Jul 16 20:35:32.629793 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:32.629732    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.612830238 podStartE2EDuration="2.629713802s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.883564ms" isStatefulPod=false podCreationTimestamp="2026-07-16 20:35:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:35:32.627705807 +0000 UTC m=+178.593640041" watchObservedRunningTime="2026-07-16 20:35:32.629713802 +0000 UTC m=+178.595648040"
Jul 16 20:35:35.938501 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:35:35.938461    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:36:30.938606 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:36:30.938562    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:36:30.955803 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:36:30.955773    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:36:31.830673 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:36:31.830640    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 16 20:37:34.412736 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:37:34.412698    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 20:37:34.413876 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:37:34.413850    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 20:37:34.431094 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:37:34.431073    2576 kubelet.go:1648] "Image garbage collection succeeded"
Jul 16 20:38:30.245751 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.245666    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"]
Jul 16 20:38:30.248207 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.248185    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:30.250890 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.250866    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 16 20:38:30.253275 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.253256    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 16 20:38:30.257803 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.257778    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 16 20:38:30.257896 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.257790    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qrv96\""
Jul 16 20:38:30.259197 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.259181    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 16 20:38:30.274825 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.274795    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"]
Jul 16 20:38:30.300461 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.300436    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23383b4a-08bf-4e83-b4f7-13443a9e940e-tmp\") pod \"opendatahub-operator-controller-manager-58fdbbf655-jpk7x\" (UID: \"23383b4a-08bf-4e83-b4f7-13443a9e940e\") " pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:30.300551 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.300475    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23383b4a-08bf-4e83-b4f7-13443a9e940e-webhook-cert\") pod \"opendatahub-operator-controller-manager-58fdbbf655-jpk7x\" (UID: \"23383b4a-08bf-4e83-b4f7-13443a9e940e\") " pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:30.300551 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.300496    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/23383b4a-08bf-4e83-b4f7-13443a9e940e-manifests\") pod \"opendatahub-operator-controller-manager-58fdbbf655-jpk7x\" (UID: \"23383b4a-08bf-4e83-b4f7-13443a9e940e\") " pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:30.300626 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.300561    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmg4p\" (UniqueName: \"kubernetes.io/projected/23383b4a-08bf-4e83-b4f7-13443a9e940e-kube-api-access-xmg4p\") pod \"opendatahub-operator-controller-manager-58fdbbf655-jpk7x\" (UID: \"23383b4a-08bf-4e83-b4f7-13443a9e940e\") " pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:30.300626 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.300621    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/23383b4a-08bf-4e83-b4f7-13443a9e940e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58fdbbf655-jpk7x\" (UID: \"23383b4a-08bf-4e83-b4f7-13443a9e940e\") " pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:30.400855 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.400831    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23383b4a-08bf-4e83-b4f7-13443a9e940e-webhook-cert\") pod \"opendatahub-operator-controller-manager-58fdbbf655-jpk7x\" (UID: \"23383b4a-08bf-4e83-b4f7-13443a9e940e\") " pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:30.400943 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.400860    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/23383b4a-08bf-4e83-b4f7-13443a9e940e-manifests\") pod \"opendatahub-operator-controller-manager-58fdbbf655-jpk7x\" (UID: \"23383b4a-08bf-4e83-b4f7-13443a9e940e\") " pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:30.400943 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.400880    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmg4p\" (UniqueName: \"kubernetes.io/projected/23383b4a-08bf-4e83-b4f7-13443a9e940e-kube-api-access-xmg4p\") pod \"opendatahub-operator-controller-manager-58fdbbf655-jpk7x\" (UID: \"23383b4a-08bf-4e83-b4f7-13443a9e940e\") " pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:30.400943 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.400926    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/23383b4a-08bf-4e83-b4f7-13443a9e940e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58fdbbf655-jpk7x\" (UID: \"23383b4a-08bf-4e83-b4f7-13443a9e940e\") " pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:30.401081 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.400960    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23383b4a-08bf-4e83-b4f7-13443a9e940e-tmp\") pod \"opendatahub-operator-controller-manager-58fdbbf655-jpk7x\" (UID: \"23383b4a-08bf-4e83-b4f7-13443a9e940e\") " pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:30.401244 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.401219    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/23383b4a-08bf-4e83-b4f7-13443a9e940e-manifests\") pod \"opendatahub-operator-controller-manager-58fdbbf655-jpk7x\" (UID: \"23383b4a-08bf-4e83-b4f7-13443a9e940e\") " pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:30.403219 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.403189    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23383b4a-08bf-4e83-b4f7-13443a9e940e-tmp\") pod \"opendatahub-operator-controller-manager-58fdbbf655-jpk7x\" (UID: \"23383b4a-08bf-4e83-b4f7-13443a9e940e\") " pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:30.403469 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.403451    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/23383b4a-08bf-4e83-b4f7-13443a9e940e-webhook-cert\") pod \"opendatahub-operator-controller-manager-58fdbbf655-jpk7x\" (UID: \"23383b4a-08bf-4e83-b4f7-13443a9e940e\") " pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:30.403609 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.403590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/23383b4a-08bf-4e83-b4f7-13443a9e940e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58fdbbf655-jpk7x\" (UID: \"23383b4a-08bf-4e83-b4f7-13443a9e940e\") " pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:30.409752 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.409730    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmg4p\" (UniqueName: \"kubernetes.io/projected/23383b4a-08bf-4e83-b4f7-13443a9e940e-kube-api-access-xmg4p\") pod \"opendatahub-operator-controller-manager-58fdbbf655-jpk7x\" (UID: \"23383b4a-08bf-4e83-b4f7-13443a9e940e\") " pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:30.558758 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.558703    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:30.693249 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.693224    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"]
Jul 16 20:38:30.695965 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:38:30.695938    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23383b4a_08bf_4e83_b4f7_13443a9e940e.slice/crio-75ca051ca0ce1ac677ed29399316efcbb5f657a4f3f10bfccb0ca2220b119a49 WatchSource:0}: Error finding container 75ca051ca0ce1ac677ed29399316efcbb5f657a4f3f10bfccb0ca2220b119a49: Status 404 returned error can't find the container with id 75ca051ca0ce1ac677ed29399316efcbb5f657a4f3f10bfccb0ca2220b119a49
Jul 16 20:38:30.697635 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:30.697616    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 20:38:31.270639 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:31.270607    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x" event={"ID":"23383b4a-08bf-4e83-b4f7-13443a9e940e","Type":"ContainerStarted","Data":"75ca051ca0ce1ac677ed29399316efcbb5f657a4f3f10bfccb0ca2220b119a49"}
Jul 16 20:38:34.286284 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:34.286251    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x" event={"ID":"23383b4a-08bf-4e83-b4f7-13443a9e940e","Type":"ContainerStarted","Data":"183bf0e5dfe57c358cc52816b454e1024491c798b8d38559ba3d037ead34216e"}
Jul 16 20:38:35.292630 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:35.292587    2576 generic.go:356] "Generic (PLEG): container finished" podID="23383b4a-08bf-4e83-b4f7-13443a9e940e" containerID="183bf0e5dfe57c358cc52816b454e1024491c798b8d38559ba3d037ead34216e" exitCode=0
Jul 16 20:38:35.292993 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:35.292677    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x" event={"ID":"23383b4a-08bf-4e83-b4f7-13443a9e940e","Type":"ContainerDied","Data":"183bf0e5dfe57c358cc52816b454e1024491c798b8d38559ba3d037ead34216e"}
Jul 16 20:38:36.299165 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:36.299129    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x" event={"ID":"23383b4a-08bf-4e83-b4f7-13443a9e940e","Type":"ContainerStarted","Data":"87332b135fb950bd8c8a3ac1f4b0978862ca78533803eb3a546bc50eaea9b4ba"}
Jul 16 20:38:36.299585 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:36.299257    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:36.322607 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:36.322558    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x" podStartSLOduration=1.987253669 podStartE2EDuration="6.32254239s" totalImagesPullingTime="3.248025602s" totalInitContainerRuntime="1.087263119s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:38:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:38:36.321776452 +0000 UTC m=+362.287710687" watchObservedRunningTime="2026-07-16 20:38:36.32254239 +0000 UTC m=+362.288476622"
Jul 16 20:38:47.305850 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:47.305820    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58fdbbf655-jpk7x"
Jul 16 20:38:50.238594 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:50.238554    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xz7gs"]
Jul 16 20:38:50.248514 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:50.248483    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xz7gs"
Jul 16 20:38:50.251104 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:50.251078    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-nsgxs\""
Jul 16 20:38:50.251250 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:50.251184    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 16 20:38:50.257057 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:50.256996    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xz7gs"]
Jul 16 20:38:50.345880 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:50.345853    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d0e5db1-b544-4e53-ae7b-9e798d79581d-cert\") pod \"odh-model-controller-7998c67dc4-xz7gs\" (UID: \"3d0e5db1-b544-4e53-ae7b-9e798d79581d\") " pod="opendatahub/odh-model-controller-7998c67dc4-xz7gs"
Jul 16 20:38:50.345979 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:50.345908    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r69m\" (UniqueName: \"kubernetes.io/projected/3d0e5db1-b544-4e53-ae7b-9e798d79581d-kube-api-access-9r69m\") pod \"odh-model-controller-7998c67dc4-xz7gs\" (UID: \"3d0e5db1-b544-4e53-ae7b-9e798d79581d\") " pod="opendatahub/odh-model-controller-7998c67dc4-xz7gs"
Jul 16 20:38:50.447052 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:50.447031    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d0e5db1-b544-4e53-ae7b-9e798d79581d-cert\") pod \"odh-model-controller-7998c67dc4-xz7gs\" (UID: \"3d0e5db1-b544-4e53-ae7b-9e798d79581d\") " pod="opendatahub/odh-model-controller-7998c67dc4-xz7gs"
Jul 16 20:38:50.447152 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:50.447072    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r69m\" (UniqueName: \"kubernetes.io/projected/3d0e5db1-b544-4e53-ae7b-9e798d79581d-kube-api-access-9r69m\") pod \"odh-model-controller-7998c67dc4-xz7gs\" (UID: \"3d0e5db1-b544-4e53-ae7b-9e798d79581d\") " pod="opendatahub/odh-model-controller-7998c67dc4-xz7gs"
Jul 16 20:38:50.447197 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:38:50.447163    2576 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 16 20:38:50.447232 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:38:50.447226    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d0e5db1-b544-4e53-ae7b-9e798d79581d-cert podName:3d0e5db1-b544-4e53-ae7b-9e798d79581d nodeName:}" failed. No retries permitted until 2026-07-16 20:38:50.947209248 +0000 UTC m=+376.913143459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3d0e5db1-b544-4e53-ae7b-9e798d79581d-cert") pod "odh-model-controller-7998c67dc4-xz7gs" (UID: "3d0e5db1-b544-4e53-ae7b-9e798d79581d") : secret "odh-model-controller-webhook-cert" not found
Jul 16 20:38:50.461007 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:50.460982    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r69m\" (UniqueName: \"kubernetes.io/projected/3d0e5db1-b544-4e53-ae7b-9e798d79581d-kube-api-access-9r69m\") pod \"odh-model-controller-7998c67dc4-xz7gs\" (UID: \"3d0e5db1-b544-4e53-ae7b-9e798d79581d\") " pod="opendatahub/odh-model-controller-7998c67dc4-xz7gs"
Jul 16 20:38:50.950767 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:50.950731    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d0e5db1-b544-4e53-ae7b-9e798d79581d-cert\") pod \"odh-model-controller-7998c67dc4-xz7gs\" (UID: \"3d0e5db1-b544-4e53-ae7b-9e798d79581d\") " pod="opendatahub/odh-model-controller-7998c67dc4-xz7gs"
Jul 16 20:38:50.953398 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:50.953342    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d0e5db1-b544-4e53-ae7b-9e798d79581d-cert\") pod \"odh-model-controller-7998c67dc4-xz7gs\" (UID: \"3d0e5db1-b544-4e53-ae7b-9e798d79581d\") " pod="opendatahub/odh-model-controller-7998c67dc4-xz7gs"
Jul 16 20:38:51.111293 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:51.111267    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mhswl"]
Jul 16 20:38:51.113788 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:51.113769    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mhswl"
Jul 16 20:38:51.116729 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:51.116708    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 16 20:38:51.116851 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:51.116713    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-lwbzn\""
Jul 16 20:38:51.125034 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:51.125013    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mhswl"]
Jul 16 20:38:51.152192 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:51.152165    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/993d7e8c-d616-4f37-b49d-1e6a7f4b44ba-tls-certs\") pod \"model-serving-api-54d89bd5fc-mhswl\" (UID: \"993d7e8c-d616-4f37-b49d-1e6a7f4b44ba\") " pod="opendatahub/model-serving-api-54d89bd5fc-mhswl"
Jul 16 20:38:51.152278 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:51.152208    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpqdz\" (UniqueName: \"kubernetes.io/projected/993d7e8c-d616-4f37-b49d-1e6a7f4b44ba-kube-api-access-rpqdz\") pod \"model-serving-api-54d89bd5fc-mhswl\" (UID: \"993d7e8c-d616-4f37-b49d-1e6a7f4b44ba\") " pod="opendatahub/model-serving-api-54d89bd5fc-mhswl"
Jul 16 20:38:51.160189 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:51.160169    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xz7gs"
Jul 16 20:38:51.252830 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:51.252803    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/993d7e8c-d616-4f37-b49d-1e6a7f4b44ba-tls-certs\") pod \"model-serving-api-54d89bd5fc-mhswl\" (UID: \"993d7e8c-d616-4f37-b49d-1e6a7f4b44ba\") " pod="opendatahub/model-serving-api-54d89bd5fc-mhswl"
Jul 16 20:38:51.253182 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:51.252861    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpqdz\" (UniqueName: \"kubernetes.io/projected/993d7e8c-d616-4f37-b49d-1e6a7f4b44ba-kube-api-access-rpqdz\") pod \"model-serving-api-54d89bd5fc-mhswl\" (UID: \"993d7e8c-d616-4f37-b49d-1e6a7f4b44ba\") " pod="opendatahub/model-serving-api-54d89bd5fc-mhswl"
Jul 16 20:38:51.255170 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:51.255151    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/993d7e8c-d616-4f37-b49d-1e6a7f4b44ba-tls-certs\") pod \"model-serving-api-54d89bd5fc-mhswl\" (UID: \"993d7e8c-d616-4f37-b49d-1e6a7f4b44ba\") " pod="opendatahub/model-serving-api-54d89bd5fc-mhswl"
Jul 16 20:38:51.260907 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:51.260876    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpqdz\" (UniqueName: \"kubernetes.io/projected/993d7e8c-d616-4f37-b49d-1e6a7f4b44ba-kube-api-access-rpqdz\") pod \"model-serving-api-54d89bd5fc-mhswl\" (UID: \"993d7e8c-d616-4f37-b49d-1e6a7f4b44ba\") " pod="opendatahub/model-serving-api-54d89bd5fc-mhswl"
Jul 16 20:38:51.285901 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:51.285884    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xz7gs"]
Jul 16 20:38:51.288299 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:38:51.288275    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d0e5db1_b544_4e53_ae7b_9e798d79581d.slice/crio-590373ccb8287deb148742afcb722d0c3462bcd91ec5cc26825f5004245c49ee WatchSource:0}: Error finding container 590373ccb8287deb148742afcb722d0c3462bcd91ec5cc26825f5004245c49ee: Status 404 returned error can't find the container with id 590373ccb8287deb148742afcb722d0c3462bcd91ec5cc26825f5004245c49ee
Jul 16 20:38:51.369996 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:51.369967    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xz7gs" event={"ID":"3d0e5db1-b544-4e53-ae7b-9e798d79581d","Type":"ContainerStarted","Data":"590373ccb8287deb148742afcb722d0c3462bcd91ec5cc26825f5004245c49ee"}
Jul 16 20:38:51.424319 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:51.424297    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mhswl"
Jul 16 20:38:51.554192 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:51.554170    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mhswl"]
Jul 16 20:38:51.556516 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:38:51.556489    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod993d7e8c_d616_4f37_b49d_1e6a7f4b44ba.slice/crio-ec1bb95b63f661482a5e51b396b5ba726be81afb71c25d9298d8d46a626f4668 WatchSource:0}: Error finding container ec1bb95b63f661482a5e51b396b5ba726be81afb71c25d9298d8d46a626f4668: Status 404 returned error can't find the container with id ec1bb95b63f661482a5e51b396b5ba726be81afb71c25d9298d8d46a626f4668
Jul 16 20:38:52.378006 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:52.377967    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mhswl" event={"ID":"993d7e8c-d616-4f37-b49d-1e6a7f4b44ba","Type":"ContainerStarted","Data":"ec1bb95b63f661482a5e51b396b5ba726be81afb71c25d9298d8d46a626f4668"}
Jul 16 20:38:55.394000 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:55.393967    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mhswl" event={"ID":"993d7e8c-d616-4f37-b49d-1e6a7f4b44ba","Type":"ContainerStarted","Data":"8120d1bf93a0cd3e0e39038a21f498f24df345d7a085795e294ae78bbdd463f5"}
Jul 16 20:38:55.394410 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:55.394029    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-mhswl"
Jul 16 20:38:55.395529 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:55.395500    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xz7gs" event={"ID":"3d0e5db1-b544-4e53-ae7b-9e798d79581d","Type":"ContainerStarted","Data":"5607dd6327883fa11bb843e45be5ae6906fad410b087d6af4cb6e9235f6ffeca"}
Jul 16 20:38:55.395657 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:55.395549    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-xz7gs"
Jul 16 20:38:55.414771 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:55.414730    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-mhswl" podStartSLOduration=1.205713202 podStartE2EDuration="4.414717433s" totalImagesPullingTime="3.209004231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:38:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:38:55.414344392 +0000 UTC m=+381.380278627" watchObservedRunningTime="2026-07-16 20:38:55.414717433 +0000 UTC m=+381.380651705"
Jul 16 20:38:55.431900 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:55.431857    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-xz7gs" podStartSLOduration=1.945697665 podStartE2EDuration="5.43184391s" totalImagesPullingTime="3.486146245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:38:55.429955811 +0000 UTC m=+381.395890046" watchObservedRunningTime="2026-07-16 20:38:55.43184391 +0000 UTC m=+381.397778143"
Jul 16 20:38:58.467312 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.467266    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"]
Jul 16 20:38:58.469643 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.469618    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"
Jul 16 20:38:58.472197 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.472168    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 16 20:38:58.473095 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.473062    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 16 20:38:58.473218 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.473130    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 16 20:38:58.473218 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.473197    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-76cz9\""
Jul 16 20:38:58.473338 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.473221    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 16 20:38:58.473574 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.473556    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 16 20:38:58.481390 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.481348    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"]
Jul 16 20:38:58.501387 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.501344    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0903bef2-f62e-42ca-92ce-4765aca0163d-manager-config\") pod \"lws-controller-manager-56b64d78d7-wmvqs\" (UID: \"0903bef2-f62e-42ca-92ce-4765aca0163d\") " pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"
Jul 16 20:38:58.501516 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.501414    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0903bef2-f62e-42ca-92ce-4765aca0163d-cert\") pod \"lws-controller-manager-56b64d78d7-wmvqs\" (UID: \"0903bef2-f62e-42ca-92ce-4765aca0163d\") " pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"
Jul 16 20:38:58.501516 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.501457    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0903bef2-f62e-42ca-92ce-4765aca0163d-metrics-cert\") pod \"lws-controller-manager-56b64d78d7-wmvqs\" (UID: \"0903bef2-f62e-42ca-92ce-4765aca0163d\") " pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"
Jul 16 20:38:58.501516 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.501487    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw5cm\" (UniqueName: \"kubernetes.io/projected/0903bef2-f62e-42ca-92ce-4765aca0163d-kube-api-access-xw5cm\") pod \"lws-controller-manager-56b64d78d7-wmvqs\" (UID: \"0903bef2-f62e-42ca-92ce-4765aca0163d\") " pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"
Jul 16 20:38:58.602698 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.602677    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0903bef2-f62e-42ca-92ce-4765aca0163d-manager-config\") pod \"lws-controller-manager-56b64d78d7-wmvqs\" (UID: \"0903bef2-f62e-42ca-92ce-4765aca0163d\") " pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"
Jul 16 20:38:58.602788 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.602708    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0903bef2-f62e-42ca-92ce-4765aca0163d-cert\") pod \"lws-controller-manager-56b64d78d7-wmvqs\" (UID: \"0903bef2-f62e-42ca-92ce-4765aca0163d\") " pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"
Jul 16 20:38:58.602788 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.602726    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0903bef2-f62e-42ca-92ce-4765aca0163d-metrics-cert\") pod \"lws-controller-manager-56b64d78d7-wmvqs\" (UID: \"0903bef2-f62e-42ca-92ce-4765aca0163d\") " pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"
Jul 16 20:38:58.602788 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.602742    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw5cm\" (UniqueName: \"kubernetes.io/projected/0903bef2-f62e-42ca-92ce-4765aca0163d-kube-api-access-xw5cm\") pod \"lws-controller-manager-56b64d78d7-wmvqs\" (UID: \"0903bef2-f62e-42ca-92ce-4765aca0163d\") " pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"
Jul 16 20:38:58.603439 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.603418    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0903bef2-f62e-42ca-92ce-4765aca0163d-manager-config\") pod \"lws-controller-manager-56b64d78d7-wmvqs\" (UID: \"0903bef2-f62e-42ca-92ce-4765aca0163d\") " pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"
Jul 16 20:38:58.605058 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.605034    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0903bef2-f62e-42ca-92ce-4765aca0163d-metrics-cert\") pod \"lws-controller-manager-56b64d78d7-wmvqs\" (UID: \"0903bef2-f62e-42ca-92ce-4765aca0163d\") " pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"
Jul 16 20:38:58.605138 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.605086    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0903bef2-f62e-42ca-92ce-4765aca0163d-cert\") pod \"lws-controller-manager-56b64d78d7-wmvqs\" (UID: \"0903bef2-f62e-42ca-92ce-4765aca0163d\") " pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"
Jul 16 20:38:58.610668 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.610647    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw5cm\" (UniqueName: \"kubernetes.io/projected/0903bef2-f62e-42ca-92ce-4765aca0163d-kube-api-access-xw5cm\") pod \"lws-controller-manager-56b64d78d7-wmvqs\" (UID: \"0903bef2-f62e-42ca-92ce-4765aca0163d\") " pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"
Jul 16 20:38:58.780168 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.780119    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"
Jul 16 20:38:58.922350 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:58.922326    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"]
Jul 16 20:38:58.924618 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:38:58.924588    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0903bef2_f62e_42ca_92ce_4765aca0163d.slice/crio-cefcbdbbbd385ac8bad0e598be18fd272309ae2035ad28049c2cf5d3ade3e60a WatchSource:0}: Error finding container cefcbdbbbd385ac8bad0e598be18fd272309ae2035ad28049c2cf5d3ade3e60a: Status 404 returned error can't find the container with id cefcbdbbbd385ac8bad0e598be18fd272309ae2035ad28049c2cf5d3ade3e60a
Jul 16 20:38:59.416941 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:38:59.416898    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs" event={"ID":"0903bef2-f62e-42ca-92ce-4765aca0163d","Type":"ContainerStarted","Data":"cefcbdbbbd385ac8bad0e598be18fd272309ae2035ad28049c2cf5d3ade3e60a"}
Jul 16 20:39:01.427694 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:01.427661    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs" event={"ID":"0903bef2-f62e-42ca-92ce-4765aca0163d","Type":"ContainerStarted","Data":"35ba1d2566381eef77f8d3a4e3559b3a170ea8d18c34a83310d6b62597d04715"}
Jul 16 20:39:01.428131 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:01.427740    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"
Jul 16 20:39:01.457297 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:01.457220    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs" podStartSLOduration=1.199817066 podStartE2EDuration="3.457204467s" totalImagesPullingTime="2.257387401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:38:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:39:01.456057068 +0000 UTC m=+387.421991437" watchObservedRunningTime="2026-07-16 20:39:01.457204467 +0000 UTC m=+387.423138782"
Jul 16 20:39:06.402078 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:06.402045    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-xz7gs"
Jul 16 20:39:06.404171 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:06.404151    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-mhswl"
Jul 16 20:39:09.183940 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.183911    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"]
Jul 16 20:39:09.186253 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.186235    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.188639 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.188623    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-2v2h8\""
Jul 16 20:39:09.189034 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.189019    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 16 20:39:09.204375 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.202393    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"]
Jul 16 20:39:09.286167 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.286138    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acd59d37-8c14-4b5e-97c3-00774e15dbe2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.286276 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.286170    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb6tl\" (UniqueName: \"kubernetes.io/projected/acd59d37-8c14-4b5e-97c3-00774e15dbe2-kube-api-access-nb6tl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.286276 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.286195    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acd59d37-8c14-4b5e-97c3-00774e15dbe2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.286276 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.286216    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acd59d37-8c14-4b5e-97c3-00774e15dbe2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.286276 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.286251    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acd59d37-8c14-4b5e-97c3-00774e15dbe2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.286276 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.286276    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acd59d37-8c14-4b5e-97c3-00774e15dbe2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.286464 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.286347    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd59d37-8c14-4b5e-97c3-00774e15dbe2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.286464 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.286389    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd59d37-8c14-4b5e-97c3-00774e15dbe2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.286464 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.286428    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acd59d37-8c14-4b5e-97c3-00774e15dbe2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.387688 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.387662    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd59d37-8c14-4b5e-97c3-00774e15dbe2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.387688 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.387689    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd59d37-8c14-4b5e-97c3-00774e15dbe2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.387856 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.387722    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acd59d37-8c14-4b5e-97c3-00774e15dbe2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.387856 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.387766    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acd59d37-8c14-4b5e-97c3-00774e15dbe2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.387856 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.387791    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb6tl\" (UniqueName: \"kubernetes.io/projected/acd59d37-8c14-4b5e-97c3-00774e15dbe2-kube-api-access-nb6tl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.387856 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.387824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acd59d37-8c14-4b5e-97c3-00774e15dbe2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.387856 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.387844    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acd59d37-8c14-4b5e-97c3-00774e15dbe2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.388106 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.388083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acd59d37-8c14-4b5e-97c3-00774e15dbe2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.388165 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.388133    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acd59d37-8c14-4b5e-97c3-00774e15dbe2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.388217 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.388133    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd59d37-8c14-4b5e-97c3-00774e15dbe2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.388347 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.388315    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acd59d37-8c14-4b5e-97c3-00774e15dbe2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.388464 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.388375    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acd59d37-8c14-4b5e-97c3-00774e15dbe2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.388464 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.388426    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acd59d37-8c14-4b5e-97c3-00774e15dbe2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.388785 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.388768    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acd59d37-8c14-4b5e-97c3-00774e15dbe2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.390078 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.390052    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acd59d37-8c14-4b5e-97c3-00774e15dbe2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.390328 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.390307    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acd59d37-8c14-4b5e-97c3-00774e15dbe2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.407987 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.407970    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb6tl\" (UniqueName: \"kubernetes.io/projected/acd59d37-8c14-4b5e-97c3-00774e15dbe2-kube-api-access-nb6tl\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.410829 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.410812    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acd59d37-8c14-4b5e-97c3-00774e15dbe2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mlcjj\" (UID: \"acd59d37-8c14-4b5e-97c3-00774e15dbe2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.496028 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.495969    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:09.649590 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:09.649565    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"]
Jul 16 20:39:09.652141 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:39:09.652111    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd59d37_8c14_4b5e_97c3_00774e15dbe2.slice/crio-590812e08e7b0ebe4505e5f88541775728e31a4a1ec42c09ac3e28cd28c0c410 WatchSource:0}: Error finding container 590812e08e7b0ebe4505e5f88541775728e31a4a1ec42c09ac3e28cd28c0c410: Status 404 returned error can't find the container with id 590812e08e7b0ebe4505e5f88541775728e31a4a1ec42c09ac3e28cd28c0c410
Jul 16 20:39:10.472213 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:10.472173    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj" event={"ID":"acd59d37-8c14-4b5e-97c3-00774e15dbe2","Type":"ContainerStarted","Data":"590812e08e7b0ebe4505e5f88541775728e31a4a1ec42c09ac3e28cd28c0c410"}
Jul 16 20:39:12.060888 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:12.060851    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 20:39:12.061281 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:12.060937    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 20:39:12.061281 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:12.060986    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 20:39:12.434146 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:12.434121    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-56b64d78d7-wmvqs"
Jul 16 20:39:12.482759 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:12.482726    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj" event={"ID":"acd59d37-8c14-4b5e-97c3-00774e15dbe2","Type":"ContainerStarted","Data":"469b0f31bccf640f3dd9ed1e3be7a9db6f18d2bd855a2b4857b3e449d48d2859"}
Jul 16 20:39:12.496846 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:12.496823    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:12.501854 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:12.501812    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj" podStartSLOduration=1.095385993 podStartE2EDuration="3.501797951s" totalImagesPullingTime="2.406411958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:39:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:39:12.500736346 +0000 UTC m=+398.466670590" watchObservedRunningTime="2026-07-16 20:39:12.501797951 +0000 UTC m=+398.467732184"
Jul 16 20:39:13.501920 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:13.501885    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:14.491760 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:14.491730    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:14.492741 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:14.492713    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mlcjj"
Jul 16 20:39:51.195885 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:51.195850    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-v6wkg"]
Jul 16 20:39:51.203702 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:51.203679    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-v6wkg"
Jul 16 20:39:51.206239 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:51.206211    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 16 20:39:51.206416 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:51.206274    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 16 20:39:51.207077 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:51.207059    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-cd8xg\""
Jul 16 20:39:51.213478 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:51.213457    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-v6wkg"]
Jul 16 20:39:51.301704 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:51.301676    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj6hb\" (UniqueName: \"kubernetes.io/projected/922b0cbd-0274-466b-8df4-c529707bbd07-kube-api-access-wj6hb\") pod \"kuadrant-operator-catalog-v6wkg\" (UID: \"922b0cbd-0274-466b-8df4-c529707bbd07\") " pod="kuadrant-system/kuadrant-operator-catalog-v6wkg"
Jul 16 20:39:51.402827 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:51.402800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj6hb\" (UniqueName: \"kubernetes.io/projected/922b0cbd-0274-466b-8df4-c529707bbd07-kube-api-access-wj6hb\") pod \"kuadrant-operator-catalog-v6wkg\" (UID: \"922b0cbd-0274-466b-8df4-c529707bbd07\") " pod="kuadrant-system/kuadrant-operator-catalog-v6wkg"
Jul 16 20:39:51.410479 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:51.410459    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj6hb\" (UniqueName: \"kubernetes.io/projected/922b0cbd-0274-466b-8df4-c529707bbd07-kube-api-access-wj6hb\") pod \"kuadrant-operator-catalog-v6wkg\" (UID: \"922b0cbd-0274-466b-8df4-c529707bbd07\") " pod="kuadrant-system/kuadrant-operator-catalog-v6wkg"
Jul 16 20:39:51.513401 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:51.513342    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-v6wkg"]
Jul 16 20:39:51.513598 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:51.513583    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-v6wkg"
Jul 16 20:39:51.639827 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:51.639801    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-v6wkg"]
Jul 16 20:39:51.641447 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:39:51.641401    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod922b0cbd_0274_466b_8df4_c529707bbd07.slice/crio-5461a880615c2bb1b3c95522f93a1ae96a6249ce4643c7916ff78026899d763f WatchSource:0}: Error finding container 5461a880615c2bb1b3c95522f93a1ae96a6249ce4643c7916ff78026899d763f: Status 404 returned error can't find the container with id 5461a880615c2bb1b3c95522f93a1ae96a6249ce4643c7916ff78026899d763f
Jul 16 20:39:51.660577 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:51.660554    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-v6wkg" event={"ID":"922b0cbd-0274-466b-8df4-c529707bbd07","Type":"ContainerStarted","Data":"5461a880615c2bb1b3c95522f93a1ae96a6249ce4643c7916ff78026899d763f"}
Jul 16 20:39:51.722627 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:51.722600    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2dw5w"]
Jul 16 20:39:51.725902 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:51.725886    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2dw5w"
Jul 16 20:39:51.732441 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:51.732417    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2dw5w"]
Jul 16 20:39:51.906320 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:51.906294    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg2j8\" (UniqueName: \"kubernetes.io/projected/450291ef-6f9f-4952-b4d0-be1d00f94da7-kube-api-access-mg2j8\") pod \"kuadrant-operator-catalog-2dw5w\" (UID: \"450291ef-6f9f-4952-b4d0-be1d00f94da7\") " pod="kuadrant-system/kuadrant-operator-catalog-2dw5w"
Jul 16 20:39:52.006834 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:52.006806    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg2j8\" (UniqueName: \"kubernetes.io/projected/450291ef-6f9f-4952-b4d0-be1d00f94da7-kube-api-access-mg2j8\") pod \"kuadrant-operator-catalog-2dw5w\" (UID: \"450291ef-6f9f-4952-b4d0-be1d00f94da7\") " pod="kuadrant-system/kuadrant-operator-catalog-2dw5w"
Jul 16 20:39:52.013981 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:52.013953    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg2j8\" (UniqueName: \"kubernetes.io/projected/450291ef-6f9f-4952-b4d0-be1d00f94da7-kube-api-access-mg2j8\") pod \"kuadrant-operator-catalog-2dw5w\" (UID: \"450291ef-6f9f-4952-b4d0-be1d00f94da7\") " pod="kuadrant-system/kuadrant-operator-catalog-2dw5w"
Jul 16 20:39:52.035860 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:52.035834    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2dw5w"
Jul 16 20:39:52.158934 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:52.158909    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2dw5w"]
Jul 16 20:39:52.160836 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:39:52.160801    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod450291ef_6f9f_4952_b4d0_be1d00f94da7.slice/crio-518f032bc3bd5d7b2409c7c9b4652a1805ca6bc528f0b1a448dcf8224def8faf WatchSource:0}: Error finding container 518f032bc3bd5d7b2409c7c9b4652a1805ca6bc528f0b1a448dcf8224def8faf: Status 404 returned error can't find the container with id 518f032bc3bd5d7b2409c7c9b4652a1805ca6bc528f0b1a448dcf8224def8faf
Jul 16 20:39:52.668946 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:52.668917    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2dw5w" event={"ID":"450291ef-6f9f-4952-b4d0-be1d00f94da7","Type":"ContainerStarted","Data":"518f032bc3bd5d7b2409c7c9b4652a1805ca6bc528f0b1a448dcf8224def8faf"}
Jul 16 20:39:54.679733 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:54.679693    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2dw5w" event={"ID":"450291ef-6f9f-4952-b4d0-be1d00f94da7","Type":"ContainerStarted","Data":"9d2cabd4e303d4c77ed539d111e74691636b3713f46a24d6277839e28a2999b8"}
Jul 16 20:39:54.681174 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:54.681144    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-v6wkg" event={"ID":"922b0cbd-0274-466b-8df4-c529707bbd07","Type":"ContainerStarted","Data":"316b01e8afd8df4c8fc7dbd1445de6329ae56ff5a812df77ef412fda5877d3ed"}
Jul 16 20:39:54.681303 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:54.681210    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-v6wkg" podUID="922b0cbd-0274-466b-8df4-c529707bbd07" containerName="registry-server" containerID="cri-o://316b01e8afd8df4c8fc7dbd1445de6329ae56ff5a812df77ef412fda5877d3ed" gracePeriod=2
Jul 16 20:39:54.696209 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:54.696123    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2dw5w" podStartSLOduration=1.802311631 podStartE2EDuration="3.696108018s" totalImagesPullingTime="1.893796387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:39:54.695183764 +0000 UTC m=+440.661118000" watchObservedRunningTime="2026-07-16 20:39:54.696108018 +0000 UTC m=+440.662042250"
Jul 16 20:39:54.711833 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:54.711793    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-v6wkg" podStartSLOduration=1.294533415 podStartE2EDuration="3.71177861s" totalImagesPullingTime="2.417245195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:39:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:39:54.710422078 +0000 UTC m=+440.676356316" watchObservedRunningTime="2026-07-16 20:39:54.71177861 +0000 UTC m=+440.677712842"
Jul 16 20:39:54.923837 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:54.923813    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-v6wkg"
Jul 16 20:39:54.930126 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:54.930077    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj6hb\" (UniqueName: \"kubernetes.io/projected/922b0cbd-0274-466b-8df4-c529707bbd07-kube-api-access-wj6hb\") pod \"922b0cbd-0274-466b-8df4-c529707bbd07\" (UID: \"922b0cbd-0274-466b-8df4-c529707bbd07\") "
Jul 16 20:39:54.932086 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:54.932062    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/922b0cbd-0274-466b-8df4-c529707bbd07-kube-api-access-wj6hb" (OuterVolumeSpecName: "kube-api-access-wj6hb") pod "922b0cbd-0274-466b-8df4-c529707bbd07" (UID: "922b0cbd-0274-466b-8df4-c529707bbd07"). InnerVolumeSpecName "kube-api-access-wj6hb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:39:55.031027 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:55.031006    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wj6hb\" (UniqueName: \"kubernetes.io/projected/922b0cbd-0274-466b-8df4-c529707bbd07-kube-api-access-wj6hb\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:39:55.687299 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:55.687264    2576 generic.go:356] "Generic (PLEG): container finished" podID="922b0cbd-0274-466b-8df4-c529707bbd07" containerID="316b01e8afd8df4c8fc7dbd1445de6329ae56ff5a812df77ef412fda5877d3ed" exitCode=0
Jul 16 20:39:55.687720 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:55.687304    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-v6wkg"
Jul 16 20:39:55.687720 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:55.687340    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-v6wkg" event={"ID":"922b0cbd-0274-466b-8df4-c529707bbd07","Type":"ContainerDied","Data":"316b01e8afd8df4c8fc7dbd1445de6329ae56ff5a812df77ef412fda5877d3ed"}
Jul 16 20:39:55.687720 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:55.687391    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-v6wkg" event={"ID":"922b0cbd-0274-466b-8df4-c529707bbd07","Type":"ContainerDied","Data":"5461a880615c2bb1b3c95522f93a1ae96a6249ce4643c7916ff78026899d763f"}
Jul 16 20:39:55.687720 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:55.687417    2576 scope.go:117] "RemoveContainer" containerID="316b01e8afd8df4c8fc7dbd1445de6329ae56ff5a812df77ef412fda5877d3ed"
Jul 16 20:39:55.697801 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:55.697783    2576 scope.go:117] "RemoveContainer" containerID="316b01e8afd8df4c8fc7dbd1445de6329ae56ff5a812df77ef412fda5877d3ed"
Jul 16 20:39:55.698087 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:39:55.698067    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"316b01e8afd8df4c8fc7dbd1445de6329ae56ff5a812df77ef412fda5877d3ed\": container with ID starting with 316b01e8afd8df4c8fc7dbd1445de6329ae56ff5a812df77ef412fda5877d3ed not found: ID does not exist" containerID="316b01e8afd8df4c8fc7dbd1445de6329ae56ff5a812df77ef412fda5877d3ed"
Jul 16 20:39:55.698163 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:55.698094    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"316b01e8afd8df4c8fc7dbd1445de6329ae56ff5a812df77ef412fda5877d3ed"} err="failed to get container status \"316b01e8afd8df4c8fc7dbd1445de6329ae56ff5a812df77ef412fda5877d3ed\": rpc error: code = NotFound desc = could not find container \"316b01e8afd8df4c8fc7dbd1445de6329ae56ff5a812df77ef412fda5877d3ed\": container with ID starting with 316b01e8afd8df4c8fc7dbd1445de6329ae56ff5a812df77ef412fda5877d3ed not found: ID does not exist"
Jul 16 20:39:55.708703 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:55.708674    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-v6wkg"]
Jul 16 20:39:55.711586 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:55.711559    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-v6wkg"]
Jul 16 20:39:56.689502 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:39:56.689468    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="922b0cbd-0274-466b-8df4-c529707bbd07" path="/var/lib/kubelet/pods/922b0cbd-0274-466b-8df4-c529707bbd07/volumes"
Jul 16 20:40:02.036959 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:02.036875    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-2dw5w"
Jul 16 20:40:02.036959 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:02.036931    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-2dw5w"
Jul 16 20:40:02.059266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:02.059243    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-2dw5w"
Jul 16 20:40:02.746876 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:02.746837    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-2dw5w"
Jul 16 20:40:22.501392 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:22.501342    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6ldw"]
Jul 16 20:40:22.501874 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:22.501694    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="922b0cbd-0274-466b-8df4-c529707bbd07" containerName="registry-server"
Jul 16 20:40:22.501874 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:22.501705    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="922b0cbd-0274-466b-8df4-c529707bbd07" containerName="registry-server"
Jul 16 20:40:22.501874 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:22.501760    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="922b0cbd-0274-466b-8df4-c529707bbd07" containerName="registry-server"
Jul 16 20:40:22.505971 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:22.505954    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6ldw"
Jul 16 20:40:22.508591 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:22.508570    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 16 20:40:22.508695 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:22.508624    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-rp5np\""
Jul 16 20:40:22.514173 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:22.514150    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6ldw"]
Jul 16 20:40:22.627934 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:22.627907    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djvmd\" (UniqueName: \"kubernetes.io/projected/e5b3f53c-d73e-45dd-abfd-b17dbad05a47-kube-api-access-djvmd\") pod \"dns-operator-controller-manager-864dd78c4c-p6ldw\" (UID: \"e5b3f53c-d73e-45dd-abfd-b17dbad05a47\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6ldw"
Jul 16 20:40:22.728659 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:22.728640    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djvmd\" (UniqueName: \"kubernetes.io/projected/e5b3f53c-d73e-45dd-abfd-b17dbad05a47-kube-api-access-djvmd\") pod \"dns-operator-controller-manager-864dd78c4c-p6ldw\" (UID: \"e5b3f53c-d73e-45dd-abfd-b17dbad05a47\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6ldw"
Jul 16 20:40:22.742954 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:22.742932    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djvmd\" (UniqueName: \"kubernetes.io/projected/e5b3f53c-d73e-45dd-abfd-b17dbad05a47-kube-api-access-djvmd\") pod \"dns-operator-controller-manager-864dd78c4c-p6ldw\" (UID: \"e5b3f53c-d73e-45dd-abfd-b17dbad05a47\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6ldw"
Jul 16 20:40:22.817058 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:22.817002    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6ldw"
Jul 16 20:40:22.954305 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:22.954282    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6ldw"]
Jul 16 20:40:22.956790 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:40:22.956766    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5b3f53c_d73e_45dd_abfd_b17dbad05a47.slice/crio-81a8dae6be40eec90942778ce96011b01a8323fb2f2a2be2f334a33c9094ab56 WatchSource:0}: Error finding container 81a8dae6be40eec90942778ce96011b01a8323fb2f2a2be2f334a33c9094ab56: Status 404 returned error can't find the container with id 81a8dae6be40eec90942778ce96011b01a8323fb2f2a2be2f334a33c9094ab56
Jul 16 20:40:23.821462 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:23.821416    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6ldw" event={"ID":"e5b3f53c-d73e-45dd-abfd-b17dbad05a47","Type":"ContainerStarted","Data":"81a8dae6be40eec90942778ce96011b01a8323fb2f2a2be2f334a33c9094ab56"}
Jul 16 20:40:25.833795 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:25.833743    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6ldw" event={"ID":"e5b3f53c-d73e-45dd-abfd-b17dbad05a47","Type":"ContainerStarted","Data":"4cd990ea60e3e08739ca4a43eb2c8a584288271ae9142d8ecc3ba8d115cba22a"}
Jul 16 20:40:25.834183 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:25.833816    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6ldw"
Jul 16 20:40:25.856576 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:25.856531    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6ldw" podStartSLOduration=1.579060983 podStartE2EDuration="3.856517705s" totalImagesPullingTime="2.277456722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:40:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:40:25.855081818 +0000 UTC m=+471.821016048" watchObservedRunningTime="2026-07-16 20:40:25.856517705 +0000 UTC m=+471.822451937"
Jul 16 20:40:28.033091 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:28.033055    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"]
Jul 16 20:40:28.036276 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:28.036257    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:28.040148 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:28.040127    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xxgmh\""
Jul 16 20:40:28.067239 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:28.067216    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"]
Jul 16 20:40:28.168852 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:28.168826    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slsbz\" (UniqueName: \"kubernetes.io/projected/7c527d75-21ad-4e5a-b19b-11c54db9ad71-kube-api-access-slsbz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6hlv9\" (UID: \"7c527d75-21ad-4e5a-b19b-11c54db9ad71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:28.168951 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:28.168886    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c527d75-21ad-4e5a-b19b-11c54db9ad71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6hlv9\" (UID: \"7c527d75-21ad-4e5a-b19b-11c54db9ad71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:28.269520 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:28.269493    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slsbz\" (UniqueName: \"kubernetes.io/projected/7c527d75-21ad-4e5a-b19b-11c54db9ad71-kube-api-access-slsbz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6hlv9\" (UID: \"7c527d75-21ad-4e5a-b19b-11c54db9ad71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:28.269614 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:28.269571    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c527d75-21ad-4e5a-b19b-11c54db9ad71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6hlv9\" (UID: \"7c527d75-21ad-4e5a-b19b-11c54db9ad71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:28.269879 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:28.269861    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c527d75-21ad-4e5a-b19b-11c54db9ad71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6hlv9\" (UID: \"7c527d75-21ad-4e5a-b19b-11c54db9ad71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:28.292926 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:28.292868    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slsbz\" (UniqueName: \"kubernetes.io/projected/7c527d75-21ad-4e5a-b19b-11c54db9ad71-kube-api-access-slsbz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6hlv9\" (UID: \"7c527d75-21ad-4e5a-b19b-11c54db9ad71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:28.346639 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:28.346617    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:28.493287 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:28.493266    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"]
Jul 16 20:40:28.496163 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:40:28.496138    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c527d75_21ad_4e5a_b19b_11c54db9ad71.slice/crio-984cf5892940c0e5135fa9a52cda966a101e5ede0049fba8c6706987729e620b WatchSource:0}: Error finding container 984cf5892940c0e5135fa9a52cda966a101e5ede0049fba8c6706987729e620b: Status 404 returned error can't find the container with id 984cf5892940c0e5135fa9a52cda966a101e5ede0049fba8c6706987729e620b
Jul 16 20:40:28.848423 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:28.848391    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9" event={"ID":"7c527d75-21ad-4e5a-b19b-11c54db9ad71","Type":"ContainerStarted","Data":"984cf5892940c0e5135fa9a52cda966a101e5ede0049fba8c6706987729e620b"}
Jul 16 20:40:31.809602 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:31.809570    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-xlskn"]
Jul 16 20:40:31.812751 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:31.812731    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlskn"
Jul 16 20:40:31.815452 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:31.815428    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-4ltlt\""
Jul 16 20:40:31.834853 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:31.834828    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-xlskn"]
Jul 16 20:40:31.898406 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:31.898376    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctdlh\" (UniqueName: \"kubernetes.io/projected/cb5301bc-a55e-45a7-a752-20fe36cad4fe-kube-api-access-ctdlh\") pod \"authorino-operator-75cc8d5cb7-xlskn\" (UID: \"cb5301bc-a55e-45a7-a752-20fe36cad4fe\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlskn"
Jul 16 20:40:31.998749 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:31.998722    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctdlh\" (UniqueName: \"kubernetes.io/projected/cb5301bc-a55e-45a7-a752-20fe36cad4fe-kube-api-access-ctdlh\") pod \"authorino-operator-75cc8d5cb7-xlskn\" (UID: \"cb5301bc-a55e-45a7-a752-20fe36cad4fe\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlskn"
Jul 16 20:40:32.013063 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:32.013046    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctdlh\" (UniqueName: \"kubernetes.io/projected/cb5301bc-a55e-45a7-a752-20fe36cad4fe-kube-api-access-ctdlh\") pod \"authorino-operator-75cc8d5cb7-xlskn\" (UID: \"cb5301bc-a55e-45a7-a752-20fe36cad4fe\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlskn"
Jul 16 20:40:32.124054 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:32.123995    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlskn"
Jul 16 20:40:32.272444 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:32.272420    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-xlskn"]
Jul 16 20:40:32.275442 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:40:32.275411    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb5301bc_a55e_45a7_a752_20fe36cad4fe.slice/crio-a992d92ed15d6d5e1fa872993c5fc7f220379749312036418155b92c6e6082d3 WatchSource:0}: Error finding container a992d92ed15d6d5e1fa872993c5fc7f220379749312036418155b92c6e6082d3: Status 404 returned error can't find the container with id a992d92ed15d6d5e1fa872993c5fc7f220379749312036418155b92c6e6082d3
Jul 16 20:40:32.868986 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:32.868949    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlskn" event={"ID":"cb5301bc-a55e-45a7-a752-20fe36cad4fe","Type":"ContainerStarted","Data":"a992d92ed15d6d5e1fa872993c5fc7f220379749312036418155b92c6e6082d3"}
Jul 16 20:40:35.885538 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:35.885460    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlskn" event={"ID":"cb5301bc-a55e-45a7-a752-20fe36cad4fe","Type":"ContainerStarted","Data":"b1822569e46c661f69d334410016c00a56f4eec85f50e11ebbabcbbecd39e008"}
Jul 16 20:40:35.885538 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:35.885514    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlskn"
Jul 16 20:40:35.887079 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:35.887048    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9" event={"ID":"7c527d75-21ad-4e5a-b19b-11c54db9ad71","Type":"ContainerStarted","Data":"745494a0c059308012e26428f19c556064657d5214a2bfd9a153a82c7ca0f986"}
Jul 16 20:40:35.887217 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:35.887159    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:35.910034 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:35.909989    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlskn" podStartSLOduration=1.543831912 podStartE2EDuration="4.909977513s" totalImagesPullingTime="3.366145601s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:40:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:40:35.907950714 +0000 UTC m=+481.873884946" watchObservedRunningTime="2026-07-16 20:40:35.909977513 +0000 UTC m=+481.875911777"
Jul 16 20:40:35.928446 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:35.928402    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9" podStartSLOduration=0.818991759 podStartE2EDuration="7.928387168s" totalImagesPullingTime="7.109395409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:40:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:40:35.927437543 +0000 UTC m=+481.893371776" watchObservedRunningTime="2026-07-16 20:40:35.928387168 +0000 UTC m=+481.894321401"
Jul 16 20:40:36.840939 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:36.840913    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-p6ldw"
Jul 16 20:40:46.894610 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:46.894579    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-xlskn"
Jul 16 20:40:46.895000 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:46.894728    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:49.098334 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.098294    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"]
Jul 16 20:40:49.098738 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.098476    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9" podUID="7c527d75-21ad-4e5a-b19b-11c54db9ad71" containerName="manager" containerID="cri-o://745494a0c059308012e26428f19c556064657d5214a2bfd9a153a82c7ca0f986" gracePeriod=2
Jul 16 20:40:49.101768 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:40:49.101722    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6hlv9\" is forbidden: User \"system:node:ip-10-0-143-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-245.ec2.internal' and this object" podUID="7c527d75-21ad-4e5a-b19b-11c54db9ad71" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:49.103019 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.102993    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"]
Jul 16 20:40:49.142268 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.142237    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"]
Jul 16 20:40:49.142691 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.142665    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c527d75-21ad-4e5a-b19b-11c54db9ad71" containerName="manager"
Jul 16 20:40:49.142691 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.142692    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c527d75-21ad-4e5a-b19b-11c54db9ad71" containerName="manager"
Jul 16 20:40:49.142885 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.142750    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c527d75-21ad-4e5a-b19b-11c54db9ad71" containerName="manager"
Jul 16 20:40:49.145491 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.145474    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"
Jul 16 20:40:49.147872 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:40:49.147847    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6hlv9\" is forbidden: User \"system:node:ip-10-0-143-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-245.ec2.internal' and this object" podUID="7c527d75-21ad-4e5a-b19b-11c54db9ad71" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:49.162445 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.162423    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"]
Jul 16 20:40:49.243543 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.243513    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43202afd-123f-406b-afbb-cc9c9bb623c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-575mp\" (UID: \"43202afd-123f-406b-afbb-cc9c9bb623c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"
Jul 16 20:40:49.243683 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.243645    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xw6j\" (UniqueName: \"kubernetes.io/projected/43202afd-123f-406b-afbb-cc9c9bb623c4-kube-api-access-9xw6j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-575mp\" (UID: \"43202afd-123f-406b-afbb-cc9c9bb623c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"
Jul 16 20:40:49.344088 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.344061    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xw6j\" (UniqueName: \"kubernetes.io/projected/43202afd-123f-406b-afbb-cc9c9bb623c4-kube-api-access-9xw6j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-575mp\" (UID: \"43202afd-123f-406b-afbb-cc9c9bb623c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"
Jul 16 20:40:49.344188 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.344106    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43202afd-123f-406b-afbb-cc9c9bb623c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-575mp\" (UID: \"43202afd-123f-406b-afbb-cc9c9bb623c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"
Jul 16 20:40:49.344451 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.344426    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43202afd-123f-406b-afbb-cc9c9bb623c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-575mp\" (UID: \"43202afd-123f-406b-afbb-cc9c9bb623c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"
Jul 16 20:40:49.356843 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.356797    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:49.359085 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:40:49.359060    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6hlv9\" is forbidden: User \"system:node:ip-10-0-143-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-245.ec2.internal' and this object" podUID="7c527d75-21ad-4e5a-b19b-11c54db9ad71" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:49.361629 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.361608    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xw6j\" (UniqueName: \"kubernetes.io/projected/43202afd-123f-406b-afbb-cc9c9bb623c4-kube-api-access-9xw6j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-575mp\" (UID: \"43202afd-123f-406b-afbb-cc9c9bb623c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"
Jul 16 20:40:49.444414 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.444389    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c527d75-21ad-4e5a-b19b-11c54db9ad71-extensions-socket-volume\") pod \"7c527d75-21ad-4e5a-b19b-11c54db9ad71\" (UID: \"7c527d75-21ad-4e5a-b19b-11c54db9ad71\") "
Jul 16 20:40:49.444511 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.444482    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slsbz\" (UniqueName: \"kubernetes.io/projected/7c527d75-21ad-4e5a-b19b-11c54db9ad71-kube-api-access-slsbz\") pod \"7c527d75-21ad-4e5a-b19b-11c54db9ad71\" (UID: \"7c527d75-21ad-4e5a-b19b-11c54db9ad71\") "
Jul 16 20:40:49.444814 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.444793    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c527d75-21ad-4e5a-b19b-11c54db9ad71-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7c527d75-21ad-4e5a-b19b-11c54db9ad71" (UID: "7c527d75-21ad-4e5a-b19b-11c54db9ad71"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:40:49.446271 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.446252    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c527d75-21ad-4e5a-b19b-11c54db9ad71-kube-api-access-slsbz" (OuterVolumeSpecName: "kube-api-access-slsbz") pod "7c527d75-21ad-4e5a-b19b-11c54db9ad71" (UID: "7c527d75-21ad-4e5a-b19b-11c54db9ad71"). InnerVolumeSpecName "kube-api-access-slsbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:40:49.480206 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.480171    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"
Jul 16 20:40:49.545866 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.545839    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slsbz\" (UniqueName: \"kubernetes.io/projected/7c527d75-21ad-4e5a-b19b-11c54db9ad71-kube-api-access-slsbz\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:40:49.545866 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.545867    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7c527d75-21ad-4e5a-b19b-11c54db9ad71-extensions-socket-volume\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:40:49.626661 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.626595    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"]
Jul 16 20:40:49.629941 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:40:49.629915    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43202afd_123f_406b_afbb_cc9c9bb623c4.slice/crio-3a6ca8b0ae6baf00304ee9c4624fe579762cf069931740569f91c56fef2ab1ad WatchSource:0}: Error finding container 3a6ca8b0ae6baf00304ee9c4624fe579762cf069931740569f91c56fef2ab1ad: Status 404 returned error can't find the container with id 3a6ca8b0ae6baf00304ee9c4624fe579762cf069931740569f91c56fef2ab1ad
Jul 16 20:40:49.954999 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.954966    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp" event={"ID":"43202afd-123f-406b-afbb-cc9c9bb623c4","Type":"ContainerStarted","Data":"126f580f2b1bfcc719d68be6f739624bc77471d432c8147269e4a09a8f3798a6"}
Jul 16 20:40:49.955156 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.955027    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp" event={"ID":"43202afd-123f-406b-afbb-cc9c9bb623c4","Type":"ContainerStarted","Data":"3a6ca8b0ae6baf00304ee9c4624fe579762cf069931740569f91c56fef2ab1ad"}
Jul 16 20:40:49.955156 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.955059    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"
Jul 16 20:40:49.956328 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.956304    2576 generic.go:356] "Generic (PLEG): container finished" podID="7c527d75-21ad-4e5a-b19b-11c54db9ad71" containerID="745494a0c059308012e26428f19c556064657d5214a2bfd9a153a82c7ca0f986" exitCode=0
Jul 16 20:40:49.956494 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.956338    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:49.956494 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.956403    2576 scope.go:117] "RemoveContainer" containerID="745494a0c059308012e26428f19c556064657d5214a2bfd9a153a82c7ca0f986"
Jul 16 20:40:49.960348 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:40:49.960327    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6hlv9\" is forbidden: User \"system:node:ip-10-0-143-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-245.ec2.internal' and this object" podUID="7c527d75-21ad-4e5a-b19b-11c54db9ad71" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:49.964850 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.964827    2576 scope.go:117] "RemoveContainer" containerID="745494a0c059308012e26428f19c556064657d5214a2bfd9a153a82c7ca0f986"
Jul 16 20:40:49.965099 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:40:49.965083    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"745494a0c059308012e26428f19c556064657d5214a2bfd9a153a82c7ca0f986\": container with ID starting with 745494a0c059308012e26428f19c556064657d5214a2bfd9a153a82c7ca0f986 not found: ID does not exist" containerID="745494a0c059308012e26428f19c556064657d5214a2bfd9a153a82c7ca0f986"
Jul 16 20:40:49.965144 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.965107    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"745494a0c059308012e26428f19c556064657d5214a2bfd9a153a82c7ca0f986"} err="failed to get container status \"745494a0c059308012e26428f19c556064657d5214a2bfd9a153a82c7ca0f986\": rpc error: code = NotFound desc = could not find container \"745494a0c059308012e26428f19c556064657d5214a2bfd9a153a82c7ca0f986\": container with ID starting with 745494a0c059308012e26428f19c556064657d5214a2bfd9a153a82c7ca0f986 not found: ID does not exist"
Jul 16 20:40:49.998630 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:49.998594    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp" podStartSLOduration=0.998581684 podStartE2EDuration="998.581684ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:40:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:40:49.997215447 +0000 UTC m=+495.963149683" watchObservedRunningTime="2026-07-16 20:40:49.998581684 +0000 UTC m=+495.964515917"
Jul 16 20:40:50.007448 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:40:50.007424    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6hlv9\" is forbidden: User \"system:node:ip-10-0-143-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-245.ec2.internal' and this object" podUID="7c527d75-21ad-4e5a-b19b-11c54db9ad71" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:50.679210 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:50.679175    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"]
Jul 16 20:40:50.684119 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:50.684095    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"
Jul 16 20:40:50.686526 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:40:50.686496    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6hlv9\" is forbidden: User \"system:node:ip-10-0-143-245.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-245.ec2.internal' and this object" podUID="7c527d75-21ad-4e5a-b19b-11c54db9ad71" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6hlv9"
Jul 16 20:40:50.690600 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:50.690576    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c527d75-21ad-4e5a-b19b-11c54db9ad71" path="/var/lib/kubelet/pods/7c527d75-21ad-4e5a-b19b-11c54db9ad71/volumes"
Jul 16 20:40:50.702591 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:50.702567    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"]
Jul 16 20:40:50.856618 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:50.856596    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5mzm\" (UniqueName: \"kubernetes.io/projected/7576757c-a2a3-4a20-8ec6-e84083f6a5d0-kube-api-access-m5mzm\") pod \"kuadrant-operator-controller-manager-86878c769f-8bp6g\" (UID: \"7576757c-a2a3-4a20-8ec6-e84083f6a5d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"
Jul 16 20:40:50.856722 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:50.856632    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7576757c-a2a3-4a20-8ec6-e84083f6a5d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8bp6g\" (UID: \"7576757c-a2a3-4a20-8ec6-e84083f6a5d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"
Jul 16 20:40:50.957159 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:50.957099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5mzm\" (UniqueName: \"kubernetes.io/projected/7576757c-a2a3-4a20-8ec6-e84083f6a5d0-kube-api-access-m5mzm\") pod \"kuadrant-operator-controller-manager-86878c769f-8bp6g\" (UID: \"7576757c-a2a3-4a20-8ec6-e84083f6a5d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"
Jul 16 20:40:50.957159 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:50.957139    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7576757c-a2a3-4a20-8ec6-e84083f6a5d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8bp6g\" (UID: \"7576757c-a2a3-4a20-8ec6-e84083f6a5d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"
Jul 16 20:40:50.957567 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:50.957547    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7576757c-a2a3-4a20-8ec6-e84083f6a5d0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8bp6g\" (UID: \"7576757c-a2a3-4a20-8ec6-e84083f6a5d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"
Jul 16 20:40:50.968630 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:50.968609    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5mzm\" (UniqueName: \"kubernetes.io/projected/7576757c-a2a3-4a20-8ec6-e84083f6a5d0-kube-api-access-m5mzm\") pod \"kuadrant-operator-controller-manager-86878c769f-8bp6g\" (UID: \"7576757c-a2a3-4a20-8ec6-e84083f6a5d0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"
Jul 16 20:40:50.995552 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:50.995527    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"
Jul 16 20:40:51.140824 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:51.140798    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"]
Jul 16 20:40:51.142915 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:40:51.142890    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7576757c_a2a3_4a20_8ec6_e84083f6a5d0.slice/crio-ad14ff4d9477c148b810376cd620825730c3aea6ac306cc9478527d0a1caf6c9 WatchSource:0}: Error finding container ad14ff4d9477c148b810376cd620825730c3aea6ac306cc9478527d0a1caf6c9: Status 404 returned error can't find the container with id ad14ff4d9477c148b810376cd620825730c3aea6ac306cc9478527d0a1caf6c9
Jul 16 20:40:51.970849 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:51.970813    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g" event={"ID":"7576757c-a2a3-4a20-8ec6-e84083f6a5d0","Type":"ContainerStarted","Data":"d707e3c9c046fb7d1c9cc7f2acb284508364cc962b0725ca9baea8243478cec5"}
Jul 16 20:40:51.971258 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:51.970864    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g" event={"ID":"7576757c-a2a3-4a20-8ec6-e84083f6a5d0","Type":"ContainerStarted","Data":"ad14ff4d9477c148b810376cd620825730c3aea6ac306cc9478527d0a1caf6c9"}
Jul 16 20:40:51.971258 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:51.970947    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"
Jul 16 20:40:52.002036 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:40:52.001994    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g" podStartSLOduration=2.001981346 podStartE2EDuration="2.001981346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:40:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:40:52.000015612 +0000 UTC m=+497.965949865" watchObservedRunningTime="2026-07-16 20:40:52.001981346 +0000 UTC m=+497.967915576"
Jul 16 20:41:00.964400 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:00.964348    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"
Jul 16 20:41:02.977998 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:02.977968    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"
Jul 16 20:41:03.038778 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:03.038736    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"]
Jul 16 20:41:03.038962 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:03.038925    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp" podUID="43202afd-123f-406b-afbb-cc9c9bb623c4" containerName="manager" containerID="cri-o://126f580f2b1bfcc719d68be6f739624bc77471d432c8147269e4a09a8f3798a6" gracePeriod=10
Jul 16 20:41:03.287687 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:03.287664    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"
Jul 16 20:41:03.455022 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:03.454989    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43202afd-123f-406b-afbb-cc9c9bb623c4-extensions-socket-volume\") pod \"43202afd-123f-406b-afbb-cc9c9bb623c4\" (UID: \"43202afd-123f-406b-afbb-cc9c9bb623c4\") "
Jul 16 20:41:03.455161 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:03.455057    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xw6j\" (UniqueName: \"kubernetes.io/projected/43202afd-123f-406b-afbb-cc9c9bb623c4-kube-api-access-9xw6j\") pod \"43202afd-123f-406b-afbb-cc9c9bb623c4\" (UID: \"43202afd-123f-406b-afbb-cc9c9bb623c4\") "
Jul 16 20:41:03.455447 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:03.455425    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43202afd-123f-406b-afbb-cc9c9bb623c4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "43202afd-123f-406b-afbb-cc9c9bb623c4" (UID: "43202afd-123f-406b-afbb-cc9c9bb623c4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:41:03.456961 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:03.456936    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43202afd-123f-406b-afbb-cc9c9bb623c4-kube-api-access-9xw6j" (OuterVolumeSpecName: "kube-api-access-9xw6j") pod "43202afd-123f-406b-afbb-cc9c9bb623c4" (UID: "43202afd-123f-406b-afbb-cc9c9bb623c4"). InnerVolumeSpecName "kube-api-access-9xw6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:41:03.556007 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:03.555954    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/43202afd-123f-406b-afbb-cc9c9bb623c4-extensions-socket-volume\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:41:03.556007 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:03.555975    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xw6j\" (UniqueName: \"kubernetes.io/projected/43202afd-123f-406b-afbb-cc9c9bb623c4-kube-api-access-9xw6j\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:41:04.032616 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:04.032583    2576 generic.go:356] "Generic (PLEG): container finished" podID="43202afd-123f-406b-afbb-cc9c9bb623c4" containerID="126f580f2b1bfcc719d68be6f739624bc77471d432c8147269e4a09a8f3798a6" exitCode=0
Jul 16 20:41:04.033047 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:04.032643    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"
Jul 16 20:41:04.033047 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:04.032656    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp" event={"ID":"43202afd-123f-406b-afbb-cc9c9bb623c4","Type":"ContainerDied","Data":"126f580f2b1bfcc719d68be6f739624bc77471d432c8147269e4a09a8f3798a6"}
Jul 16 20:41:04.033047 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:04.032698    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp" event={"ID":"43202afd-123f-406b-afbb-cc9c9bb623c4","Type":"ContainerDied","Data":"3a6ca8b0ae6baf00304ee9c4624fe579762cf069931740569f91c56fef2ab1ad"}
Jul 16 20:41:04.033047 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:04.032723    2576 scope.go:117] "RemoveContainer" containerID="126f580f2b1bfcc719d68be6f739624bc77471d432c8147269e4a09a8f3798a6"
Jul 16 20:41:04.042545 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:04.042332    2576 scope.go:117] "RemoveContainer" containerID="126f580f2b1bfcc719d68be6f739624bc77471d432c8147269e4a09a8f3798a6"
Jul 16 20:41:04.042726 ip-10-0-143-245 kubenswrapper[2576]: E0716 20:41:04.042707    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"126f580f2b1bfcc719d68be6f739624bc77471d432c8147269e4a09a8f3798a6\": container with ID starting with 126f580f2b1bfcc719d68be6f739624bc77471d432c8147269e4a09a8f3798a6 not found: ID does not exist" containerID="126f580f2b1bfcc719d68be6f739624bc77471d432c8147269e4a09a8f3798a6"
Jul 16 20:41:04.042771 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:04.042736    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"126f580f2b1bfcc719d68be6f739624bc77471d432c8147269e4a09a8f3798a6"} err="failed to get container status \"126f580f2b1bfcc719d68be6f739624bc77471d432c8147269e4a09a8f3798a6\": rpc error: code = NotFound desc = could not find container \"126f580f2b1bfcc719d68be6f739624bc77471d432c8147269e4a09a8f3798a6\": container with ID starting with 126f580f2b1bfcc719d68be6f739624bc77471d432c8147269e4a09a8f3798a6 not found: ID does not exist"
Jul 16 20:41:04.054866 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:04.054842    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"]
Jul 16 20:41:04.061619 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:04.061593    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-575mp"]
Jul 16 20:41:04.688989 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:04.688964    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43202afd-123f-406b-afbb-cc9c9bb623c4" path="/var/lib/kubelet/pods/43202afd-123f-406b-afbb-cc9c9bb623c4/volumes"
Jul 16 20:41:19.395919 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.395888    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"]
Jul 16 20:41:19.396273 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.396227    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43202afd-123f-406b-afbb-cc9c9bb623c4" containerName="manager"
Jul 16 20:41:19.396273 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.396238    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="43202afd-123f-406b-afbb-cc9c9bb623c4" containerName="manager"
Jul 16 20:41:19.396342 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.396307    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="43202afd-123f-406b-afbb-cc9c9bb623c4" containerName="manager"
Jul 16 20:41:19.401315 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.401294    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.403857 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.403835    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-4mtn8\""
Jul 16 20:41:19.413252 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.413226    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"]
Jul 16 20:41:19.479582 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.479556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e041dc27-7c87-4f74-902c-10c2393683bd-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.479671 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.479583    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcffv\" (UniqueName: \"kubernetes.io/projected/e041dc27-7c87-4f74-902c-10c2393683bd-kube-api-access-zcffv\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.479671 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.479611    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e041dc27-7c87-4f74-902c-10c2393683bd-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.479742 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.479704    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e041dc27-7c87-4f74-902c-10c2393683bd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.479742 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.479738    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e041dc27-7c87-4f74-902c-10c2393683bd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.479809 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.479761    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e041dc27-7c87-4f74-902c-10c2393683bd-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.479809 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.479776    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e041dc27-7c87-4f74-902c-10c2393683bd-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.479905 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.479812    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e041dc27-7c87-4f74-902c-10c2393683bd-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.479905 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.479878    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e041dc27-7c87-4f74-902c-10c2393683bd-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.580983 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.580958    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e041dc27-7c87-4f74-902c-10c2393683bd-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.581081 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.580988    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcffv\" (UniqueName: \"kubernetes.io/projected/e041dc27-7c87-4f74-902c-10c2393683bd-kube-api-access-zcffv\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.581081 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.581019    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e041dc27-7c87-4f74-902c-10c2393683bd-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.581081 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.581050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e041dc27-7c87-4f74-902c-10c2393683bd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.581081 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.581076    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e041dc27-7c87-4f74-902c-10c2393683bd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.581258 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.581112    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e041dc27-7c87-4f74-902c-10c2393683bd-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.581258 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.581132    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e041dc27-7c87-4f74-902c-10c2393683bd-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.581258 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.581191    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e041dc27-7c87-4f74-902c-10c2393683bd-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.581258 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.581217    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e041dc27-7c87-4f74-902c-10c2393683bd-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.581483 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.581442    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e041dc27-7c87-4f74-902c-10c2393683bd-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.581632 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.581611    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e041dc27-7c87-4f74-902c-10c2393683bd-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.581760 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.581644    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e041dc27-7c87-4f74-902c-10c2393683bd-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.581760 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.581674    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e041dc27-7c87-4f74-902c-10c2393683bd-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.581921 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.581900    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e041dc27-7c87-4f74-902c-10c2393683bd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.583510 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.583486    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e041dc27-7c87-4f74-902c-10c2393683bd-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.583651 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.583633    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e041dc27-7c87-4f74-902c-10c2393683bd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.589721 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.589701    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcffv\" (UniqueName: \"kubernetes.io/projected/e041dc27-7c87-4f74-902c-10c2393683bd-kube-api-access-zcffv\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.589831 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.589810    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e041dc27-7c87-4f74-902c-10c2393683bd-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-rdg46\" (UID: \"e041dc27-7c87-4f74-902c-10c2393683bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.712703 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.712650    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:19.849908 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.849870    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"]
Jul 16 20:41:19.853547 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:41:19.853521    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode041dc27_7c87_4f74_902c_10c2393683bd.slice/crio-7d3d0a843606c367ee144cebda925073788cf2a12ce803776018fa770e0d0240 WatchSource:0}: Error finding container 7d3d0a843606c367ee144cebda925073788cf2a12ce803776018fa770e0d0240: Status 404 returned error can't find the container with id 7d3d0a843606c367ee144cebda925073788cf2a12ce803776018fa770e0d0240
Jul 16 20:41:19.855574 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.855546    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 20:41:19.855635 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.855608    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 20:41:19.855711 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:19.855637    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 20:41:20.120935 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:20.120900    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46" event={"ID":"e041dc27-7c87-4f74-902c-10c2393683bd","Type":"ContainerStarted","Data":"236c088e8b501338c8910ac97bea521f30f47ee9663cd2ec2ddc98f768474308"}
Jul 16 20:41:20.121065 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:20.120952    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46" event={"ID":"e041dc27-7c87-4f74-902c-10c2393683bd","Type":"ContainerStarted","Data":"7d3d0a843606c367ee144cebda925073788cf2a12ce803776018fa770e0d0240"}
Jul 16 20:41:20.141896 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:20.141848    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46" podStartSLOduration=1.14183005 podStartE2EDuration="1.14183005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:41:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:41:20.140257969 +0000 UTC m=+526.106192205" watchObservedRunningTime="2026-07-16 20:41:20.14183005 +0000 UTC m=+526.107764283"
Jul 16 20:41:20.713203 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:20.713173    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:20.717953 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:20.717932    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:21.126132 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:21.126055    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:21.127018 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:21.127002    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:41:21.360733 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:21.360702    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:41:21.363169 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:21.363151    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-djq6h"
Jul 16 20:41:21.365595 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:21.365572    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 16 20:41:21.365810 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:21.365796    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-27m6w\""
Jul 16 20:41:21.374031 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:21.374002    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:41:21.399500 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:21.399476    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2r4r\" (UniqueName: \"kubernetes.io/projected/9a11f252-e376-4afb-a7a2-0327e64e8628-kube-api-access-v2r4r\") pod \"limitador-limitador-54d6d988d5-djq6h\" (UID: \"9a11f252-e376-4afb-a7a2-0327e64e8628\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-djq6h"
Jul 16 20:41:21.399599 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:21.399541    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9a11f252-e376-4afb-a7a2-0327e64e8628-config-file\") pod \"limitador-limitador-54d6d988d5-djq6h\" (UID: \"9a11f252-e376-4afb-a7a2-0327e64e8628\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-djq6h"
Jul 16 20:41:21.399803 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:21.399786    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:41:21.500882 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:21.500853    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2r4r\" (UniqueName: \"kubernetes.io/projected/9a11f252-e376-4afb-a7a2-0327e64e8628-kube-api-access-v2r4r\") pod \"limitador-limitador-54d6d988d5-djq6h\" (UID: \"9a11f252-e376-4afb-a7a2-0327e64e8628\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-djq6h"
Jul 16 20:41:21.500992 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:21.500949    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9a11f252-e376-4afb-a7a2-0327e64e8628-config-file\") pod \"limitador-limitador-54d6d988d5-djq6h\" (UID: \"9a11f252-e376-4afb-a7a2-0327e64e8628\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-djq6h"
Jul 16 20:41:21.501555 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:21.501533    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9a11f252-e376-4afb-a7a2-0327e64e8628-config-file\") pod \"limitador-limitador-54d6d988d5-djq6h\" (UID: \"9a11f252-e376-4afb-a7a2-0327e64e8628\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-djq6h"
Jul 16 20:41:21.509137 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:21.509119    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2r4r\" (UniqueName: \"kubernetes.io/projected/9a11f252-e376-4afb-a7a2-0327e64e8628-kube-api-access-v2r4r\") pod \"limitador-limitador-54d6d988d5-djq6h\" (UID: \"9a11f252-e376-4afb-a7a2-0327e64e8628\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-djq6h"
Jul 16 20:41:21.674492 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:21.674443    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-djq6h"
Jul 16 20:41:21.801852 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:21.801829    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:41:21.804091 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:41:21.804065    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a11f252_e376_4afb_a7a2_0327e64e8628.slice/crio-593c329392ac33f0f9c17343bd79f92fe5ba16931c86c433d693bd34bbe148b6 WatchSource:0}: Error finding container 593c329392ac33f0f9c17343bd79f92fe5ba16931c86c433d693bd34bbe148b6: Status 404 returned error can't find the container with id 593c329392ac33f0f9c17343bd79f92fe5ba16931c86c433d693bd34bbe148b6
Jul 16 20:41:22.132906 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:22.132811    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-djq6h" event={"ID":"9a11f252-e376-4afb-a7a2-0327e64e8628","Type":"ContainerStarted","Data":"593c329392ac33f0f9c17343bd79f92fe5ba16931c86c433d693bd34bbe148b6"}
Jul 16 20:41:25.148577 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:25.148539    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-djq6h" event={"ID":"9a11f252-e376-4afb-a7a2-0327e64e8628","Type":"ContainerStarted","Data":"a75cddd1c2050a2e2ca20abd80aa48fe0cffc6067dcc387f2743bc371567c4a0"}
Jul 16 20:41:25.149030 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:25.148683    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-djq6h"
Jul 16 20:41:25.166073 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:25.166022    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-djq6h" podStartSLOduration=1.805141509 podStartE2EDuration="4.166009728s" totalImagesPullingTime="2.360868219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:41:25.164685677 +0000 UTC m=+531.130619910" watchObservedRunningTime="2026-07-16 20:41:25.166009728 +0000 UTC m=+531.131943961"
Jul 16 20:41:36.154164 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:41:36.154136    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-djq6h"
Jul 16 20:42:34.449955 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:34.449925    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 20:42:34.450750 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:34.450731    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 20:42:35.128219 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:35.128191    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:42:42.537243 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:42.537178    2576 generic.go:356] "Generic (PLEG): container finished" podID="e041dc27-7c87-4f74-902c-10c2393683bd" containerID="236c088e8b501338c8910ac97bea521f30f47ee9663cd2ec2ddc98f768474308" exitCode=0
Jul 16 20:42:42.537708 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:42.537249    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46" event={"ID":"e041dc27-7c87-4f74-902c-10c2393683bd","Type":"ContainerDied","Data":"236c088e8b501338c8910ac97bea521f30f47ee9663cd2ec2ddc98f768474308"}
Jul 16 20:42:42.537708 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:42.537644    2576 scope.go:117] "RemoveContainer" containerID="236c088e8b501338c8910ac97bea521f30f47ee9663cd2ec2ddc98f768474308"
Jul 16 20:42:42.538585 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:42.538559    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 20:42:42.538652 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:42.538626    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 20:42:42.538691 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:42.538656    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 20:42:43.548247 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:43.548199    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46" event={"ID":"e041dc27-7c87-4f74-902c-10c2393683bd","Type":"ContainerStarted","Data":"7717a2486b4939804fa4b65d295fd85ec17652d582eb43ca06e99dbf241fc7b7"}
Jul 16 20:42:43.713551 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:43.713513    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:42:43.713842 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:43.713818    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rdg46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 16 20:42:43.713897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:43.713869    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46" podUID="e041dc27-7c87-4f74-902c-10c2393683bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 16 20:42:44.713008 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:44.712975    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rdg46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 16 20:42:44.713407 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:44.713035    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46" podUID="e041dc27-7c87-4f74-902c-10c2393683bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 16 20:42:45.713682 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:45.713649    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rdg46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 16 20:42:45.714059 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:45.713700    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46" podUID="e041dc27-7c87-4f74-902c-10c2393683bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 16 20:42:46.713317 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:46.713286    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-rdg46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 16 20:42:46.713546 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:46.713334    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46" podUID="e041dc27-7c87-4f74-902c-10c2393683bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 16 20:42:47.716995 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:47.716965    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:42:47.717465 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:47.717264    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:42:47.717962 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:42:47.717942    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-rdg46"
Jul 16 20:43:00.894849 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:43:00.894763    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:43:12.987541 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:43:12.987507    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:43:30.898147 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:43:30.898114    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:43:50.999526 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:43:50.999493    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:44:10.349673 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:10.349634    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:44:14.808715 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.808685    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"]
Jul 16 20:44:14.811067 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.811049    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.814676 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.814656    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 16 20:44:14.814788 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.814660    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 16 20:44:14.814788 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.814702    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2ztqv\""
Jul 16 20:44:14.814788 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.814718    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 16 20:44:14.841376 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.841343    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24568b9e-97df-4ce0-9108-cfc50b4f67b2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.841494 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.841419    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24568b9e-97df-4ce0-9108-cfc50b4f67b2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.841494 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.841470    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jfnz\" (UniqueName: \"kubernetes.io/projected/24568b9e-97df-4ce0-9108-cfc50b4f67b2-kube-api-access-4jfnz\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.841613 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.841577    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24568b9e-97df-4ce0-9108-cfc50b4f67b2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.841666 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.841623    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24568b9e-97df-4ce0-9108-cfc50b4f67b2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.841666 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.841650    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24568b9e-97df-4ce0-9108-cfc50b4f67b2-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.862860 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.862838    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"]
Jul 16 20:44:14.942081 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.942055    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24568b9e-97df-4ce0-9108-cfc50b4f67b2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.942188 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.942090    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24568b9e-97df-4ce0-9108-cfc50b4f67b2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.942188 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.942110    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24568b9e-97df-4ce0-9108-cfc50b4f67b2-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.942188 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.942128    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24568b9e-97df-4ce0-9108-cfc50b4f67b2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.942188 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.942160    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24568b9e-97df-4ce0-9108-cfc50b4f67b2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.942188 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.942186    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jfnz\" (UniqueName: \"kubernetes.io/projected/24568b9e-97df-4ce0-9108-cfc50b4f67b2-kube-api-access-4jfnz\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.942517 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.942498    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24568b9e-97df-4ce0-9108-cfc50b4f67b2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.942567 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.942516    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24568b9e-97df-4ce0-9108-cfc50b4f67b2-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.942605 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.942590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24568b9e-97df-4ce0-9108-cfc50b4f67b2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.944448 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.944430    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24568b9e-97df-4ce0-9108-cfc50b4f67b2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.944566 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.944547    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24568b9e-97df-4ce0-9108-cfc50b4f67b2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:14.955238 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:14.955216    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jfnz\" (UniqueName: \"kubernetes.io/projected/24568b9e-97df-4ce0-9108-cfc50b4f67b2-kube-api-access-4jfnz\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w\" (UID: \"24568b9e-97df-4ce0-9108-cfc50b4f67b2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:15.137430 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:15.137341    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:15.289048 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:15.289019    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"]
Jul 16 20:44:15.292884 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:44:15.292851    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24568b9e_97df_4ce0_9108_cfc50b4f67b2.slice/crio-b6577c090aaa08e64ef07eec0e6ee39a59e9498a780e699493bedcbc49febcf6 WatchSource:0}: Error finding container b6577c090aaa08e64ef07eec0e6ee39a59e9498a780e699493bedcbc49febcf6: Status 404 returned error can't find the container with id b6577c090aaa08e64ef07eec0e6ee39a59e9498a780e699493bedcbc49febcf6
Jul 16 20:44:15.294241 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:15.294223    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 20:44:15.992107 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:15.992069    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w" event={"ID":"24568b9e-97df-4ce0-9108-cfc50b4f67b2","Type":"ContainerStarted","Data":"b6577c090aaa08e64ef07eec0e6ee39a59e9498a780e699493bedcbc49febcf6"}
Jul 16 20:44:16.998108 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:16.998076    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w" event={"ID":"24568b9e-97df-4ce0-9108-cfc50b4f67b2","Type":"ContainerStarted","Data":"fa2e4eaa9ededb717ab469a531b2b699745bec4a48aa4eafbea272539beb424b"}
Jul 16 20:44:17.027667 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:17.027626    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w" podStartSLOduration=2.066303862 podStartE2EDuration="3.027611322s" totalImagesPullingTime="961.30746ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:44:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:44:17.027424442 +0000 UTC m=+702.993358675" watchObservedRunningTime="2026-07-16 20:44:17.027611322 +0000 UTC m=+702.993545555"
Jul 16 20:44:18.336152 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:18.336112    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:44:25.138510 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:25.138473    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:25.138510 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:25.138515    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:25.151816 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:25.151785    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:44:26.050163 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:44:26.050135    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w"
Jul 16 20:45:31.907719 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:45:31.907689    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:45:42.402337 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:45:42.402303    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:45:53.098679 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:45:53.098648    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:47:33.290418 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:47:33.290331    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:47:34.478615 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:47:34.478584    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 20:47:34.481040 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:47:34.481020    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 20:47:45.483345 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:47:45.483313    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:48:19.088930 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:48:19.088893    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:48:28.388295 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:48:28.388258    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:48:35.691903 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:48:35.691872    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:48:44.581578 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:48:44.581546    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:48:53.496855 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:48:53.496821    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:49:02.691266 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:49:02.691183    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:49:14.387307 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:49:14.387270    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:49:25.089952 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:49:25.089921    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:49:49.996710 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:49:49.996673    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:49:59.689565 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:49:59.689533    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:50:22.805049 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:50:22.805022    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:50:34.987744 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:50:34.987670    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:50:49.992752 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:50:49.992718    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:50:59.396530 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:50:59.396488    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:51:09.194208 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:09.194176    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:51:18.291650 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:18.291615    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:51:26.880710 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:26.880677    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:51:53.807027 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:53.806991    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"]
Jul 16 20:51:53.807598 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:53.807200    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g" podUID="7576757c-a2a3-4a20-8ec6-e84083f6a5d0" containerName="manager" containerID="cri-o://d707e3c9c046fb7d1c9cc7f2acb284508364cc962b0725ca9baea8243478cec5" gracePeriod=10
Jul 16 20:51:56.196454 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:56.196418    2576 generic.go:356] "Generic (PLEG): container finished" podID="7576757c-a2a3-4a20-8ec6-e84083f6a5d0" containerID="d707e3c9c046fb7d1c9cc7f2acb284508364cc962b0725ca9baea8243478cec5" exitCode=0
Jul 16 20:51:56.196795 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:56.196491    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g" event={"ID":"7576757c-a2a3-4a20-8ec6-e84083f6a5d0","Type":"ContainerDied","Data":"d707e3c9c046fb7d1c9cc7f2acb284508364cc962b0725ca9baea8243478cec5"}
Jul 16 20:51:56.453695 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:56.453638    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"
Jul 16 20:51:56.492458 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:56.492424    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5mzm\" (UniqueName: \"kubernetes.io/projected/7576757c-a2a3-4a20-8ec6-e84083f6a5d0-kube-api-access-m5mzm\") pod \"7576757c-a2a3-4a20-8ec6-e84083f6a5d0\" (UID: \"7576757c-a2a3-4a20-8ec6-e84083f6a5d0\") "
Jul 16 20:51:56.492458 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:56.492458    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7576757c-a2a3-4a20-8ec6-e84083f6a5d0-extensions-socket-volume\") pod \"7576757c-a2a3-4a20-8ec6-e84083f6a5d0\" (UID: \"7576757c-a2a3-4a20-8ec6-e84083f6a5d0\") "
Jul 16 20:51:56.492872 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:56.492848    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7576757c-a2a3-4a20-8ec6-e84083f6a5d0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7576757c-a2a3-4a20-8ec6-e84083f6a5d0" (UID: "7576757c-a2a3-4a20-8ec6-e84083f6a5d0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 20:51:56.494556 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:56.494528    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7576757c-a2a3-4a20-8ec6-e84083f6a5d0-kube-api-access-m5mzm" (OuterVolumeSpecName: "kube-api-access-m5mzm") pod "7576757c-a2a3-4a20-8ec6-e84083f6a5d0" (UID: "7576757c-a2a3-4a20-8ec6-e84083f6a5d0"). InnerVolumeSpecName "kube-api-access-m5mzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 20:51:56.593320 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:56.593297    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5mzm\" (UniqueName: \"kubernetes.io/projected/7576757c-a2a3-4a20-8ec6-e84083f6a5d0-kube-api-access-m5mzm\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:51:56.593320 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:56.593320    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7576757c-a2a3-4a20-8ec6-e84083f6a5d0-extensions-socket-volume\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 20:51:57.203268 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:57.203235    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"
Jul 16 20:51:57.203268 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:57.203250    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g" event={"ID":"7576757c-a2a3-4a20-8ec6-e84083f6a5d0","Type":"ContainerDied","Data":"ad14ff4d9477c148b810376cd620825730c3aea6ac306cc9478527d0a1caf6c9"}
Jul 16 20:51:57.203797 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:57.203314    2576 scope.go:117] "RemoveContainer" containerID="d707e3c9c046fb7d1c9cc7f2acb284508364cc962b0725ca9baea8243478cec5"
Jul 16 20:51:57.224139 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:57.224111    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"]
Jul 16 20:51:57.228234 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:57.228208    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8bp6g"]
Jul 16 20:51:58.690142 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:51:58.690105    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7576757c-a2a3-4a20-8ec6-e84083f6a5d0" path="/var/lib/kubelet/pods/7576757c-a2a3-4a20-8ec6-e84083f6a5d0/volumes"
Jul 16 20:52:34.511216 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:34.511182    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 20:52:34.514023 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:34.514000    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 20:52:54.922037 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:54.921999    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4k4fb"]
Jul 16 20:52:54.929587 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:54.922950    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7576757c-a2a3-4a20-8ec6-e84083f6a5d0" containerName="manager"
Jul 16 20:52:54.929587 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:54.922976    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7576757c-a2a3-4a20-8ec6-e84083f6a5d0" containerName="manager"
Jul 16 20:52:54.929587 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:54.923235    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7576757c-a2a3-4a20-8ec6-e84083f6a5d0" containerName="manager"
Jul 16 20:52:54.929743 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:54.929631    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4k4fb"
Jul 16 20:52:54.932010 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:54.931988    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xxgmh\""
Jul 16 20:52:54.936866 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:54.936756    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4k4fb"]
Jul 16 20:52:55.030371 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:55.030341    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a953200c-dcb6-4c55-bafd-d6ac67dfadd6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4k4fb\" (UID: \"a953200c-dcb6-4c55-bafd-d6ac67dfadd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4k4fb"
Jul 16 20:52:55.030485 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:55.030445    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpx2v\" (UniqueName: \"kubernetes.io/projected/a953200c-dcb6-4c55-bafd-d6ac67dfadd6-kube-api-access-qpx2v\") pod \"kuadrant-operator-controller-manager-86878c769f-4k4fb\" (UID: \"a953200c-dcb6-4c55-bafd-d6ac67dfadd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4k4fb"
Jul 16 20:52:55.131750 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:55.131723    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a953200c-dcb6-4c55-bafd-d6ac67dfadd6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4k4fb\" (UID: \"a953200c-dcb6-4c55-bafd-d6ac67dfadd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4k4fb"
Jul 16 20:52:55.131858 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:55.131795    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpx2v\" (UniqueName: \"kubernetes.io/projected/a953200c-dcb6-4c55-bafd-d6ac67dfadd6-kube-api-access-qpx2v\") pod \"kuadrant-operator-controller-manager-86878c769f-4k4fb\" (UID: \"a953200c-dcb6-4c55-bafd-d6ac67dfadd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4k4fb"
Jul 16 20:52:55.132080 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:55.132060    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a953200c-dcb6-4c55-bafd-d6ac67dfadd6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-4k4fb\" (UID: \"a953200c-dcb6-4c55-bafd-d6ac67dfadd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4k4fb"
Jul 16 20:52:55.139738 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:55.139717    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpx2v\" (UniqueName: \"kubernetes.io/projected/a953200c-dcb6-4c55-bafd-d6ac67dfadd6-kube-api-access-qpx2v\") pod \"kuadrant-operator-controller-manager-86878c769f-4k4fb\" (UID: \"a953200c-dcb6-4c55-bafd-d6ac67dfadd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4k4fb"
Jul 16 20:52:55.240991 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:55.240938    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4k4fb"
Jul 16 20:52:55.376673 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:55.376640    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4k4fb"]
Jul 16 20:52:55.381459 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:52:55.381414    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda953200c_dcb6_4c55_bafd_d6ac67dfadd6.slice/crio-782303f80c371386075c000f0dee22a61b303de125367dc89d552a6423ad8bbc WatchSource:0}: Error finding container 782303f80c371386075c000f0dee22a61b303de125367dc89d552a6423ad8bbc: Status 404 returned error can't find the container with id 782303f80c371386075c000f0dee22a61b303de125367dc89d552a6423ad8bbc
Jul 16 20:52:55.492197 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:55.492147    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4k4fb" event={"ID":"a953200c-dcb6-4c55-bafd-d6ac67dfadd6","Type":"ContainerStarted","Data":"c721a65e2af437dfcb0bbcf507f096743064795932119a039e15eb6625ab8445"}
Jul 16 20:52:55.492197 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:55.492193    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4k4fb" event={"ID":"a953200c-dcb6-4c55-bafd-d6ac67dfadd6","Type":"ContainerStarted","Data":"782303f80c371386075c000f0dee22a61b303de125367dc89d552a6423ad8bbc"}
Jul 16 20:52:55.492330 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:55.492248    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4k4fb"
Jul 16 20:52:55.514644 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:52:55.514601    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4k4fb" podStartSLOduration=1.514586697 podStartE2EDuration="1.514586697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:52:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:52:55.512280455 +0000 UTC m=+1221.478214689" watchObservedRunningTime="2026-07-16 20:52:55.514586697 +0000 UTC m=+1221.480520930"
Jul 16 20:53:06.499096 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:53:06.499067    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-4k4fb"
Jul 16 20:53:14.702190 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:53:14.702157    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:53:26.208335 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:53:26.208303    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:53:34.689552 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:53:34.689485    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:53:46.293871 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:53:46.293836    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:53:57.885946 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:53:57.885914    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:54:08.296955 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:54:08.296915    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:54:17.196031 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:54:17.196000    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:54:28.186580 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:54:28.186540    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:54:38.288942 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:54:38.288908    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:54:48.786473 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:54:48.786441    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:55:01.691148 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:55:01.691071    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:55:25.594868 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:55:25.594833    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:55:33.587867 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:55:33.587826    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:55:43.289572 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:55:43.289539    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:55:52.690967 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:55:52.690933    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:55:59.286934 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:55:59.286900    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:56:15.497151 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:56:15.497115    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:56:24.482848 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:56:24.482807    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:56:34.788331 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:56:34.788259    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:56:44.395694 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:56:44.395662    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:56:53.588175 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:56:53.588143    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:57:02.887459 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:57:02.887428    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:57:15.489250 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:57:15.489214    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:57:24.088945 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:57:24.088911    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:57:33.988484 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:57:33.988450    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:57:34.540510 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:57:34.540477    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 20:57:34.544897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:57:34.544869    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 20:57:46.386283 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:57:46.386247    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:57:56.487046 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:57:56.487011    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:58:06.552802 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:58:06.552725    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:58:19.897183 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:58:19.897147    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:58:31.097372 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:58:31.097338    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:58:46.788677 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:58:46.788644    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:58:58.787404 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:58:58.787356    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:59:10.095608 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:10.095573    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:59:16.802450 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.800746    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"]
Jul 16 20:59:16.808712 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.808683    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.811639 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.811619    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-23a5a19c-openshift-default-dockercfg-f9zt2\""
Jul 16 20:59:16.827056 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.827033    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"]
Jul 16 20:59:16.872233 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.872201    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-podinfo\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.872348 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.872243    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwn2r\" (UniqueName: \"kubernetes.io/projected/bc6be4ea-a4c9-43a8-9474-e47385aa0069-kube-api-access-lwn2r\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.872348 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.872312    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-workload-certs\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.872348 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.872339    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-workload-socket\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.872487 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.872410    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-credential-socket\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.872487 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.872460    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-token\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.872556 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.872487    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-envoy\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.872556 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.872510    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-data\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.872645 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.872574    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istiod-ca-cert\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.973349 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.973322    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-podinfo\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.973522 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.973374    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwn2r\" (UniqueName: \"kubernetes.io/projected/bc6be4ea-a4c9-43a8-9474-e47385aa0069-kube-api-access-lwn2r\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.973522 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.973421    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-workload-certs\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.973522 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.973438    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-workload-socket\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.973522 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.973456    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-credential-socket\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.973522 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.973485    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-token\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.973522 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.973511    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-envoy\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.973807 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.973543    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-data\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.973807 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.973585    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istiod-ca-cert\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.973915 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.973857    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-workload-certs\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.973962 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.973906    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-workload-socket\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.974023 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.973995    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-credential-socket\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.974068 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.974053    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-data\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.974289 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.974267    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istiod-ca-cert\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.975606 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.975591    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-podinfo\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.975788 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.975774    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-envoy\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.982492 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.982464    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwn2r\" (UniqueName: \"kubernetes.io/projected/bc6be4ea-a4c9-43a8-9474-e47385aa0069-kube-api-access-lwn2r\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:16.982915 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:16.982892    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-token\") pod \"e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") " pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:17.123078 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:17.123021    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:17.260384 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:17.260344    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"]
Jul 16 20:59:17.262336 ip-10-0-143-245 kubenswrapper[2576]: W0716 20:59:17.262309    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc6be4ea_a4c9_43a8_9474_e47385aa0069.slice/crio-5b57f090a86061b5cec30d2166f5a399a9c94deb14445224c1872b7495f98b71 WatchSource:0}: Error finding container 5b57f090a86061b5cec30d2166f5a399a9c94deb14445224c1872b7495f98b71: Status 404 returned error can't find the container with id 5b57f090a86061b5cec30d2166f5a399a9c94deb14445224c1872b7495f98b71
Jul 16 20:59:17.264567 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:17.264536    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 20:59:17.264645 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:17.264600    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 20:59:17.264645 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:17.264633    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 20:59:17.285064 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:17.285039    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh" event={"ID":"bc6be4ea-a4c9-43a8-9474-e47385aa0069","Type":"ContainerStarted","Data":"5b57f090a86061b5cec30d2166f5a399a9c94deb14445224c1872b7495f98b71"}
Jul 16 20:59:18.291187 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:18.291154    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh" event={"ID":"bc6be4ea-a4c9-43a8-9474-e47385aa0069","Type":"ContainerStarted","Data":"be19e67652c1d0e9260b581c97ce13eeb93e1ef5f2e40952dc5ddf6fb333cb0a"}
Jul 16 20:59:18.321000 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:18.320949    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh" podStartSLOduration=2.320934231 podStartE2EDuration="2.320934231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 20:59:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 20:59:18.319247144 +0000 UTC m=+1604.285181376" watchObservedRunningTime="2026-07-16 20:59:18.320934231 +0000 UTC m=+1604.286868443"
Jul 16 20:59:19.124115 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:19.124081    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:19.129191 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:19.129164    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:19.296885 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:19.296861    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:19.297709 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:19.297691    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:25.691494 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:25.691459    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 20:59:34.297988 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:34.297955    2576 patch_prober.go:28] interesting pod/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 16 20:59:34.298392 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:34.298009    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh" podUID="bc6be4ea-a4c9-43a8-9474-e47385aa0069" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 16 20:59:34.379458 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:34.379431    2576 generic.go:356] "Generic (PLEG): container finished" podID="bc6be4ea-a4c9-43a8-9474-e47385aa0069" containerID="be19e67652c1d0e9260b581c97ce13eeb93e1ef5f2e40952dc5ddf6fb333cb0a" exitCode=0
Jul 16 20:59:34.379581 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:34.379505    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh" event={"ID":"bc6be4ea-a4c9-43a8-9474-e47385aa0069","Type":"ContainerDied","Data":"be19e67652c1d0e9260b581c97ce13eeb93e1ef5f2e40952dc5ddf6fb333cb0a"}
Jul 16 20:59:34.379878 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:34.379854    2576 scope.go:117] "RemoveContainer" containerID="be19e67652c1d0e9260b581c97ce13eeb93e1ef5f2e40952dc5ddf6fb333cb0a"
Jul 16 20:59:34.380828 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:34.380800    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 20:59:34.380931 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:34.380865    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 20:59:34.380931 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:34.380897    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 20:59:35.387218 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:35.387185    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh" event={"ID":"bc6be4ea-a4c9-43a8-9474-e47385aa0069","Type":"ContainerStarted","Data":"be27b322c402d8ad692b17d99dc49685c0bfdebe8ced93525f22aec1e4c6c5b2"}
Jul 16 20:59:36.124007 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:36.123971    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:36.124174 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:36.124095    2576 patch_prober.go:28] interesting pod/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 16 20:59:36.124174 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:36.124150    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh" podUID="bc6be4ea-a4c9-43a8-9474-e47385aa0069" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 16 20:59:37.124074 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:37.124038    2576 patch_prober.go:28] interesting pod/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 16 20:59:37.124475 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:37.124104    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh" podUID="bc6be4ea-a4c9-43a8-9474-e47385aa0069" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 16 20:59:38.123897 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:38.123858    2576 patch_prober.go:28] interesting pod/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 16 20:59:38.124081 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:38.123928    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh" podUID="bc6be4ea-a4c9-43a8-9474-e47385aa0069" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 16 20:59:39.124469 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:39.124426    2576 patch_prober.go:28] interesting pod/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 16 20:59:39.124855 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:39.124493    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh" podUID="bc6be4ea-a4c9-43a8-9474-e47385aa0069" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 16 20:59:40.127417 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:40.127380    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:40.127796 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:40.127640    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 20:59:40.128374 ip-10-0-143-245 kubenswrapper[2576]: I0716 20:59:40.128339    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 21:00:00.173671 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:00:00.173638    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737260-nrvbs"]
Jul 16 21:00:00.177193 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:00:00.177176    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-nrvbs"
Jul 16 21:00:00.181015 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:00:00.180994    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:00:00.186470 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:00:00.186448    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-nrvbs"
Jul 16 21:00:00.203812 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:00:00.203786    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737260-nrvbs"]
Jul 16 21:00:00.323926 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:00:00.323899    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737260-nrvbs"]
Jul 16 21:00:00.325503 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:00:00.325473    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71d2cccc_202a_4177_b416_bbc58fa05427.slice/crio-8725206bf0f21c038c6c44fa65a8570d55e396c85613f8e011fad02d2731e973 WatchSource:0}: Error finding container 8725206bf0f21c038c6c44fa65a8570d55e396c85613f8e011fad02d2731e973: Status 404 returned error can't find the container with id 8725206bf0f21c038c6c44fa65a8570d55e396c85613f8e011fad02d2731e973
Jul 16 21:00:00.327436 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:00:00.327414    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 21:00:00.516380 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:00:00.516300    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-nrvbs" event={"ID":"71d2cccc-202a-4177-b416-bbc58fa05427","Type":"ContainerStarted","Data":"8725206bf0f21c038c6c44fa65a8570d55e396c85613f8e011fad02d2731e973"}
Jul 16 21:00:04.540110 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:00:04.540074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-nrvbs" event={"ID":"71d2cccc-202a-4177-b416-bbc58fa05427","Type":"ContainerStarted","Data":"2b6750a956eb735afd9ca381d6a781d57b9b6de66411a784c07692120c2b01a7"}
Jul 16 21:00:04.557779 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:00:04.557735    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-nrvbs" podStartSLOduration=1.043006934 podStartE2EDuration="4.557719775s" totalImagesPullingTime="3.514712841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:00:04.556017543 +0000 UTC m=+1650.521951776" watchObservedRunningTime="2026-07-16 21:00:04.557719775 +0000 UTC m=+1650.523654077"
Jul 16 21:00:05.546152 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:00:05.546111    2576 generic.go:356] "Generic (PLEG): container finished" podID="71d2cccc-202a-4177-b416-bbc58fa05427" containerID="2b6750a956eb735afd9ca381d6a781d57b9b6de66411a784c07692120c2b01a7" exitCode=0
Jul 16 21:00:05.546553 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:00:05.546199    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-nrvbs" event={"ID":"71d2cccc-202a-4177-b416-bbc58fa05427","Type":"ContainerDied","Data":"2b6750a956eb735afd9ca381d6a781d57b9b6de66411a784c07692120c2b01a7"}
Jul 16 21:00:06.684732 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:00:06.684709    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-nrvbs"
Jul 16 21:00:07.557101 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:00:07.557074    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-nrvbs"
Jul 16 21:00:07.557252 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:00:07.557074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737260-nrvbs" event={"ID":"71d2cccc-202a-4177-b416-bbc58fa05427","Type":"ContainerDied","Data":"8725206bf0f21c038c6c44fa65a8570d55e396c85613f8e011fad02d2731e973"}
Jul 16 21:00:07.557252 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:00:07.557192    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8725206bf0f21c038c6c44fa65a8570d55e396c85613f8e011fad02d2731e973"
Jul 16 21:02:23.192491 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:23.192459    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"]
Jul 16 21:02:23.193006 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:23.192647    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh" podUID="bc6be4ea-a4c9-43a8-9474-e47385aa0069" containerName="istio-proxy" containerID="cri-o://be27b322c402d8ad692b17d99dc49685c0bfdebe8ced93525f22aec1e4c6c5b2" gracePeriod=30
Jul 16 21:02:25.130163 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:25.130126    2576 patch_prober.go:28] interesting pod/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 21:02:25.130581 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:25.130188    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh" podUID="bc6be4ea-a4c9-43a8-9474-e47385aa0069" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 21:02:26.301557 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:26.301524    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:02:28.453397 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.453323    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 21:02:28.612269 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.612242    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-podinfo\") pod \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") "
Jul 16 21:02:28.612416 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.612274    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-workload-socket\") pod \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") "
Jul 16 21:02:28.612416 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.612333    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwn2r\" (UniqueName: \"kubernetes.io/projected/bc6be4ea-a4c9-43a8-9474-e47385aa0069-kube-api-access-lwn2r\") pod \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") "
Jul 16 21:02:28.612416 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.612351    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-envoy\") pod \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") "
Jul 16 21:02:28.612416 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.612392    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istiod-ca-cert\") pod \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") "
Jul 16 21:02:28.612617 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.612420    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-data\") pod \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") "
Jul 16 21:02:28.612617 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.612446    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-credential-socket\") pod \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") "
Jul 16 21:02:28.612617 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.612515    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-token\") pod \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") "
Jul 16 21:02:28.612617 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.612547    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-workload-certs\") pod \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\" (UID: \"bc6be4ea-a4c9-43a8-9474-e47385aa0069\") "
Jul 16 21:02:28.612824 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.612632    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bc6be4ea-a4c9-43a8-9474-e47385aa0069" (UID: "bc6be4ea-a4c9-43a8-9474-e47385aa0069"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:02:28.612883 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.612855    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bc6be4ea-a4c9-43a8-9474-e47385aa0069" (UID: "bc6be4ea-a4c9-43a8-9474-e47385aa0069"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:02:28.612883 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.612865    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bc6be4ea-a4c9-43a8-9474-e47385aa0069" (UID: "bc6be4ea-a4c9-43a8-9474-e47385aa0069"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:02:28.613000 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.612891    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bc6be4ea-a4c9-43a8-9474-e47385aa0069" (UID: "bc6be4ea-a4c9-43a8-9474-e47385aa0069"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:02:28.613000 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.612923    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-workload-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:02:28.613448 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.613417    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-data" (OuterVolumeSpecName: "istio-data") pod "bc6be4ea-a4c9-43a8-9474-e47385aa0069" (UID: "bc6be4ea-a4c9-43a8-9474-e47385aa0069"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:02:28.614587 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.614564    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bc6be4ea-a4c9-43a8-9474-e47385aa0069" (UID: "bc6be4ea-a4c9-43a8-9474-e47385aa0069"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 21:02:28.614698 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.614680    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-token" (OuterVolumeSpecName: "istio-token") pod "bc6be4ea-a4c9-43a8-9474-e47385aa0069" (UID: "bc6be4ea-a4c9-43a8-9474-e47385aa0069"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:02:28.614889 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.614860    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bc6be4ea-a4c9-43a8-9474-e47385aa0069" (UID: "bc6be4ea-a4c9-43a8-9474-e47385aa0069"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:02:28.614889 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.614869    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc6be4ea-a4c9-43a8-9474-e47385aa0069-kube-api-access-lwn2r" (OuterVolumeSpecName: "kube-api-access-lwn2r") pod "bc6be4ea-a4c9-43a8-9474-e47385aa0069" (UID: "bc6be4ea-a4c9-43a8-9474-e47385aa0069"). InnerVolumeSpecName "kube-api-access-lwn2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:02:28.714370 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.714311    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwn2r\" (UniqueName: \"kubernetes.io/projected/bc6be4ea-a4c9-43a8-9474-e47385aa0069-kube-api-access-lwn2r\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:02:28.714370 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.714334    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-envoy\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:02:28.714370 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.714343    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istiod-ca-cert\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:02:28.714370 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.714351    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-data\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:02:28.714588 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.714518    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-credential-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:02:28.714588 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.714530    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-token\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:02:28.714588 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.714538    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc6be4ea-a4c9-43a8-9474-e47385aa0069-workload-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:02:28.714588 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:28.714548    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc6be4ea-a4c9-43a8-9474-e47385aa0069-istio-podinfo\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:02:29.260555 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:29.260525    2576 generic.go:356] "Generic (PLEG): container finished" podID="bc6be4ea-a4c9-43a8-9474-e47385aa0069" containerID="be27b322c402d8ad692b17d99dc49685c0bfdebe8ced93525f22aec1e4c6c5b2" exitCode=0
Jul 16 21:02:29.260728 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:29.260561    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh" event={"ID":"bc6be4ea-a4c9-43a8-9474-e47385aa0069","Type":"ContainerDied","Data":"be27b322c402d8ad692b17d99dc49685c0bfdebe8ced93525f22aec1e4c6c5b2"}
Jul 16 21:02:29.260728 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:29.260573    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"
Jul 16 21:02:29.260728 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:29.260602    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh" event={"ID":"bc6be4ea-a4c9-43a8-9474-e47385aa0069","Type":"ContainerDied","Data":"5b57f090a86061b5cec30d2166f5a399a9c94deb14445224c1872b7495f98b71"}
Jul 16 21:02:29.260728 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:29.260623    2576 scope.go:117] "RemoveContainer" containerID="be27b322c402d8ad692b17d99dc49685c0bfdebe8ced93525f22aec1e4c6c5b2"
Jul 16 21:02:29.269410 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:29.269383    2576 scope.go:117] "RemoveContainer" containerID="be19e67652c1d0e9260b581c97ce13eeb93e1ef5f2e40952dc5ddf6fb333cb0a"
Jul 16 21:02:29.277345 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:29.277329    2576 scope.go:117] "RemoveContainer" containerID="be27b322c402d8ad692b17d99dc49685c0bfdebe8ced93525f22aec1e4c6c5b2"
Jul 16 21:02:29.277552 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:29.277533    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"]
Jul 16 21:02:29.277663 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:02:29.277641    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be27b322c402d8ad692b17d99dc49685c0bfdebe8ced93525f22aec1e4c6c5b2\": container with ID starting with be27b322c402d8ad692b17d99dc49685c0bfdebe8ced93525f22aec1e4c6c5b2 not found: ID does not exist" containerID="be27b322c402d8ad692b17d99dc49685c0bfdebe8ced93525f22aec1e4c6c5b2"
Jul 16 21:02:29.277730 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:29.277674    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be27b322c402d8ad692b17d99dc49685c0bfdebe8ced93525f22aec1e4c6c5b2"} err="failed to get container status \"be27b322c402d8ad692b17d99dc49685c0bfdebe8ced93525f22aec1e4c6c5b2\": rpc error: code = NotFound desc = could not find container \"be27b322c402d8ad692b17d99dc49685c0bfdebe8ced93525f22aec1e4c6c5b2\": container with ID starting with be27b322c402d8ad692b17d99dc49685c0bfdebe8ced93525f22aec1e4c6c5b2 not found: ID does not exist"
Jul 16 21:02:29.277730 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:29.277698    2576 scope.go:117] "RemoveContainer" containerID="be19e67652c1d0e9260b581c97ce13eeb93e1ef5f2e40952dc5ddf6fb333cb0a"
Jul 16 21:02:29.277962 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:02:29.277945    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be19e67652c1d0e9260b581c97ce13eeb93e1ef5f2e40952dc5ddf6fb333cb0a\": container with ID starting with be19e67652c1d0e9260b581c97ce13eeb93e1ef5f2e40952dc5ddf6fb333cb0a not found: ID does not exist" containerID="be19e67652c1d0e9260b581c97ce13eeb93e1ef5f2e40952dc5ddf6fb333cb0a"
Jul 16 21:02:29.278002 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:29.277972    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be19e67652c1d0e9260b581c97ce13eeb93e1ef5f2e40952dc5ddf6fb333cb0a"} err="failed to get container status \"be19e67652c1d0e9260b581c97ce13eeb93e1ef5f2e40952dc5ddf6fb333cb0a\": rpc error: code = NotFound desc = could not find container \"be19e67652c1d0e9260b581c97ce13eeb93e1ef5f2e40952dc5ddf6fb333cb0a\": container with ID starting with be19e67652c1d0e9260b581c97ce13eeb93e1ef5f2e40952dc5ddf6fb333cb0a not found: ID does not exist"
Jul 16 21:02:29.282696 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:29.282674    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-23a5a19c-openshift-default-65699459f7-ck4wh"]
Jul 16 21:02:30.689546 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:30.689515    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc6be4ea-a4c9-43a8-9474-e47385aa0069" path="/var/lib/kubelet/pods/bc6be4ea-a4c9-43a8-9474-e47385aa0069/volumes"
Jul 16 21:02:34.575478 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:34.575448    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 21:02:34.580202 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:34.580179    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 21:02:35.523828 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.523800    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"]
Jul 16 21:02:35.524179 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.524164    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71d2cccc-202a-4177-b416-bbc58fa05427" containerName="cleanup"
Jul 16 21:02:35.524179 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.524179    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="71d2cccc-202a-4177-b416-bbc58fa05427" containerName="cleanup"
Jul 16 21:02:35.524343 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.524190    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc6be4ea-a4c9-43a8-9474-e47385aa0069" containerName="istio-proxy"
Jul 16 21:02:35.524343 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.524196    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc6be4ea-a4c9-43a8-9474-e47385aa0069" containerName="istio-proxy"
Jul 16 21:02:35.524343 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.524271    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc6be4ea-a4c9-43a8-9474-e47385aa0069" containerName="istio-proxy"
Jul 16 21:02:35.524343 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.524282    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc6be4ea-a4c9-43a8-9474-e47385aa0069" containerName="istio-proxy"
Jul 16 21:02:35.524343 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.524289    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="71d2cccc-202a-4177-b416-bbc58fa05427" containerName="cleanup"
Jul 16 21:02:35.528580 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.528560    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.531165 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.531143    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-ad09ff70-gw-openshift-default-dockercfg-c5bxb\""
Jul 16 21:02:35.541448 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.541428    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"]
Jul 16 21:02:35.669694 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.669667    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-token\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.670029 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.669713    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-data\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.670029 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.669734    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-podinfo\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.670029 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.669794    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-envoy\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.670029 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.669826    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt8pn\" (UniqueName: \"kubernetes.io/projected/5df78fc2-516d-4dee-a343-f5bd2279610e-kube-api-access-dt8pn\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.670029 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.669846    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-workload-socket\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.670029 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.669881    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-workload-certs\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.670029 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.669926    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5df78fc2-516d-4dee-a343-f5bd2279610e-istiod-ca-cert\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.670029 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.669974    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-credential-socket\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.771045 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.770982    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-data\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.771045 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.771012    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-podinfo\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.771045 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.771033    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-envoy\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.771284 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.771052    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt8pn\" (UniqueName: \"kubernetes.io/projected/5df78fc2-516d-4dee-a343-f5bd2279610e-kube-api-access-dt8pn\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.771284 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.771102    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-workload-socket\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.771284 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.771141    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-workload-certs\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.771478 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.771294    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5df78fc2-516d-4dee-a343-f5bd2279610e-istiod-ca-cert\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.771478 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.771378    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-credential-socket\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.771478 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.771424    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-token\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.771478 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.771377    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-data\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.771688 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.771483    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-workload-socket\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.771744 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.771717    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-workload-certs\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.771802 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.771781    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-credential-socket\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.771959 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.771939    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5df78fc2-516d-4dee-a343-f5bd2279610e-istiod-ca-cert\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.773421 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.773392    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-envoy\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.773510 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.773487    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-podinfo\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.781675 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.781654    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt8pn\" (UniqueName: \"kubernetes.io/projected/5df78fc2-516d-4dee-a343-f5bd2279610e-kube-api-access-dt8pn\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.782343 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.782326    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-token\") pod \"e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") " pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.838917 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.838897    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:35.981120 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.981098    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"]
Jul 16 21:02:35.982831 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:02:35.982804    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5df78fc2_516d_4dee_a343_f5bd2279610e.slice/crio-1b3946f4932bf3fc73032fa624df61c71a068f9d52acec86075654cb19c106a4 WatchSource:0}: Error finding container 1b3946f4932bf3fc73032fa624df61c71a068f9d52acec86075654cb19c106a4: Status 404 returned error can't find the container with id 1b3946f4932bf3fc73032fa624df61c71a068f9d52acec86075654cb19c106a4
Jul 16 21:02:35.984831 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.984799    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:02:35.984911 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.984859    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:02:35.984911 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:35.984885    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:02:36.296493 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:36.296407    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh" event={"ID":"5df78fc2-516d-4dee-a343-f5bd2279610e","Type":"ContainerStarted","Data":"7899f1acda8d808b43245e7fb9a6f4f063e79ed9135ed3e77ac9e17ad2471437"}
Jul 16 21:02:36.296493 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:36.296468    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh" event={"ID":"5df78fc2-516d-4dee-a343-f5bd2279610e","Type":"ContainerStarted","Data":"1b3946f4932bf3fc73032fa624df61c71a068f9d52acec86075654cb19c106a4"}
Jul 16 21:02:36.317844 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:36.317798    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh" podStartSLOduration=1.3177823960000001 podStartE2EDuration="1.317782396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:02:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:02:36.315442244 +0000 UTC m=+1802.281376477" watchObservedRunningTime="2026-07-16 21:02:36.317782396 +0000 UTC m=+1802.283716628"
Jul 16 21:02:36.839457 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:36.839430    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:37.844634 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:37.844607    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:38.307473 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:38.307441    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:38.308410 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:38.308392    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:39.987625 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:39.987587    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:02:48.364552 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:48.364490    2576 generic.go:356] "Generic (PLEG): container finished" podID="5df78fc2-516d-4dee-a343-f5bd2279610e" containerID="7899f1acda8d808b43245e7fb9a6f4f063e79ed9135ed3e77ac9e17ad2471437" exitCode=0
Jul 16 21:02:48.364936 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:48.364564    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh" event={"ID":"5df78fc2-516d-4dee-a343-f5bd2279610e","Type":"ContainerDied","Data":"7899f1acda8d808b43245e7fb9a6f4f063e79ed9135ed3e77ac9e17ad2471437"}
Jul 16 21:02:48.364936 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:48.364915    2576 scope.go:117] "RemoveContainer" containerID="7899f1acda8d808b43245e7fb9a6f4f063e79ed9135ed3e77ac9e17ad2471437"
Jul 16 21:02:48.365914 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:48.365887    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:02:48.365974 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:48.365958    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:02:48.366018 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:48.365990    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:02:49.371777 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:49.371747    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh" event={"ID":"5df78fc2-516d-4dee-a343-f5bd2279610e","Type":"ContainerStarted","Data":"4a97093364db5f51f22e5f9e06061104ba8e7b2eef88f145713e88ca559ea7fb"}
Jul 16 21:02:49.839573 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:49.839536    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:49.840659 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:49.840632    2576 patch_prober.go:28] interesting pod/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Jul 16 21:02:49.840907 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:49.840862    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh" podUID="5df78fc2-516d-4dee-a343-f5bd2279610e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Jul 16 21:02:50.839970 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:50.839932    2576 patch_prober.go:28] interesting pod/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Jul 16 21:02:50.840496 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:50.839998    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh" podUID="5df78fc2-516d-4dee-a343-f5bd2279610e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Jul 16 21:02:51.839839 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:51.839795    2576 patch_prober.go:28] interesting pod/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Jul 16 21:02:51.840030 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:51.839864    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh" podUID="5df78fc2-516d-4dee-a343-f5bd2279610e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Jul 16 21:02:52.840568 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:52.840106    2576 patch_prober.go:28] interesting pod/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body=
Jul 16 21:02:52.840568 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:52.840193    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh" podUID="5df78fc2-516d-4dee-a343-f5bd2279610e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused"
Jul 16 21:02:53.308015 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:53.307975    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:53.844163 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:53.844135    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:02:53.845387 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:02:53.845333    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:03:23.270096 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:23.270064    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg"]
Jul 16 21:03:23.270551 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:23.270442    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc6be4ea-a4c9-43a8-9474-e47385aa0069" containerName="istio-proxy"
Jul 16 21:03:23.270551 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:23.270454    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc6be4ea-a4c9-43a8-9474-e47385aa0069" containerName="istio-proxy"
Jul 16 21:03:23.273100 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:23.273081    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg"
Jul 16 21:03:23.276388 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:23.276344    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:03:23.276388 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:23.276381    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:03:23.282271 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:23.282247    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg"]
Jul 16 21:03:23.389354 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:23.389330    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1579f07-e7e5-418f-8aba-d4f7df036d4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg\" (UID: \"d1579f07-e7e5-418f-8aba-d4f7df036d4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg"
Jul 16 21:03:23.490348 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:23.490323    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1579f07-e7e5-418f-8aba-d4f7df036d4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg\" (UID: \"d1579f07-e7e5-418f-8aba-d4f7df036d4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg"
Jul 16 21:03:23.490913 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:23.490894    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1579f07-e7e5-418f-8aba-d4f7df036d4e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg\" (UID: \"d1579f07-e7e5-418f-8aba-d4f7df036d4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg"
Jul 16 21:03:23.584586 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:23.584511    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg"
Jul 16 21:03:23.719199 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:23.719178    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg"]
Jul 16 21:03:23.720946 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:03:23.720922    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1579f07_e7e5_418f_8aba_d4f7df036d4e.slice/crio-bb3cfd32143d5bfc837285e79b2db57f0024f3f0376b8f63f10808c07bde9906 WatchSource:0}: Error finding container bb3cfd32143d5bfc837285e79b2db57f0024f3f0376b8f63f10808c07bde9906: Status 404 returned error can't find the container with id bb3cfd32143d5bfc837285e79b2db57f0024f3f0376b8f63f10808c07bde9906
Jul 16 21:03:24.546570 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:24.546537    2576 generic.go:356] "Generic (PLEG): container finished" podID="d1579f07-e7e5-418f-8aba-d4f7df036d4e" containerID="ee8b2bc2cf4144f1c7b03f51acef31a27af42d44ea4bbb1976392acf90c51c19" exitCode=6
Jul 16 21:03:24.547010 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:24.546593    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg" event={"ID":"d1579f07-e7e5-418f-8aba-d4f7df036d4e","Type":"ContainerDied","Data":"ee8b2bc2cf4144f1c7b03f51acef31a27af42d44ea4bbb1976392acf90c51c19"}
Jul 16 21:03:24.547010 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:24.546645    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg" event={"ID":"d1579f07-e7e5-418f-8aba-d4f7df036d4e","Type":"ContainerStarted","Data":"bb3cfd32143d5bfc837285e79b2db57f0024f3f0376b8f63f10808c07bde9906"}
Jul 16 21:03:24.547010 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:24.546858    2576 scope.go:117] "RemoveContainer" containerID="ee8b2bc2cf4144f1c7b03f51acef31a27af42d44ea4bbb1976392acf90c51c19"
Jul 16 21:03:25.558471 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:25.558401    2576 generic.go:356] "Generic (PLEG): container finished" podID="d1579f07-e7e5-418f-8aba-d4f7df036d4e" containerID="9b911a1c8858e49960dd8b44812bcc5138bead855acc6090042a17e6cc7aa83d" exitCode=6
Jul 16 21:03:25.558471 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:25.558458    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg" event={"ID":"d1579f07-e7e5-418f-8aba-d4f7df036d4e","Type":"ContainerDied","Data":"9b911a1c8858e49960dd8b44812bcc5138bead855acc6090042a17e6cc7aa83d"}
Jul 16 21:03:25.558912 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:25.558518    2576 scope.go:117] "RemoveContainer" containerID="ee8b2bc2cf4144f1c7b03f51acef31a27af42d44ea4bbb1976392acf90c51c19"
Jul 16 21:03:25.558912 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:25.558653    2576 scope.go:117] "RemoveContainer" containerID="9b911a1c8858e49960dd8b44812bcc5138bead855acc6090042a17e6cc7aa83d"
Jul 16 21:03:25.558912 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:03:25.558868    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg_odh-ai-gateway-infra(d1579f07-e7e5-418f-8aba-d4f7df036d4e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg" podUID="d1579f07-e7e5-418f-8aba-d4f7df036d4e"
Jul 16 21:03:26.565341 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:26.565312    2576 scope.go:117] "RemoveContainer" containerID="9b911a1c8858e49960dd8b44812bcc5138bead855acc6090042a17e6cc7aa83d"
Jul 16 21:03:26.565743 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:03:26.565520    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg_odh-ai-gateway-infra(d1579f07-e7e5-418f-8aba-d4f7df036d4e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg" podUID="d1579f07-e7e5-418f-8aba-d4f7df036d4e"
Jul 16 21:03:38.684338 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:38.684301    2576 scope.go:117] "RemoveContainer" containerID="9b911a1c8858e49960dd8b44812bcc5138bead855acc6090042a17e6cc7aa83d"
Jul 16 21:03:39.637417 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:39.637313    2576 generic.go:356] "Generic (PLEG): container finished" podID="d1579f07-e7e5-418f-8aba-d4f7df036d4e" containerID="691c49b76b3fbb20294c689759df4faa573cbf504cb8f08ec7279994b28831e0" exitCode=6
Jul 16 21:03:39.637417 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:39.637385    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg" event={"ID":"d1579f07-e7e5-418f-8aba-d4f7df036d4e","Type":"ContainerDied","Data":"691c49b76b3fbb20294c689759df4faa573cbf504cb8f08ec7279994b28831e0"}
Jul 16 21:03:39.637622 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:39.637447    2576 scope.go:117] "RemoveContainer" containerID="9b911a1c8858e49960dd8b44812bcc5138bead855acc6090042a17e6cc7aa83d"
Jul 16 21:03:39.637770 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:39.637744    2576 scope.go:117] "RemoveContainer" containerID="691c49b76b3fbb20294c689759df4faa573cbf504cb8f08ec7279994b28831e0"
Jul 16 21:03:39.637991 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:03:39.637965    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg_odh-ai-gateway-infra(d1579f07-e7e5-418f-8aba-d4f7df036d4e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg" podUID="d1579f07-e7e5-418f-8aba-d4f7df036d4e"
Jul 16 21:03:40.669550 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:40.669518    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg"]
Jul 16 21:03:40.803595 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:40.803575    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg"
Jul 16 21:03:40.820215 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:40.820192    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1579f07-e7e5-418f-8aba-d4f7df036d4e-maas-api-service-ca\") pod \"d1579f07-e7e5-418f-8aba-d4f7df036d4e\" (UID: \"d1579f07-e7e5-418f-8aba-d4f7df036d4e\") "
Jul 16 21:03:40.820521 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:40.820499    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1579f07-e7e5-418f-8aba-d4f7df036d4e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d1579f07-e7e5-418f-8aba-d4f7df036d4e" (UID: "d1579f07-e7e5-418f-8aba-d4f7df036d4e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:03:40.921291 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:40.921228    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1579f07-e7e5-418f-8aba-d4f7df036d4e-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:03:41.461692 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:41.461658    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"]
Jul 16 21:03:41.461869 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:41.461835    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh" podUID="5df78fc2-516d-4dee-a343-f5bd2279610e" containerName="istio-proxy" containerID="cri-o://4a97093364db5f51f22e5f9e06061104ba8e7b2eef88f145713e88ca559ea7fb" gracePeriod=30
Jul 16 21:03:41.649290 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:41.649262    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg"
Jul 16 21:03:41.649482 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:41.649301    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg" event={"ID":"d1579f07-e7e5-418f-8aba-d4f7df036d4e","Type":"ContainerDied","Data":"bb3cfd32143d5bfc837285e79b2db57f0024f3f0376b8f63f10808c07bde9906"}
Jul 16 21:03:41.649482 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:41.649386    2576 scope.go:117] "RemoveContainer" containerID="691c49b76b3fbb20294c689759df4faa573cbf504cb8f08ec7279994b28831e0"
Jul 16 21:03:41.675425 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:41.675403    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg"]
Jul 16 21:03:41.678903 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:41.678882    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-6jvrg"]
Jul 16 21:03:42.689641 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:42.689610    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1579f07-e7e5-418f-8aba-d4f7df036d4e" path="/var/lib/kubelet/pods/d1579f07-e7e5-418f-8aba-d4f7df036d4e/volumes"
Jul 16 21:03:44.488211 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:44.488180    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:03:46.678242 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.678210    2576 generic.go:356] "Generic (PLEG): container finished" podID="5df78fc2-516d-4dee-a343-f5bd2279610e" containerID="4a97093364db5f51f22e5f9e06061104ba8e7b2eef88f145713e88ca559ea7fb" exitCode=0
Jul 16 21:03:46.678619 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.678241    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh" event={"ID":"5df78fc2-516d-4dee-a343-f5bd2279610e","Type":"ContainerDied","Data":"4a97093364db5f51f22e5f9e06061104ba8e7b2eef88f145713e88ca559ea7fb"}
Jul 16 21:03:46.678619 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.678304    2576 scope.go:117] "RemoveContainer" containerID="7899f1acda8d808b43245e7fb9a6f4f063e79ed9135ed3e77ac9e17ad2471437"
Jul 16 21:03:46.714422 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.714402    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:03:46.771076 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.771051    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-credential-socket\") pod \"5df78fc2-516d-4dee-a343-f5bd2279610e\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") "
Jul 16 21:03:46.771197 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.771095    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-workload-socket\") pod \"5df78fc2-516d-4dee-a343-f5bd2279610e\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") "
Jul 16 21:03:46.771197 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.771123    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-envoy\") pod \"5df78fc2-516d-4dee-a343-f5bd2279610e\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") "
Jul 16 21:03:46.771197 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.771176    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dt8pn\" (UniqueName: \"kubernetes.io/projected/5df78fc2-516d-4dee-a343-f5bd2279610e-kube-api-access-dt8pn\") pod \"5df78fc2-516d-4dee-a343-f5bd2279610e\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") "
Jul 16 21:03:46.771401 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.771221    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-data\") pod \"5df78fc2-516d-4dee-a343-f5bd2279610e\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") "
Jul 16 21:03:46.771401 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.771241    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-workload-certs\") pod \"5df78fc2-516d-4dee-a343-f5bd2279610e\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") "
Jul 16 21:03:46.771401 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.771292    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5df78fc2-516d-4dee-a343-f5bd2279610e-istiod-ca-cert\") pod \"5df78fc2-516d-4dee-a343-f5bd2279610e\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") "
Jul 16 21:03:46.771401 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.771318    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5df78fc2-516d-4dee-a343-f5bd2279610e" (UID: "5df78fc2-516d-4dee-a343-f5bd2279610e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:03:46.771401 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.771346    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-token\") pod \"5df78fc2-516d-4dee-a343-f5bd2279610e\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") "
Jul 16 21:03:46.771681 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.771409    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-podinfo\") pod \"5df78fc2-516d-4dee-a343-f5bd2279610e\" (UID: \"5df78fc2-516d-4dee-a343-f5bd2279610e\") "
Jul 16 21:03:46.771681 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.771412    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5df78fc2-516d-4dee-a343-f5bd2279610e" (UID: "5df78fc2-516d-4dee-a343-f5bd2279610e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:03:46.771681 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.771654    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-credential-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:03:46.771681 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.771672    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-workload-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:03:46.772203 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.772174    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-data" (OuterVolumeSpecName: "istio-data") pod "5df78fc2-516d-4dee-a343-f5bd2279610e" (UID: "5df78fc2-516d-4dee-a343-f5bd2279610e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:03:46.772303 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.772259    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5df78fc2-516d-4dee-a343-f5bd2279610e" (UID: "5df78fc2-516d-4dee-a343-f5bd2279610e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:03:46.772303 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.772270    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5df78fc2-516d-4dee-a343-f5bd2279610e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5df78fc2-516d-4dee-a343-f5bd2279610e" (UID: "5df78fc2-516d-4dee-a343-f5bd2279610e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:03:46.774302 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.774268    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5df78fc2-516d-4dee-a343-f5bd2279610e" (UID: "5df78fc2-516d-4dee-a343-f5bd2279610e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:03:46.774429 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.774316    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5df78fc2-516d-4dee-a343-f5bd2279610e" (UID: "5df78fc2-516d-4dee-a343-f5bd2279610e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 21:03:46.774491 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.774422    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5df78fc2-516d-4dee-a343-f5bd2279610e-kube-api-access-dt8pn" (OuterVolumeSpecName: "kube-api-access-dt8pn") pod "5df78fc2-516d-4dee-a343-f5bd2279610e" (UID: "5df78fc2-516d-4dee-a343-f5bd2279610e"). InnerVolumeSpecName "kube-api-access-dt8pn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:03:46.774557 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.774525    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-token" (OuterVolumeSpecName: "istio-token") pod "5df78fc2-516d-4dee-a343-f5bd2279610e" (UID: "5df78fc2-516d-4dee-a343-f5bd2279610e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:03:46.873030 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.873008    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-data\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:03:46.873030 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.873030    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-workload-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:03:46.873173 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.873039    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5df78fc2-516d-4dee-a343-f5bd2279610e-istiod-ca-cert\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:03:46.873173 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.873047    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-token\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:03:46.873173 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.873055    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-podinfo\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:03:46.873173 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.873062    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5df78fc2-516d-4dee-a343-f5bd2279610e-istio-envoy\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:03:46.873173 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:46.873070    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt8pn\" (UniqueName: \"kubernetes.io/projected/5df78fc2-516d-4dee-a343-f5bd2279610e-kube-api-access-dt8pn\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:03:47.684372 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:47.684331    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"
Jul 16 21:03:47.684781 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:47.684394    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh" event={"ID":"5df78fc2-516d-4dee-a343-f5bd2279610e","Type":"ContainerDied","Data":"1b3946f4932bf3fc73032fa624df61c71a068f9d52acec86075654cb19c106a4"}
Jul 16 21:03:47.684781 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:47.684447    2576 scope.go:117] "RemoveContainer" containerID="4a97093364db5f51f22e5f9e06061104ba8e7b2eef88f145713e88ca559ea7fb"
Jul 16 21:03:47.705800 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:47.705777    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"]
Jul 16 21:03:47.710373 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:47.710341    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-ad09ff70-gw-openshift-default-6567b48dd7-hntxh"]
Jul 16 21:03:48.690119 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:48.690090    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5df78fc2-516d-4dee-a343-f5bd2279610e" path="/var/lib/kubelet/pods/5df78fc2-516d-4dee-a343-f5bd2279610e/volumes"
Jul 16 21:03:54.379013 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.378977    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"]
Jul 16 21:03:54.379407 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.379321    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5df78fc2-516d-4dee-a343-f5bd2279610e" containerName="istio-proxy"
Jul 16 21:03:54.379407 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.379333    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5df78fc2-516d-4dee-a343-f5bd2279610e" containerName="istio-proxy"
Jul 16 21:03:54.379407 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.379343    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1579f07-e7e5-418f-8aba-d4f7df036d4e" containerName="revoke-keys"
Jul 16 21:03:54.379407 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.379348    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1579f07-e7e5-418f-8aba-d4f7df036d4e" containerName="revoke-keys"
Jul 16 21:03:54.379407 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.379371    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1579f07-e7e5-418f-8aba-d4f7df036d4e" containerName="revoke-keys"
Jul 16 21:03:54.379407 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.379377    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1579f07-e7e5-418f-8aba-d4f7df036d4e" containerName="revoke-keys"
Jul 16 21:03:54.379605 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.379447    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5df78fc2-516d-4dee-a343-f5bd2279610e" containerName="istio-proxy"
Jul 16 21:03:54.379605 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.379456    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1579f07-e7e5-418f-8aba-d4f7df036d4e" containerName="revoke-keys"
Jul 16 21:03:54.379605 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.379464    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1579f07-e7e5-418f-8aba-d4f7df036d4e" containerName="revoke-keys"
Jul 16 21:03:54.384022 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.384001    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.387455 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.387438    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e65739ab-openshift-default-dockercfg-lbqmp\""
Jul 16 21:03:54.394994 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.394969    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"]
Jul 16 21:03:54.429984 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.429955    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-data\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.430092 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.429987    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-workload-socket\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.430092 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.430026    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-credential-socket\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.430092 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.430049    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-envoy\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.430092 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.430065    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istiod-ca-cert\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.430092 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.430086    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p92nv\" (UniqueName: \"kubernetes.io/projected/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-kube-api-access-p92nv\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.430298 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.430178    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-workload-certs\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.430298 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.430229    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-podinfo\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.430298 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.430257    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-token\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.530854 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.530825    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-credential-socket\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.530854 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.530858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-envoy\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.531107 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.530875    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istiod-ca-cert\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.531107 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.530897    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p92nv\" (UniqueName: \"kubernetes.io/projected/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-kube-api-access-p92nv\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.531107 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.530964    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-workload-certs\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.531107 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.530995    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-podinfo\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.531107 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.531020    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-token\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.531107 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.531067    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-data\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.531107 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.531090    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-workload-socket\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.531654 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.531603    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-data\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.531654 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.531626    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-workload-socket\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.531854 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.531661    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-workload-certs\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.531854 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.531671    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istiod-ca-cert\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.531854 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.531799    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-credential-socket\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.533296 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.533274    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-envoy\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.533593 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.533571    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-podinfo\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.539754 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.539726    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-token\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.541347 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.541326    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p92nv\" (UniqueName: \"kubernetes.io/projected/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-kube-api-access-p92nv\") pod \"e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") " pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.694682 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.694659    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:54.839115 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.839088    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"]
Jul 16 21:03:54.841104 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:03:54.841082    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21c8dab_0ff9_4bd9_8a0f_fb0d0f75d815.slice/crio-9e31ca370a0cf84595c55a1f88732a6167e111ecccee2b22e38beaadc917d6f4 WatchSource:0}: Error finding container 9e31ca370a0cf84595c55a1f88732a6167e111ecccee2b22e38beaadc917d6f4: Status 404 returned error can't find the container with id 9e31ca370a0cf84595c55a1f88732a6167e111ecccee2b22e38beaadc917d6f4
Jul 16 21:03:54.843380 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.843336    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:03:54.843465 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.843407    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:03:54.843465 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:54.843435    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:03:55.724908 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:55.724877    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47" event={"ID":"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815","Type":"ContainerStarted","Data":"63b687107976f082e439e3f37c51941ef5fc8198b7ee4823920b07575030e840"}
Jul 16 21:03:55.725286 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:55.724924    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47" event={"ID":"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815","Type":"ContainerStarted","Data":"9e31ca370a0cf84595c55a1f88732a6167e111ecccee2b22e38beaadc917d6f4"}
Jul 16 21:03:55.793135 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:55.793086    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47" podStartSLOduration=1.7930661479999999 podStartE2EDuration="1.793066148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:03:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:03:55.790969371 +0000 UTC m=+1881.756903605" watchObservedRunningTime="2026-07-16 21:03:55.793066148 +0000 UTC m=+1881.759000382"
Jul 16 21:03:56.694839 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:56.694739    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:56.699893 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:56.699869    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:56.729733 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:56.729702    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:03:56.730393 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:03:56.730375    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:04:04.199717 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:04.199687    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:04:12.772568 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:12.772523    2576 patch_prober.go:28] interesting pod/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 16 21:04:12.773010 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:12.772589    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47" podUID="d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 16 21:04:14.816298 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:14.816263    2576 generic.go:356] "Generic (PLEG): container finished" podID="d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" containerID="63b687107976f082e439e3f37c51941ef5fc8198b7ee4823920b07575030e840" exitCode=0
Jul 16 21:04:14.816725 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:14.816340    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47" event={"ID":"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815","Type":"ContainerDied","Data":"63b687107976f082e439e3f37c51941ef5fc8198b7ee4823920b07575030e840"}
Jul 16 21:04:14.816770 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:14.816735    2576 scope.go:117] "RemoveContainer" containerID="63b687107976f082e439e3f37c51941ef5fc8198b7ee4823920b07575030e840"
Jul 16 21:04:14.817848 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:14.817813    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:04:14.817940 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:14.817889    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:04:14.817940 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:14.817928    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:04:15.822769 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:15.822738    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47" event={"ID":"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815","Type":"ContainerStarted","Data":"ba4538e7adc525368c7cf4d1c9a3a44ff77ad0635f45da4a4013503f41062e18"}
Jul 16 21:04:16.695651 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:16.695617    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:04:16.695819 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:16.695781    2576 patch_prober.go:28] interesting pod/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 16 21:04:16.695864 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:16.695823    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47" podUID="d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 16 21:04:17.695690 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:17.695649    2576 patch_prober.go:28] interesting pod/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 16 21:04:17.696082 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:17.695713    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47" podUID="d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 16 21:04:18.695820 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:18.695785    2576 patch_prober.go:28] interesting pod/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 16 21:04:18.696280 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:18.695844    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47" podUID="d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 16 21:04:19.695728 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:19.695700    2576 patch_prober.go:28] interesting pod/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 16 21:04:19.695890 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:19.695758    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47" podUID="d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 16 21:04:20.699025 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:20.698996    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:04:20.699424 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:20.699342    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:04:20.700094 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:20.700074    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:04:21.459436 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:21.459402    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7"]
Jul 16 21:04:21.459811 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:21.459798    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5df78fc2-516d-4dee-a343-f5bd2279610e" containerName="istio-proxy"
Jul 16 21:04:21.459858 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:21.459813    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5df78fc2-516d-4dee-a343-f5bd2279610e" containerName="istio-proxy"
Jul 16 21:04:21.459858 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:21.459831    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1579f07-e7e5-418f-8aba-d4f7df036d4e" containerName="revoke-keys"
Jul 16 21:04:21.459858 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:21.459838    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1579f07-e7e5-418f-8aba-d4f7df036d4e" containerName="revoke-keys"
Jul 16 21:04:21.459950 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:21.459893    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5df78fc2-516d-4dee-a343-f5bd2279610e" containerName="istio-proxy"
Jul 16 21:04:21.459950 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:21.459909    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1579f07-e7e5-418f-8aba-d4f7df036d4e" containerName="revoke-keys"
Jul 16 21:04:21.462207 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:21.462188    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7"
Jul 16 21:04:21.467668 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:21.467646    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:04:21.468851 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:21.468711    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:04:21.482682 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:21.482660    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7"]
Jul 16 21:04:21.562368 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:21.562343    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/856480d3-f137-4ea8-9f05-8f4a9db69714-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7\" (UID: \"856480d3-f137-4ea8-9f05-8f4a9db69714\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7"
Jul 16 21:04:21.662891 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:21.662869    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/856480d3-f137-4ea8-9f05-8f4a9db69714-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7\" (UID: \"856480d3-f137-4ea8-9f05-8f4a9db69714\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7"
Jul 16 21:04:21.663453 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:21.663434    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/856480d3-f137-4ea8-9f05-8f4a9db69714-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7\" (UID: \"856480d3-f137-4ea8-9f05-8f4a9db69714\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7"
Jul 16 21:04:21.772028 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:21.771969    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7"
Jul 16 21:04:21.927012 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:21.926978    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7"]
Jul 16 21:04:21.950529 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:04:21.950501    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod856480d3_f137_4ea8_9f05_8f4a9db69714.slice/crio-7f28c07c38c2b47549fe10d3624ebdc393a784e02ef3e7a2a5643fbd6ff81486 WatchSource:0}: Error finding container 7f28c07c38c2b47549fe10d3624ebdc393a784e02ef3e7a2a5643fbd6ff81486: Status 404 returned error can't find the container with id 7f28c07c38c2b47549fe10d3624ebdc393a784e02ef3e7a2a5643fbd6ff81486
Jul 16 21:04:22.865894 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:22.865812    2576 generic.go:356] "Generic (PLEG): container finished" podID="856480d3-f137-4ea8-9f05-8f4a9db69714" containerID="fe806ff874896a9c2b4e5a65121950c704c71aec1ca20afc1a1080ac723f48e3" exitCode=6
Jul 16 21:04:22.865894 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:22.865859    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7" event={"ID":"856480d3-f137-4ea8-9f05-8f4a9db69714","Type":"ContainerDied","Data":"fe806ff874896a9c2b4e5a65121950c704c71aec1ca20afc1a1080ac723f48e3"}
Jul 16 21:04:22.866272 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:22.865919    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7" event={"ID":"856480d3-f137-4ea8-9f05-8f4a9db69714","Type":"ContainerStarted","Data":"7f28c07c38c2b47549fe10d3624ebdc393a784e02ef3e7a2a5643fbd6ff81486"}
Jul 16 21:04:22.866272 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:22.866078    2576 scope.go:117] "RemoveContainer" containerID="fe806ff874896a9c2b4e5a65121950c704c71aec1ca20afc1a1080ac723f48e3"
Jul 16 21:04:23.873176 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:23.873104    2576 generic.go:356] "Generic (PLEG): container finished" podID="856480d3-f137-4ea8-9f05-8f4a9db69714" containerID="3e1416c4914097e2fabe1c0e210adff02638511f7800b3f4f5d41dedd7147c9f" exitCode=6
Jul 16 21:04:23.873630 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:23.873200    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7" event={"ID":"856480d3-f137-4ea8-9f05-8f4a9db69714","Type":"ContainerDied","Data":"3e1416c4914097e2fabe1c0e210adff02638511f7800b3f4f5d41dedd7147c9f"}
Jul 16 21:04:23.873630 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:23.873271    2576 scope.go:117] "RemoveContainer" containerID="fe806ff874896a9c2b4e5a65121950c704c71aec1ca20afc1a1080ac723f48e3"
Jul 16 21:04:23.873630 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:23.873495    2576 scope.go:117] "RemoveContainer" containerID="3e1416c4914097e2fabe1c0e210adff02638511f7800b3f4f5d41dedd7147c9f"
Jul 16 21:04:23.873794 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:04:23.873747    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7_odh-ai-gateway-infra(856480d3-f137-4ea8-9f05-8f4a9db69714)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7" podUID="856480d3-f137-4ea8-9f05-8f4a9db69714"
Jul 16 21:04:24.879574 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:24.879538    2576 scope.go:117] "RemoveContainer" containerID="3e1416c4914097e2fabe1c0e210adff02638511f7800b3f4f5d41dedd7147c9f"
Jul 16 21:04:24.880004 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:04:24.879721    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7_odh-ai-gateway-infra(856480d3-f137-4ea8-9f05-8f4a9db69714)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7" podUID="856480d3-f137-4ea8-9f05-8f4a9db69714"
Jul 16 21:04:36.685045 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:36.685002    2576 scope.go:117] "RemoveContainer" containerID="3e1416c4914097e2fabe1c0e210adff02638511f7800b3f4f5d41dedd7147c9f"
Jul 16 21:04:36.944334 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:36.944272    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7" event={"ID":"856480d3-f137-4ea8-9f05-8f4a9db69714","Type":"ContainerStarted","Data":"5acf3813cd6d20824e5f564dbdd875e912f1a6767bfd3048a6ad64c61f4edcc9"}
Jul 16 21:04:36.961732 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:36.961690    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7" podStartSLOduration=15.961678014 podStartE2EDuration="15.961678014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:04:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:04:36.960253102 +0000 UTC m=+1922.926187335" watchObservedRunningTime="2026-07-16 21:04:36.961678014 +0000 UTC m=+1922.927612247"
Jul 16 21:04:37.952833 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:37.952790    2576 generic.go:356] "Generic (PLEG): container finished" podID="856480d3-f137-4ea8-9f05-8f4a9db69714" containerID="5acf3813cd6d20824e5f564dbdd875e912f1a6767bfd3048a6ad64c61f4edcc9" exitCode=6
Jul 16 21:04:37.953333 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:37.952868    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7" event={"ID":"856480d3-f137-4ea8-9f05-8f4a9db69714","Type":"ContainerDied","Data":"5acf3813cd6d20824e5f564dbdd875e912f1a6767bfd3048a6ad64c61f4edcc9"}
Jul 16 21:04:37.953333 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:37.952939    2576 scope.go:117] "RemoveContainer" containerID="3e1416c4914097e2fabe1c0e210adff02638511f7800b3f4f5d41dedd7147c9f"
Jul 16 21:04:37.953333 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:37.953284    2576 scope.go:117] "RemoveContainer" containerID="5acf3813cd6d20824e5f564dbdd875e912f1a6767bfd3048a6ad64c61f4edcc9"
Jul 16 21:04:37.953603 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:04:37.953572    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7_odh-ai-gateway-infra(856480d3-f137-4ea8-9f05-8f4a9db69714)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7" podUID="856480d3-f137-4ea8-9f05-8f4a9db69714"
Jul 16 21:04:37.970571 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:37.970545    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7"]
Jul 16 21:04:39.093573 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:39.093550    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7"
Jul 16 21:04:39.198851 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:39.198827    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/856480d3-f137-4ea8-9f05-8f4a9db69714-maas-api-service-ca\") pod \"856480d3-f137-4ea8-9f05-8f4a9db69714\" (UID: \"856480d3-f137-4ea8-9f05-8f4a9db69714\") "
Jul 16 21:04:39.199136 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:39.199117    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/856480d3-f137-4ea8-9f05-8f4a9db69714-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "856480d3-f137-4ea8-9f05-8f4a9db69714" (UID: "856480d3-f137-4ea8-9f05-8f4a9db69714"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:04:39.299684 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:39.299634    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/856480d3-f137-4ea8-9f05-8f4a9db69714-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:04:39.972994 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:39.972965    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7"
Jul 16 21:04:39.973160 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:39.973004    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7" event={"ID":"856480d3-f137-4ea8-9f05-8f4a9db69714","Type":"ContainerDied","Data":"7f28c07c38c2b47549fe10d3624ebdc393a784e02ef3e7a2a5643fbd6ff81486"}
Jul 16 21:04:39.973160 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:39.973094    2576 scope.go:117] "RemoveContainer" containerID="5acf3813cd6d20824e5f564dbdd875e912f1a6767bfd3048a6ad64c61f4edcc9"
Jul 16 21:04:40.000942 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:40.000921    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7"]
Jul 16 21:04:40.007734 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:40.007710    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-nbbr7"]
Jul 16 21:04:40.689553 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:04:40.689530    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="856480d3-f137-4ea8-9f05-8f4a9db69714" path="/var/lib/kubelet/pods/856480d3-f137-4ea8-9f05-8f4a9db69714/volumes"
Jul 16 21:05:11.472014 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:11.471981    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6"]
Jul 16 21:05:11.472525 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:11.472343    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="856480d3-f137-4ea8-9f05-8f4a9db69714" containerName="revoke-keys"
Jul 16 21:05:11.472525 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:11.472354    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="856480d3-f137-4ea8-9f05-8f4a9db69714" containerName="revoke-keys"
Jul 16 21:05:11.472525 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:11.472394    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="856480d3-f137-4ea8-9f05-8f4a9db69714" containerName="revoke-keys"
Jul 16 21:05:11.472525 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:11.472400    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="856480d3-f137-4ea8-9f05-8f4a9db69714" containerName="revoke-keys"
Jul 16 21:05:11.472525 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:11.472451    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="856480d3-f137-4ea8-9f05-8f4a9db69714" containerName="revoke-keys"
Jul 16 21:05:11.472525 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:11.472460    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="856480d3-f137-4ea8-9f05-8f4a9db69714" containerName="revoke-keys"
Jul 16 21:05:11.474956 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:11.474939    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6"
Jul 16 21:05:11.477460 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:11.477422    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:05:11.477460 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:11.477452    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:05:11.483026 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:11.482715    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6"]
Jul 16 21:05:11.566204 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:11.566179    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/120bcb49-2227-43a9-bb40-580fc25b4c15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6\" (UID: \"120bcb49-2227-43a9-bb40-580fc25b4c15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6"
Jul 16 21:05:11.667285 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:11.667255    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/120bcb49-2227-43a9-bb40-580fc25b4c15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6\" (UID: \"120bcb49-2227-43a9-bb40-580fc25b4c15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6"
Jul 16 21:05:11.667875 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:11.667854    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/120bcb49-2227-43a9-bb40-580fc25b4c15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6\" (UID: \"120bcb49-2227-43a9-bb40-580fc25b4c15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6"
Jul 16 21:05:11.787512 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:11.787455    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6"
Jul 16 21:05:11.922825 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:11.922795    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6"]
Jul 16 21:05:11.926392 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:05:11.926368    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod120bcb49_2227_43a9_bb40_580fc25b4c15.slice/crio-f3b6d2a7987e4c0e70e8a529bcbcd84639514bb3e1356760ad4914a02a31be61 WatchSource:0}: Error finding container f3b6d2a7987e4c0e70e8a529bcbcd84639514bb3e1356760ad4914a02a31be61: Status 404 returned error can't find the container with id f3b6d2a7987e4c0e70e8a529bcbcd84639514bb3e1356760ad4914a02a31be61
Jul 16 21:05:12.128233 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:12.128164    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6" event={"ID":"120bcb49-2227-43a9-bb40-580fc25b4c15","Type":"ContainerStarted","Data":"f3b6d2a7987e4c0e70e8a529bcbcd84639514bb3e1356760ad4914a02a31be61"}
Jul 16 21:05:13.134633 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:13.134604    2576 generic.go:356] "Generic (PLEG): container finished" podID="120bcb49-2227-43a9-bb40-580fc25b4c15" containerID="3e9e3d8eab636d63d9664254f57763a8507b922f9984b1c7ac1469d19eba68d1" exitCode=6
Jul 16 21:05:13.134991 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:13.134674    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6" event={"ID":"120bcb49-2227-43a9-bb40-580fc25b4c15","Type":"ContainerDied","Data":"3e9e3d8eab636d63d9664254f57763a8507b922f9984b1c7ac1469d19eba68d1"}
Jul 16 21:05:13.134991 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:13.134841    2576 scope.go:117] "RemoveContainer" containerID="3e9e3d8eab636d63d9664254f57763a8507b922f9984b1c7ac1469d19eba68d1"
Jul 16 21:05:14.141975 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:14.141939    2576 generic.go:356] "Generic (PLEG): container finished" podID="120bcb49-2227-43a9-bb40-580fc25b4c15" containerID="0cfa991d9969116996b515f45b17ba2c9ccba65759dd7e162d8fec3541763294" exitCode=6
Jul 16 21:05:14.142422 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:14.142028    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6" event={"ID":"120bcb49-2227-43a9-bb40-580fc25b4c15","Type":"ContainerDied","Data":"0cfa991d9969116996b515f45b17ba2c9ccba65759dd7e162d8fec3541763294"}
Jul 16 21:05:14.142422 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:14.142098    2576 scope.go:117] "RemoveContainer" containerID="3e9e3d8eab636d63d9664254f57763a8507b922f9984b1c7ac1469d19eba68d1"
Jul 16 21:05:14.142422 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:14.142306    2576 scope.go:117] "RemoveContainer" containerID="0cfa991d9969116996b515f45b17ba2c9ccba65759dd7e162d8fec3541763294"
Jul 16 21:05:14.142600 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:05:14.142527    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6_odh-ai-gateway-infra(120bcb49-2227-43a9-bb40-580fc25b4c15)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6" podUID="120bcb49-2227-43a9-bb40-580fc25b4c15"
Jul 16 21:05:15.148813 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:15.148788    2576 scope.go:117] "RemoveContainer" containerID="0cfa991d9969116996b515f45b17ba2c9ccba65759dd7e162d8fec3541763294"
Jul 16 21:05:15.149194 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:05:15.148959    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6_odh-ai-gateway-infra(120bcb49-2227-43a9-bb40-580fc25b4c15)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6" podUID="120bcb49-2227-43a9-bb40-580fc25b4c15"
Jul 16 21:05:26.684020 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:26.683976    2576 scope.go:117] "RemoveContainer" containerID="0cfa991d9969116996b515f45b17ba2c9ccba65759dd7e162d8fec3541763294"
Jul 16 21:05:27.210781 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:27.210749    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6" event={"ID":"120bcb49-2227-43a9-bb40-580fc25b4c15","Type":"ContainerStarted","Data":"34dc752e9e7eaec1f0a008e86b8247f6f0bfd4b2ce624de122cab23bedf89190"}
Jul 16 21:05:27.226664 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:27.226617    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6" podStartSLOduration=16.226603574 podStartE2EDuration="16.226603574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:05:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:05:27.225290279 +0000 UTC m=+1973.191224514" watchObservedRunningTime="2026-07-16 21:05:27.226603574 +0000 UTC m=+1973.192537808"
Jul 16 21:05:28.217688 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:28.217654    2576 generic.go:356] "Generic (PLEG): container finished" podID="120bcb49-2227-43a9-bb40-580fc25b4c15" containerID="34dc752e9e7eaec1f0a008e86b8247f6f0bfd4b2ce624de122cab23bedf89190" exitCode=6
Jul 16 21:05:28.218105 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:28.217728    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6" event={"ID":"120bcb49-2227-43a9-bb40-580fc25b4c15","Type":"ContainerDied","Data":"34dc752e9e7eaec1f0a008e86b8247f6f0bfd4b2ce624de122cab23bedf89190"}
Jul 16 21:05:28.218105 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:28.217788    2576 scope.go:117] "RemoveContainer" containerID="0cfa991d9969116996b515f45b17ba2c9ccba65759dd7e162d8fec3541763294"
Jul 16 21:05:28.218105 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:28.218071    2576 scope.go:117] "RemoveContainer" containerID="34dc752e9e7eaec1f0a008e86b8247f6f0bfd4b2ce624de122cab23bedf89190"
Jul 16 21:05:28.218308 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:05:28.218284    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6_odh-ai-gateway-infra(120bcb49-2227-43a9-bb40-580fc25b4c15)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6" podUID="120bcb49-2227-43a9-bb40-580fc25b4c15"
Jul 16 21:05:28.246548 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:28.246522    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6"]
Jul 16 21:05:29.362112 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:29.362089    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6"
Jul 16 21:05:29.517416 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:29.517344    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/120bcb49-2227-43a9-bb40-580fc25b4c15-maas-api-service-ca\") pod \"120bcb49-2227-43a9-bb40-580fc25b4c15\" (UID: \"120bcb49-2227-43a9-bb40-580fc25b4c15\") "
Jul 16 21:05:29.517682 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:29.517656    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/120bcb49-2227-43a9-bb40-580fc25b4c15-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "120bcb49-2227-43a9-bb40-580fc25b4c15" (UID: "120bcb49-2227-43a9-bb40-580fc25b4c15"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:05:29.618110 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:29.618076    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/120bcb49-2227-43a9-bb40-580fc25b4c15-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:05:30.230249 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:30.230221    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6"
Jul 16 21:05:30.230249 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:30.230241    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6" event={"ID":"120bcb49-2227-43a9-bb40-580fc25b4c15","Type":"ContainerDied","Data":"f3b6d2a7987e4c0e70e8a529bcbcd84639514bb3e1356760ad4914a02a31be61"}
Jul 16 21:05:30.230454 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:30.230305    2576 scope.go:117] "RemoveContainer" containerID="34dc752e9e7eaec1f0a008e86b8247f6f0bfd4b2ce624de122cab23bedf89190"
Jul 16 21:05:30.256247 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:30.256213    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6"]
Jul 16 21:05:30.264383 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:30.264347    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-79ss6"]
Jul 16 21:05:30.689921 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:05:30.689887    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="120bcb49-2227-43a9-bb40-580fc25b4c15" path="/var/lib/kubelet/pods/120bcb49-2227-43a9-bb40-580fc25b4c15/volumes"
Jul 16 21:06:00.756608 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.756571    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr"]
Jul 16 21:06:00.757268 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.757135    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="120bcb49-2227-43a9-bb40-580fc25b4c15" containerName="revoke-keys"
Jul 16 21:06:00.757268 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.757158    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="120bcb49-2227-43a9-bb40-580fc25b4c15" containerName="revoke-keys"
Jul 16 21:06:00.757268 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.757195    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="120bcb49-2227-43a9-bb40-580fc25b4c15" containerName="revoke-keys"
Jul 16 21:06:00.757268 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.757205    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="120bcb49-2227-43a9-bb40-580fc25b4c15" containerName="revoke-keys"
Jul 16 21:06:00.757268 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.757224    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="120bcb49-2227-43a9-bb40-580fc25b4c15" containerName="revoke-keys"
Jul 16 21:06:00.757268 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.757233    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="120bcb49-2227-43a9-bb40-580fc25b4c15" containerName="revoke-keys"
Jul 16 21:06:00.757268 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.757245    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="856480d3-f137-4ea8-9f05-8f4a9db69714" containerName="revoke-keys"
Jul 16 21:06:00.757268 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.757254    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="856480d3-f137-4ea8-9f05-8f4a9db69714" containerName="revoke-keys"
Jul 16 21:06:00.757730 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.757352    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="856480d3-f137-4ea8-9f05-8f4a9db69714" containerName="revoke-keys"
Jul 16 21:06:00.757730 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.757391    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="120bcb49-2227-43a9-bb40-580fc25b4c15" containerName="revoke-keys"
Jul 16 21:06:00.757730 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.757402    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="120bcb49-2227-43a9-bb40-580fc25b4c15" containerName="revoke-keys"
Jul 16 21:06:00.760810 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.760784    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr"
Jul 16 21:06:00.763067 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.763038    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:06:00.763389 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.763353    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:06:00.767689 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.767662    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr"]
Jul 16 21:06:00.864629 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.864597    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b655b4c-1265-4972-97c5-29ad8f5058b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr\" (UID: \"1b655b4c-1265-4972-97c5-29ad8f5058b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr"
Jul 16 21:06:00.965120 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.965091    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b655b4c-1265-4972-97c5-29ad8f5058b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr\" (UID: \"1b655b4c-1265-4972-97c5-29ad8f5058b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr"
Jul 16 21:06:00.965707 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:00.965689    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b655b4c-1265-4972-97c5-29ad8f5058b7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr\" (UID: \"1b655b4c-1265-4972-97c5-29ad8f5058b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr"
Jul 16 21:06:01.073971 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:01.073918    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr"
Jul 16 21:06:01.201434 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:01.201404    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr"]
Jul 16 21:06:01.203484 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:06:01.203457    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b655b4c_1265_4972_97c5_29ad8f5058b7.slice/crio-fc936d62269638238d5909606657a7047ab38c1618b9edc90ef1af78f591a006 WatchSource:0}: Error finding container fc936d62269638238d5909606657a7047ab38c1618b9edc90ef1af78f591a006: Status 404 returned error can't find the container with id fc936d62269638238d5909606657a7047ab38c1618b9edc90ef1af78f591a006
Jul 16 21:06:01.391976 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:01.391944    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr" event={"ID":"1b655b4c-1265-4972-97c5-29ad8f5058b7","Type":"ContainerStarted","Data":"fc936d62269638238d5909606657a7047ab38c1618b9edc90ef1af78f591a006"}
Jul 16 21:06:02.397892 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:02.397858    2576 generic.go:356] "Generic (PLEG): container finished" podID="1b655b4c-1265-4972-97c5-29ad8f5058b7" containerID="7a2832c035dd2dffadcf85887d9b541061c6cc24533d71621d864a5b3b695247" exitCode=7
Jul 16 21:06:02.398277 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:02.397950    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr" event={"ID":"1b655b4c-1265-4972-97c5-29ad8f5058b7","Type":"ContainerDied","Data":"7a2832c035dd2dffadcf85887d9b541061c6cc24533d71621d864a5b3b695247"}
Jul 16 21:06:02.398277 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:02.398116    2576 scope.go:117] "RemoveContainer" containerID="7a2832c035dd2dffadcf85887d9b541061c6cc24533d71621d864a5b3b695247"
Jul 16 21:06:03.404781 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:03.404750    2576 generic.go:356] "Generic (PLEG): container finished" podID="1b655b4c-1265-4972-97c5-29ad8f5058b7" containerID="e26a3a1b9a56d64cc42435d159ca5dbd787018010707ac81f567609f0a8fd717" exitCode=7
Jul 16 21:06:03.405121 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:03.404801    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr" event={"ID":"1b655b4c-1265-4972-97c5-29ad8f5058b7","Type":"ContainerDied","Data":"e26a3a1b9a56d64cc42435d159ca5dbd787018010707ac81f567609f0a8fd717"}
Jul 16 21:06:03.405121 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:03.404861    2576 scope.go:117] "RemoveContainer" containerID="7a2832c035dd2dffadcf85887d9b541061c6cc24533d71621d864a5b3b695247"
Jul 16 21:06:03.405121 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:03.405023    2576 scope.go:117] "RemoveContainer" containerID="e26a3a1b9a56d64cc42435d159ca5dbd787018010707ac81f567609f0a8fd717"
Jul 16 21:06:03.405254 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:06:03.405200    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr_odh-ai-gateway-infra(1b655b4c-1265-4972-97c5-29ad8f5058b7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr" podUID="1b655b4c-1265-4972-97c5-29ad8f5058b7"
Jul 16 21:06:04.412264 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:04.412240    2576 scope.go:117] "RemoveContainer" containerID="e26a3a1b9a56d64cc42435d159ca5dbd787018010707ac81f567609f0a8fd717"
Jul 16 21:06:04.412652 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:06:04.412442    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr_odh-ai-gateway-infra(1b655b4c-1265-4972-97c5-29ad8f5058b7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr" podUID="1b655b4c-1265-4972-97c5-29ad8f5058b7"
Jul 16 21:06:16.683940 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:16.683909    2576 scope.go:117] "RemoveContainer" containerID="e26a3a1b9a56d64cc42435d159ca5dbd787018010707ac81f567609f0a8fd717"
Jul 16 21:06:17.479269 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:17.479237    2576 generic.go:356] "Generic (PLEG): container finished" podID="1b655b4c-1265-4972-97c5-29ad8f5058b7" containerID="34f7493b5470b2c3626617083843989f20fd507d87f39c588b0daede2b78bdba" exitCode=7
Jul 16 21:06:17.479452 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:17.479309    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr" event={"ID":"1b655b4c-1265-4972-97c5-29ad8f5058b7","Type":"ContainerDied","Data":"34f7493b5470b2c3626617083843989f20fd507d87f39c588b0daede2b78bdba"}
Jul 16 21:06:17.479452 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:17.479388    2576 scope.go:117] "RemoveContainer" containerID="e26a3a1b9a56d64cc42435d159ca5dbd787018010707ac81f567609f0a8fd717"
Jul 16 21:06:17.479661 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:17.479638    2576 scope.go:117] "RemoveContainer" containerID="34f7493b5470b2c3626617083843989f20fd507d87f39c588b0daede2b78bdba"
Jul 16 21:06:17.479899 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:06:17.479879    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr_odh-ai-gateway-infra(1b655b4c-1265-4972-97c5-29ad8f5058b7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr" podUID="1b655b4c-1265-4972-97c5-29ad8f5058b7"
Jul 16 21:06:18.507276 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:18.507238    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr"]
Jul 16 21:06:18.639879 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:18.639855    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr"
Jul 16 21:06:18.709620 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:18.709599    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b655b4c-1265-4972-97c5-29ad8f5058b7-maas-api-service-ca\") pod \"1b655b4c-1265-4972-97c5-29ad8f5058b7\" (UID: \"1b655b4c-1265-4972-97c5-29ad8f5058b7\") "
Jul 16 21:06:18.709928 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:18.709908    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b655b4c-1265-4972-97c5-29ad8f5058b7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1b655b4c-1265-4972-97c5-29ad8f5058b7" (UID: "1b655b4c-1265-4972-97c5-29ad8f5058b7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:06:18.810973 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:18.810909    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b655b4c-1265-4972-97c5-29ad8f5058b7-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:06:19.501234 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:19.501204    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr"
Jul 16 21:06:19.501234 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:19.501224    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr" event={"ID":"1b655b4c-1265-4972-97c5-29ad8f5058b7","Type":"ContainerDied","Data":"fc936d62269638238d5909606657a7047ab38c1618b9edc90ef1af78f591a006"}
Jul 16 21:06:19.501492 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:19.501312    2576 scope.go:117] "RemoveContainer" containerID="34f7493b5470b2c3626617083843989f20fd507d87f39c588b0daede2b78bdba"
Jul 16 21:06:19.526983 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:19.526957    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr"]
Jul 16 21:06:19.529673 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:19.529652    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-6mrbr"]
Jul 16 21:06:20.690314 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:20.690275    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b655b4c-1265-4972-97c5-29ad8f5058b7" path="/var/lib/kubelet/pods/1b655b4c-1265-4972-97c5-29ad8f5058b7/volumes"
Jul 16 21:06:50.804031 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:50.804000    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f"]
Jul 16 21:06:50.804463 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:50.804383    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b655b4c-1265-4972-97c5-29ad8f5058b7" containerName="revoke-keys"
Jul 16 21:06:50.804463 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:50.804399    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b655b4c-1265-4972-97c5-29ad8f5058b7" containerName="revoke-keys"
Jul 16 21:06:50.804463 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:50.804413    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b655b4c-1265-4972-97c5-29ad8f5058b7" containerName="revoke-keys"
Jul 16 21:06:50.804463 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:50.804419    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b655b4c-1265-4972-97c5-29ad8f5058b7" containerName="revoke-keys"
Jul 16 21:06:50.804463 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:50.804425    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b655b4c-1265-4972-97c5-29ad8f5058b7" containerName="revoke-keys"
Jul 16 21:06:50.804463 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:50.804430    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b655b4c-1265-4972-97c5-29ad8f5058b7" containerName="revoke-keys"
Jul 16 21:06:50.804653 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:50.804516    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b655b4c-1265-4972-97c5-29ad8f5058b7" containerName="revoke-keys"
Jul 16 21:06:50.804653 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:50.804524    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b655b4c-1265-4972-97c5-29ad8f5058b7" containerName="revoke-keys"
Jul 16 21:06:50.804653 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:50.804532    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="120bcb49-2227-43a9-bb40-580fc25b4c15" containerName="revoke-keys"
Jul 16 21:06:50.807649 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:50.807630    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f"
Jul 16 21:06:50.810020 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:50.809996    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:06:50.810020 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:50.810016    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:06:50.815580 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:50.815067    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f"]
Jul 16 21:06:50.960837 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:50.960807    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/231fa191-537f-4c5c-aee1-a147cdf2c946-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-fr45f\" (UID: \"231fa191-537f-4c5c-aee1-a147cdf2c946\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f"
Jul 16 21:06:51.061237 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:51.061152    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/231fa191-537f-4c5c-aee1-a147cdf2c946-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-fr45f\" (UID: \"231fa191-537f-4c5c-aee1-a147cdf2c946\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f"
Jul 16 21:06:51.061823 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:51.061799    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/231fa191-537f-4c5c-aee1-a147cdf2c946-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-fr45f\" (UID: \"231fa191-537f-4c5c-aee1-a147cdf2c946\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f"
Jul 16 21:06:51.119922 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:51.119899    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f"
Jul 16 21:06:51.248271 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:51.248249    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f"]
Jul 16 21:06:51.250086 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:06:51.250057    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod231fa191_537f_4c5c_aee1_a147cdf2c946.slice/crio-6a8de6d1dc3bb6094abbf3255249153c39289a353765e150f0d65c575cd5a267 WatchSource:0}: Error finding container 6a8de6d1dc3bb6094abbf3255249153c39289a353765e150f0d65c575cd5a267: Status 404 returned error can't find the container with id 6a8de6d1dc3bb6094abbf3255249153c39289a353765e150f0d65c575cd5a267
Jul 16 21:06:51.668724 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:51.668693    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f" event={"ID":"231fa191-537f-4c5c-aee1-a147cdf2c946","Type":"ContainerStarted","Data":"436156db9a0f03c867c8f6864132d661b3cc0dfff2137e45b86e73319567ec04"}
Jul 16 21:06:51.668968 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:51.668755    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f" event={"ID":"231fa191-537f-4c5c-aee1-a147cdf2c946","Type":"ContainerStarted","Data":"6a8de6d1dc3bb6094abbf3255249153c39289a353765e150f0d65c575cd5a267"}
Jul 16 21:06:51.690688 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:51.690599    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f" podStartSLOduration=1.6905810510000001 podStartE2EDuration="1.690581051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:06:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:06:51.690392879 +0000 UTC m=+2057.656327109" watchObservedRunningTime="2026-07-16 21:06:51.690581051 +0000 UTC m=+2057.656515287"
Jul 16 21:06:52.675309 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:52.675268    2576 generic.go:356] "Generic (PLEG): container finished" podID="231fa191-537f-4c5c-aee1-a147cdf2c946" containerID="436156db9a0f03c867c8f6864132d661b3cc0dfff2137e45b86e73319567ec04" exitCode=7
Jul 16 21:06:52.675733 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:52.675352    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f" event={"ID":"231fa191-537f-4c5c-aee1-a147cdf2c946","Type":"ContainerDied","Data":"436156db9a0f03c867c8f6864132d661b3cc0dfff2137e45b86e73319567ec04"}
Jul 16 21:06:52.675733 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:52.675590    2576 scope.go:117] "RemoveContainer" containerID="436156db9a0f03c867c8f6864132d661b3cc0dfff2137e45b86e73319567ec04"
Jul 16 21:06:53.682917 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:53.682888    2576 generic.go:356] "Generic (PLEG): container finished" podID="231fa191-537f-4c5c-aee1-a147cdf2c946" containerID="5f3cf001a1d9c924f4fbc994298ce5fa82e067936e64dfddd71f8b5692927e42" exitCode=7
Jul 16 21:06:53.683312 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:53.682979    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f" event={"ID":"231fa191-537f-4c5c-aee1-a147cdf2c946","Type":"ContainerDied","Data":"5f3cf001a1d9c924f4fbc994298ce5fa82e067936e64dfddd71f8b5692927e42"}
Jul 16 21:06:53.683312 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:53.683050    2576 scope.go:117] "RemoveContainer" containerID="436156db9a0f03c867c8f6864132d661b3cc0dfff2137e45b86e73319567ec04"
Jul 16 21:06:53.683312 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:53.683211    2576 scope.go:117] "RemoveContainer" containerID="5f3cf001a1d9c924f4fbc994298ce5fa82e067936e64dfddd71f8b5692927e42"
Jul 16 21:06:53.683506 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:06:53.683484    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-fr45f_odh-ai-gateway-infra(231fa191-537f-4c5c-aee1-a147cdf2c946)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f" podUID="231fa191-537f-4c5c-aee1-a147cdf2c946"
Jul 16 21:06:54.691353 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:06:54.691319    2576 scope.go:117] "RemoveContainer" containerID="5f3cf001a1d9c924f4fbc994298ce5fa82e067936e64dfddd71f8b5692927e42"
Jul 16 21:06:54.691738 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:06:54.691540    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-fr45f_odh-ai-gateway-infra(231fa191-537f-4c5c-aee1-a147cdf2c946)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f" podUID="231fa191-537f-4c5c-aee1-a147cdf2c946"
Jul 16 21:07:08.684743 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:08.684650    2576 scope.go:117] "RemoveContainer" containerID="5f3cf001a1d9c924f4fbc994298ce5fa82e067936e64dfddd71f8b5692927e42"
Jul 16 21:07:09.765591 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:09.765554    2576 generic.go:356] "Generic (PLEG): container finished" podID="231fa191-537f-4c5c-aee1-a147cdf2c946" containerID="22bf670c9efd3f4780d6f0ef3638ded34e23d306fe84eff44f41627b0c5434c8" exitCode=7
Jul 16 21:07:09.766014 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:09.765626    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f" event={"ID":"231fa191-537f-4c5c-aee1-a147cdf2c946","Type":"ContainerDied","Data":"22bf670c9efd3f4780d6f0ef3638ded34e23d306fe84eff44f41627b0c5434c8"}
Jul 16 21:07:09.766014 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:09.765696    2576 scope.go:117] "RemoveContainer" containerID="5f3cf001a1d9c924f4fbc994298ce5fa82e067936e64dfddd71f8b5692927e42"
Jul 16 21:07:09.766014 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:09.765955    2576 scope.go:117] "RemoveContainer" containerID="22bf670c9efd3f4780d6f0ef3638ded34e23d306fe84eff44f41627b0c5434c8"
Jul 16 21:07:09.766204 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:07:09.766180    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-fr45f_odh-ai-gateway-infra(231fa191-537f-4c5c-aee1-a147cdf2c946)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f" podUID="231fa191-537f-4c5c-aee1-a147cdf2c946"
Jul 16 21:07:10.790956 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:10.790927    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f"]
Jul 16 21:07:10.927240 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:10.927219    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f"
Jul 16 21:07:11.008839 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:11.008813    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/231fa191-537f-4c5c-aee1-a147cdf2c946-maas-api-service-ca\") pod \"231fa191-537f-4c5c-aee1-a147cdf2c946\" (UID: \"231fa191-537f-4c5c-aee1-a147cdf2c946\") "
Jul 16 21:07:11.009155 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:11.009132    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/231fa191-537f-4c5c-aee1-a147cdf2c946-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "231fa191-537f-4c5c-aee1-a147cdf2c946" (UID: "231fa191-537f-4c5c-aee1-a147cdf2c946"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:07:11.109872 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:11.109814    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/231fa191-537f-4c5c-aee1-a147cdf2c946-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:07:11.779108 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:11.779082    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f"
Jul 16 21:07:11.779270 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:11.779128    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f" event={"ID":"231fa191-537f-4c5c-aee1-a147cdf2c946","Type":"ContainerDied","Data":"6a8de6d1dc3bb6094abbf3255249153c39289a353765e150f0d65c575cd5a267"}
Jul 16 21:07:11.779270 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:11.779196    2576 scope.go:117] "RemoveContainer" containerID="22bf670c9efd3f4780d6f0ef3638ded34e23d306fe84eff44f41627b0c5434c8"
Jul 16 21:07:11.803647 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:11.803619    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f"]
Jul 16 21:07:11.806474 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:11.806451    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-fr45f"]
Jul 16 21:07:12.690626 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:12.690590    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="231fa191-537f-4c5c-aee1-a147cdf2c946" path="/var/lib/kubelet/pods/231fa191-537f-4c5c-aee1-a147cdf2c946/volumes"
Jul 16 21:07:34.605944 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:34.605907    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 21:07:34.611282 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:34.611253    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 21:07:43.522846 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.522807    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2"]
Jul 16 21:07:43.523245 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.523178    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="231fa191-537f-4c5c-aee1-a147cdf2c946" containerName="revoke-keys"
Jul 16 21:07:43.523245 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.523189    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="231fa191-537f-4c5c-aee1-a147cdf2c946" containerName="revoke-keys"
Jul 16 21:07:43.523245 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.523199    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="231fa191-537f-4c5c-aee1-a147cdf2c946" containerName="revoke-keys"
Jul 16 21:07:43.523245 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.523205    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="231fa191-537f-4c5c-aee1-a147cdf2c946" containerName="revoke-keys"
Jul 16 21:07:43.523419 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.523281    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b655b4c-1265-4972-97c5-29ad8f5058b7" containerName="revoke-keys"
Jul 16 21:07:43.523419 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.523291    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="231fa191-537f-4c5c-aee1-a147cdf2c946" containerName="revoke-keys"
Jul 16 21:07:43.523419 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.523298    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="231fa191-537f-4c5c-aee1-a147cdf2c946" containerName="revoke-keys"
Jul 16 21:07:43.523419 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.523303    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="231fa191-537f-4c5c-aee1-a147cdf2c946" containerName="revoke-keys"
Jul 16 21:07:43.527190 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.527158    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2"
Jul 16 21:07:43.530427 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.530395    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:07:43.530704 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.530677    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:07:43.534822 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.534786    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2"]
Jul 16 21:07:43.672569 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.672545    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6ef4019-bb79-4b31-ba15-ba6f6aad3c21-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2\" (UID: \"e6ef4019-bb79-4b31-ba15-ba6f6aad3c21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2"
Jul 16 21:07:43.773196 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.773136    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6ef4019-bb79-4b31-ba15-ba6f6aad3c21-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2\" (UID: \"e6ef4019-bb79-4b31-ba15-ba6f6aad3c21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2"
Jul 16 21:07:43.773751 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.773734    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6ef4019-bb79-4b31-ba15-ba6f6aad3c21-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2\" (UID: \"e6ef4019-bb79-4b31-ba15-ba6f6aad3c21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2"
Jul 16 21:07:43.842247 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.842226    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2"
Jul 16 21:07:43.974243 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:43.974207    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2"]
Jul 16 21:07:43.977100 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:07:43.977069    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6ef4019_bb79_4b31_ba15_ba6f6aad3c21.slice/crio-5af9ec58af342fbaf0d7d185b179f13e00b4fbede752d8d073b1279acdf0d9be WatchSource:0}: Error finding container 5af9ec58af342fbaf0d7d185b179f13e00b4fbede752d8d073b1279acdf0d9be: Status 404 returned error can't find the container with id 5af9ec58af342fbaf0d7d185b179f13e00b4fbede752d8d073b1279acdf0d9be
Jul 16 21:07:44.946441 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:44.946410    2576 generic.go:356] "Generic (PLEG): container finished" podID="e6ef4019-bb79-4b31-ba15-ba6f6aad3c21" containerID="86e9511a148a5cb5500ebdc01f4ea7a0e0899bb9b4a0b87059436e842e722a3b" exitCode=6
Jul 16 21:07:44.946907 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:44.946503    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2" event={"ID":"e6ef4019-bb79-4b31-ba15-ba6f6aad3c21","Type":"ContainerDied","Data":"86e9511a148a5cb5500ebdc01f4ea7a0e0899bb9b4a0b87059436e842e722a3b"}
Jul 16 21:07:44.946907 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:44.946564    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2" event={"ID":"e6ef4019-bb79-4b31-ba15-ba6f6aad3c21","Type":"ContainerStarted","Data":"5af9ec58af342fbaf0d7d185b179f13e00b4fbede752d8d073b1279acdf0d9be"}
Jul 16 21:07:44.946907 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:44.946663    2576 scope.go:117] "RemoveContainer" containerID="86e9511a148a5cb5500ebdc01f4ea7a0e0899bb9b4a0b87059436e842e722a3b"
Jul 16 21:07:45.955625 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:45.955596    2576 generic.go:356] "Generic (PLEG): container finished" podID="e6ef4019-bb79-4b31-ba15-ba6f6aad3c21" containerID="a7b8b893cd8f34fc7c4c23fdf957d1447d80341c63d06d2db55ee92c9b59ee97" exitCode=6
Jul 16 21:07:45.956003 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:45.955677    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2" event={"ID":"e6ef4019-bb79-4b31-ba15-ba6f6aad3c21","Type":"ContainerDied","Data":"a7b8b893cd8f34fc7c4c23fdf957d1447d80341c63d06d2db55ee92c9b59ee97"}
Jul 16 21:07:45.956003 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:45.955748    2576 scope.go:117] "RemoveContainer" containerID="86e9511a148a5cb5500ebdc01f4ea7a0e0899bb9b4a0b87059436e842e722a3b"
Jul 16 21:07:45.956003 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:45.955953    2576 scope.go:117] "RemoveContainer" containerID="a7b8b893cd8f34fc7c4c23fdf957d1447d80341c63d06d2db55ee92c9b59ee97"
Jul 16 21:07:45.956156 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:07:45.956133    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2_odh-ai-gateway-infra(e6ef4019-bb79-4b31-ba15-ba6f6aad3c21)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2" podUID="e6ef4019-bb79-4b31-ba15-ba6f6aad3c21"
Jul 16 21:07:46.962021 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:07:46.961993    2576 scope.go:117] "RemoveContainer" containerID="a7b8b893cd8f34fc7c4c23fdf957d1447d80341c63d06d2db55ee92c9b59ee97"
Jul 16 21:07:46.962499 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:07:46.962228    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2_odh-ai-gateway-infra(e6ef4019-bb79-4b31-ba15-ba6f6aad3c21)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2" podUID="e6ef4019-bb79-4b31-ba15-ba6f6aad3c21"
Jul 16 21:08:01.684344 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:01.684312    2576 scope.go:117] "RemoveContainer" containerID="a7b8b893cd8f34fc7c4c23fdf957d1447d80341c63d06d2db55ee92c9b59ee97"
Jul 16 21:08:02.040402 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:02.040348    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2" event={"ID":"e6ef4019-bb79-4b31-ba15-ba6f6aad3c21","Type":"ContainerStarted","Data":"05cd767a82299e9883d1560e8ba1c8219f183c30a85ff55985be023d06c7b52b"}
Jul 16 21:08:02.058301 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:02.058258    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2" podStartSLOduration=19.058237597 podStartE2EDuration="19.058237597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:07:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:08:02.057264608 +0000 UTC m=+2128.023198842" watchObservedRunningTime="2026-07-16 21:08:02.058237597 +0000 UTC m=+2128.024171831"
Jul 16 21:08:03.048498 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:03.048459    2576 generic.go:356] "Generic (PLEG): container finished" podID="e6ef4019-bb79-4b31-ba15-ba6f6aad3c21" containerID="05cd767a82299e9883d1560e8ba1c8219f183c30a85ff55985be023d06c7b52b" exitCode=6
Jul 16 21:08:03.048975 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:03.048522    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2" event={"ID":"e6ef4019-bb79-4b31-ba15-ba6f6aad3c21","Type":"ContainerDied","Data":"05cd767a82299e9883d1560e8ba1c8219f183c30a85ff55985be023d06c7b52b"}
Jul 16 21:08:03.048975 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:03.048588    2576 scope.go:117] "RemoveContainer" containerID="a7b8b893cd8f34fc7c4c23fdf957d1447d80341c63d06d2db55ee92c9b59ee97"
Jul 16 21:08:03.048975 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:03.048849    2576 scope.go:117] "RemoveContainer" containerID="05cd767a82299e9883d1560e8ba1c8219f183c30a85ff55985be023d06c7b52b"
Jul 16 21:08:03.049157 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:08:03.049074    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2_odh-ai-gateway-infra(e6ef4019-bb79-4b31-ba15-ba6f6aad3c21)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2" podUID="e6ef4019-bb79-4b31-ba15-ba6f6aad3c21"
Jul 16 21:08:03.070785 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:03.070758    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2"]
Jul 16 21:08:04.191860 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:04.191835    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2"
Jul 16 21:08:04.346451 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:04.346385    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6ef4019-bb79-4b31-ba15-ba6f6aad3c21-maas-api-service-ca\") pod \"e6ef4019-bb79-4b31-ba15-ba6f6aad3c21\" (UID: \"e6ef4019-bb79-4b31-ba15-ba6f6aad3c21\") "
Jul 16 21:08:04.346707 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:04.346684    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6ef4019-bb79-4b31-ba15-ba6f6aad3c21-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e6ef4019-bb79-4b31-ba15-ba6f6aad3c21" (UID: "e6ef4019-bb79-4b31-ba15-ba6f6aad3c21"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:08:04.447406 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:04.447382    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6ef4019-bb79-4b31-ba15-ba6f6aad3c21-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:08:05.060791 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:05.060728    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2" event={"ID":"e6ef4019-bb79-4b31-ba15-ba6f6aad3c21","Type":"ContainerDied","Data":"5af9ec58af342fbaf0d7d185b179f13e00b4fbede752d8d073b1279acdf0d9be"}
Jul 16 21:08:05.060791 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:05.060748    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2"
Jul 16 21:08:05.060791 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:05.060778    2576 scope.go:117] "RemoveContainer" containerID="05cd767a82299e9883d1560e8ba1c8219f183c30a85ff55985be023d06c7b52b"
Jul 16 21:08:05.081926 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:05.081902    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2"]
Jul 16 21:08:05.086634 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:05.086614    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-qw7t2"]
Jul 16 21:08:06.689798 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:06.689751    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6ef4019-bb79-4b31-ba15-ba6f6aad3c21" path="/var/lib/kubelet/pods/e6ef4019-bb79-4b31-ba15-ba6f6aad3c21/volumes"
Jul 16 21:08:41.708676 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.708641    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4"]
Jul 16 21:08:41.709077 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.708994    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="231fa191-537f-4c5c-aee1-a147cdf2c946" containerName="revoke-keys"
Jul 16 21:08:41.709077 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.709005    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="231fa191-537f-4c5c-aee1-a147cdf2c946" containerName="revoke-keys"
Jul 16 21:08:41.709077 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.709015    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6ef4019-bb79-4b31-ba15-ba6f6aad3c21" containerName="revoke-keys"
Jul 16 21:08:41.709077 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.709020    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6ef4019-bb79-4b31-ba15-ba6f6aad3c21" containerName="revoke-keys"
Jul 16 21:08:41.709077 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.709030    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6ef4019-bb79-4b31-ba15-ba6f6aad3c21" containerName="revoke-keys"
Jul 16 21:08:41.709077 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.709036    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6ef4019-bb79-4b31-ba15-ba6f6aad3c21" containerName="revoke-keys"
Jul 16 21:08:41.709077 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.709043    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6ef4019-bb79-4b31-ba15-ba6f6aad3c21" containerName="revoke-keys"
Jul 16 21:08:41.709077 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.709049    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6ef4019-bb79-4b31-ba15-ba6f6aad3c21" containerName="revoke-keys"
Jul 16 21:08:41.709350 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.709105    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6ef4019-bb79-4b31-ba15-ba6f6aad3c21" containerName="revoke-keys"
Jul 16 21:08:41.709350 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.709115    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6ef4019-bb79-4b31-ba15-ba6f6aad3c21" containerName="revoke-keys"
Jul 16 21:08:41.709350 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.709121    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6ef4019-bb79-4b31-ba15-ba6f6aad3c21" containerName="revoke-keys"
Jul 16 21:08:41.712087 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.712069    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4"
Jul 16 21:08:41.721418 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.721397    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:08:41.721537 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.721398    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:08:41.743013 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.742986    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4"]
Jul 16 21:08:41.832759 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.832730    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4a7323-f128-41bf-825a-f5165904289a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4\" (UID: \"ce4a7323-f128-41bf-825a-f5165904289a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4"
Jul 16 21:08:41.933989 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.933955    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4a7323-f128-41bf-825a-f5165904289a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4\" (UID: \"ce4a7323-f128-41bf-825a-f5165904289a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4"
Jul 16 21:08:41.934630 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:41.934608    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4a7323-f128-41bf-825a-f5165904289a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4\" (UID: \"ce4a7323-f128-41bf-825a-f5165904289a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4"
Jul 16 21:08:42.022604 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:42.022533    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4"
Jul 16 21:08:42.367310 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:42.367244    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4"]
Jul 16 21:08:42.370687 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:08:42.370660    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce4a7323_f128_41bf_825a_f5165904289a.slice/crio-745a77b853037c7e41647d73757c46fc1c4db5e39f3d10065741a682e8121eaa WatchSource:0}: Error finding container 745a77b853037c7e41647d73757c46fc1c4db5e39f3d10065741a682e8121eaa: Status 404 returned error can't find the container with id 745a77b853037c7e41647d73757c46fc1c4db5e39f3d10065741a682e8121eaa
Jul 16 21:08:43.255809 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:43.255734    2576 generic.go:356] "Generic (PLEG): container finished" podID="ce4a7323-f128-41bf-825a-f5165904289a" containerID="6844a2abfda4a67a6b89d3bb8906b8165678635f9b41ef4486d6df387ce2ec8e" exitCode=6
Jul 16 21:08:43.256230 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:43.255820    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4" event={"ID":"ce4a7323-f128-41bf-825a-f5165904289a","Type":"ContainerDied","Data":"6844a2abfda4a67a6b89d3bb8906b8165678635f9b41ef4486d6df387ce2ec8e"}
Jul 16 21:08:43.256230 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:43.255875    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4" event={"ID":"ce4a7323-f128-41bf-825a-f5165904289a","Type":"ContainerStarted","Data":"745a77b853037c7e41647d73757c46fc1c4db5e39f3d10065741a682e8121eaa"}
Jul 16 21:08:43.256230 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:43.256047    2576 scope.go:117] "RemoveContainer" containerID="6844a2abfda4a67a6b89d3bb8906b8165678635f9b41ef4486d6df387ce2ec8e"
Jul 16 21:08:44.263205 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:44.263174    2576 generic.go:356] "Generic (PLEG): container finished" podID="ce4a7323-f128-41bf-825a-f5165904289a" containerID="996ea8c435e27fe1008eec13fe575e5cae3b95cafe27111bbff787ff51d4cd6c" exitCode=6
Jul 16 21:08:44.263591 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:44.263257    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4" event={"ID":"ce4a7323-f128-41bf-825a-f5165904289a","Type":"ContainerDied","Data":"996ea8c435e27fe1008eec13fe575e5cae3b95cafe27111bbff787ff51d4cd6c"}
Jul 16 21:08:44.263591 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:44.263319    2576 scope.go:117] "RemoveContainer" containerID="6844a2abfda4a67a6b89d3bb8906b8165678635f9b41ef4486d6df387ce2ec8e"
Jul 16 21:08:44.263591 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:44.263521    2576 scope.go:117] "RemoveContainer" containerID="996ea8c435e27fe1008eec13fe575e5cae3b95cafe27111bbff787ff51d4cd6c"
Jul 16 21:08:44.263715 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:08:44.263692    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4_odh-ai-gateway-infra(ce4a7323-f128-41bf-825a-f5165904289a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4" podUID="ce4a7323-f128-41bf-825a-f5165904289a"
Jul 16 21:08:45.270263 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:45.270230    2576 scope.go:117] "RemoveContainer" containerID="996ea8c435e27fe1008eec13fe575e5cae3b95cafe27111bbff787ff51d4cd6c"
Jul 16 21:08:45.270709 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:08:45.270417    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4_odh-ai-gateway-infra(ce4a7323-f128-41bf-825a-f5165904289a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4" podUID="ce4a7323-f128-41bf-825a-f5165904289a"
Jul 16 21:08:59.684537 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:08:59.684500    2576 scope.go:117] "RemoveContainer" containerID="996ea8c435e27fe1008eec13fe575e5cae3b95cafe27111bbff787ff51d4cd6c"
Jul 16 21:09:00.349522 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:00.349496    2576 generic.go:356] "Generic (PLEG): container finished" podID="ce4a7323-f128-41bf-825a-f5165904289a" containerID="91810e2329321444e180bc7244cd984c23cfba5f9cbf7287afe91780e8c7c27b" exitCode=6
Jul 16 21:09:00.349633 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:00.349552    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4" event={"ID":"ce4a7323-f128-41bf-825a-f5165904289a","Type":"ContainerDied","Data":"91810e2329321444e180bc7244cd984c23cfba5f9cbf7287afe91780e8c7c27b"}
Jul 16 21:09:00.349633 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:00.349598    2576 scope.go:117] "RemoveContainer" containerID="996ea8c435e27fe1008eec13fe575e5cae3b95cafe27111bbff787ff51d4cd6c"
Jul 16 21:09:00.349871 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:00.349854    2576 scope.go:117] "RemoveContainer" containerID="91810e2329321444e180bc7244cd984c23cfba5f9cbf7287afe91780e8c7c27b"
Jul 16 21:09:00.350045 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:09:00.350026    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4_odh-ai-gateway-infra(ce4a7323-f128-41bf-825a-f5165904289a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4" podUID="ce4a7323-f128-41bf-825a-f5165904289a"
Jul 16 21:09:01.380312 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:01.380275    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4"]
Jul 16 21:09:01.517657 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:01.517633    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4"
Jul 16 21:09:01.587669 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:01.587644    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4a7323-f128-41bf-825a-f5165904289a-maas-api-service-ca\") pod \"ce4a7323-f128-41bf-825a-f5165904289a\" (UID: \"ce4a7323-f128-41bf-825a-f5165904289a\") "
Jul 16 21:09:01.587956 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:01.587935    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce4a7323-f128-41bf-825a-f5165904289a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ce4a7323-f128-41bf-825a-f5165904289a" (UID: "ce4a7323-f128-41bf-825a-f5165904289a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:09:01.688592 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:01.688570    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4a7323-f128-41bf-825a-f5165904289a-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:09:02.361959 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:02.361930    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4"
Jul 16 21:09:02.362151 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:02.361929    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4" event={"ID":"ce4a7323-f128-41bf-825a-f5165904289a","Type":"ContainerDied","Data":"745a77b853037c7e41647d73757c46fc1c4db5e39f3d10065741a682e8121eaa"}
Jul 16 21:09:02.362151 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:02.362086    2576 scope.go:117] "RemoveContainer" containerID="91810e2329321444e180bc7244cd984c23cfba5f9cbf7287afe91780e8c7c27b"
Jul 16 21:09:02.390351 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:02.390327    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4"]
Jul 16 21:09:02.398532 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:02.398510    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-kv5n4"]
Jul 16 21:09:02.689459 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:02.689423    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce4a7323-f128-41bf-825a-f5165904289a" path="/var/lib/kubelet/pods/ce4a7323-f128-41bf-825a-f5165904289a/volumes"
Jul 16 21:09:33.650225 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:33.650189    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp"]
Jul 16 21:09:33.650733 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:33.650633    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce4a7323-f128-41bf-825a-f5165904289a" containerName="revoke-keys"
Jul 16 21:09:33.650733 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:33.650648    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce4a7323-f128-41bf-825a-f5165904289a" containerName="revoke-keys"
Jul 16 21:09:33.650733 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:33.650661    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce4a7323-f128-41bf-825a-f5165904289a" containerName="revoke-keys"
Jul 16 21:09:33.650733 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:33.650669    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce4a7323-f128-41bf-825a-f5165904289a" containerName="revoke-keys"
Jul 16 21:09:33.650876 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:33.650750    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce4a7323-f128-41bf-825a-f5165904289a" containerName="revoke-keys"
Jul 16 21:09:33.650876 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:33.650762    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce4a7323-f128-41bf-825a-f5165904289a" containerName="revoke-keys"
Jul 16 21:09:33.653915 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:33.653897    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp"
Jul 16 21:09:33.660456 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:33.660433    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:09:33.660624 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:33.660453    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:09:33.672742 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:33.672717    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp"]
Jul 16 21:09:33.827955 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:33.827932    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6754d967-e666-4a28-9c7a-b7a4c5087ba2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp\" (UID: \"6754d967-e666-4a28-9c7a-b7a4c5087ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp"
Jul 16 21:09:33.928611 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:33.928537    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6754d967-e666-4a28-9c7a-b7a4c5087ba2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp\" (UID: \"6754d967-e666-4a28-9c7a-b7a4c5087ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp"
Jul 16 21:09:33.929157 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:33.929135    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6754d967-e666-4a28-9c7a-b7a4c5087ba2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp\" (UID: \"6754d967-e666-4a28-9c7a-b7a4c5087ba2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp"
Jul 16 21:09:33.964597 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:33.964573    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp"
Jul 16 21:09:34.117543 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:34.117521    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp"]
Jul 16 21:09:34.119506 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:09:34.119480    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6754d967_e666_4a28_9c7a_b7a4c5087ba2.slice/crio-c18c86698ee722aff3647e5d8aee5a90238848506b956a81ee16d3a7318c9e44 WatchSource:0}: Error finding container c18c86698ee722aff3647e5d8aee5a90238848506b956a81ee16d3a7318c9e44: Status 404 returned error can't find the container with id c18c86698ee722aff3647e5d8aee5a90238848506b956a81ee16d3a7318c9e44
Jul 16 21:09:34.535096 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:34.535027    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp" event={"ID":"6754d967-e666-4a28-9c7a-b7a4c5087ba2","Type":"ContainerStarted","Data":"227a24948d3fe54c4b8985fe8e1ee9599c70eae74493221c43dffe2d7aa02dc2"}
Jul 16 21:09:34.535096 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:34.535082    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp" event={"ID":"6754d967-e666-4a28-9c7a-b7a4c5087ba2","Type":"ContainerStarted","Data":"c18c86698ee722aff3647e5d8aee5a90238848506b956a81ee16d3a7318c9e44"}
Jul 16 21:09:34.557667 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:34.557624    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp" podStartSLOduration=1.557609477 podStartE2EDuration="1.557609477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:09:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:09:34.555202187 +0000 UTC m=+2220.521136441" watchObservedRunningTime="2026-07-16 21:09:34.557609477 +0000 UTC m=+2220.523543779"
Jul 16 21:09:35.542158 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:35.542101    2576 generic.go:356] "Generic (PLEG): container finished" podID="6754d967-e666-4a28-9c7a-b7a4c5087ba2" containerID="227a24948d3fe54c4b8985fe8e1ee9599c70eae74493221c43dffe2d7aa02dc2" exitCode=6
Jul 16 21:09:35.542582 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:35.542190    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp" event={"ID":"6754d967-e666-4a28-9c7a-b7a4c5087ba2","Type":"ContainerDied","Data":"227a24948d3fe54c4b8985fe8e1ee9599c70eae74493221c43dffe2d7aa02dc2"}
Jul 16 21:09:35.542582 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:35.542399    2576 scope.go:117] "RemoveContainer" containerID="227a24948d3fe54c4b8985fe8e1ee9599c70eae74493221c43dffe2d7aa02dc2"
Jul 16 21:09:36.552913 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:36.552877    2576 generic.go:356] "Generic (PLEG): container finished" podID="6754d967-e666-4a28-9c7a-b7a4c5087ba2" containerID="f95567f69d2435be06bd53f13f4481f2ac76835972a0316195d68274c599edcb" exitCode=6
Jul 16 21:09:36.553376 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:36.552947    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp" event={"ID":"6754d967-e666-4a28-9c7a-b7a4c5087ba2","Type":"ContainerDied","Data":"f95567f69d2435be06bd53f13f4481f2ac76835972a0316195d68274c599edcb"}
Jul 16 21:09:36.553376 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:36.553021    2576 scope.go:117] "RemoveContainer" containerID="227a24948d3fe54c4b8985fe8e1ee9599c70eae74493221c43dffe2d7aa02dc2"
Jul 16 21:09:36.553376 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:36.553207    2576 scope.go:117] "RemoveContainer" containerID="f95567f69d2435be06bd53f13f4481f2ac76835972a0316195d68274c599edcb"
Jul 16 21:09:36.553534 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:09:36.553466    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp_odh-ai-gateway-infra(6754d967-e666-4a28-9c7a-b7a4c5087ba2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp" podUID="6754d967-e666-4a28-9c7a-b7a4c5087ba2"
Jul 16 21:09:37.560295 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:37.560261    2576 scope.go:117] "RemoveContainer" containerID="f95567f69d2435be06bd53f13f4481f2ac76835972a0316195d68274c599edcb"
Jul 16 21:09:37.560681 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:09:37.560469    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp_odh-ai-gateway-infra(6754d967-e666-4a28-9c7a-b7a4c5087ba2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp" podUID="6754d967-e666-4a28-9c7a-b7a4c5087ba2"
Jul 16 21:09:49.683723 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:49.683687    2576 scope.go:117] "RemoveContainer" containerID="f95567f69d2435be06bd53f13f4481f2ac76835972a0316195d68274c599edcb"
Jul 16 21:09:50.626875 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:50.626802    2576 generic.go:356] "Generic (PLEG): container finished" podID="6754d967-e666-4a28-9c7a-b7a4c5087ba2" containerID="cf8aa72adcfd5cae4bbbd03f741c40b376ba8a0cd6c1f03fcd1e9c95a2ef3e8f" exitCode=6
Jul 16 21:09:50.627040 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:50.626871    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp" event={"ID":"6754d967-e666-4a28-9c7a-b7a4c5087ba2","Type":"ContainerDied","Data":"cf8aa72adcfd5cae4bbbd03f741c40b376ba8a0cd6c1f03fcd1e9c95a2ef3e8f"}
Jul 16 21:09:50.627040 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:50.626948    2576 scope.go:117] "RemoveContainer" containerID="f95567f69d2435be06bd53f13f4481f2ac76835972a0316195d68274c599edcb"
Jul 16 21:09:50.627256 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:50.627239    2576 scope.go:117] "RemoveContainer" containerID="cf8aa72adcfd5cae4bbbd03f741c40b376ba8a0cd6c1f03fcd1e9c95a2ef3e8f"
Jul 16 21:09:50.627525 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:09:50.627500    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp_odh-ai-gateway-infra(6754d967-e666-4a28-9c7a-b7a4c5087ba2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp" podUID="6754d967-e666-4a28-9c7a-b7a4c5087ba2"
Jul 16 21:09:51.659158 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:51.659121    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp"]
Jul 16 21:09:51.794198 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:51.794176    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp"
Jul 16 21:09:51.871240 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:51.871208    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6754d967-e666-4a28-9c7a-b7a4c5087ba2-maas-api-service-ca\") pod \"6754d967-e666-4a28-9c7a-b7a4c5087ba2\" (UID: \"6754d967-e666-4a28-9c7a-b7a4c5087ba2\") "
Jul 16 21:09:51.871560 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:51.871539    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6754d967-e666-4a28-9c7a-b7a4c5087ba2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6754d967-e666-4a28-9c7a-b7a4c5087ba2" (UID: "6754d967-e666-4a28-9c7a-b7a4c5087ba2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:09:51.972243 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:51.972185    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6754d967-e666-4a28-9c7a-b7a4c5087ba2-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:09:52.641466 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:52.641422    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp"
Jul 16 21:09:52.641652 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:52.641451    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp" event={"ID":"6754d967-e666-4a28-9c7a-b7a4c5087ba2","Type":"ContainerDied","Data":"c18c86698ee722aff3647e5d8aee5a90238848506b956a81ee16d3a7318c9e44"}
Jul 16 21:09:52.641652 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:52.641568    2576 scope.go:117] "RemoveContainer" containerID="cf8aa72adcfd5cae4bbbd03f741c40b376ba8a0cd6c1f03fcd1e9c95a2ef3e8f"
Jul 16 21:09:52.669681 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:52.669657    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp"]
Jul 16 21:09:52.680585 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:52.680561    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-q8hvp"]
Jul 16 21:09:52.689600 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:09:52.689575    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6754d967-e666-4a28-9c7a-b7a4c5087ba2" path="/var/lib/kubelet/pods/6754d967-e666-4a28-9c7a-b7a4c5087ba2/volumes"
Jul 16 21:10:00.904076 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:00.903988    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"]
Jul 16 21:10:00.904545 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:00.904204    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47" podUID="d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" containerName="istio-proxy" containerID="cri-o://ba4538e7adc525368c7cf4d1c9a3a44ff77ad0635f45da4a4013503f41062e18" gracePeriod=30
Jul 16 21:10:04.101453 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:04.101413    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:10:05.701218 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:05.701174    2576 patch_prober.go:28] interesting pod/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 21:10:05.701715 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:05.701251    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47" podUID="d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 21:10:06.168695 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.168671    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:10:06.280520 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.280463    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-workload-certs\") pod \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") "
Jul 16 21:10:06.280520 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.280509    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-envoy\") pod \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") "
Jul 16 21:10:06.280705 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.280533    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-workload-socket\") pod \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") "
Jul 16 21:10:06.280705 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.280553    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p92nv\" (UniqueName: \"kubernetes.io/projected/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-kube-api-access-p92nv\") pod \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") "
Jul 16 21:10:06.280705 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.280570    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-data\") pod \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") "
Jul 16 21:10:06.280705 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.280593    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-credential-socket\") pod \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") "
Jul 16 21:10:06.280705 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.280664    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-podinfo\") pod \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") "
Jul 16 21:10:06.281037 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.280785    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istiod-ca-cert\") pod \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") "
Jul 16 21:10:06.281037 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.280816    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-token\") pod \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\" (UID: \"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815\") "
Jul 16 21:10:06.281037 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.280838    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" (UID: "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:10:06.281037 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.280880    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" (UID: "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:10:06.281237 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.281128    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-workload-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:10:06.281237 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.281149    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-workload-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:10:06.281237 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.281159    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" (UID: "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:10:06.281645 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.281611    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" (UID: "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:10:06.281645 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.281626    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-data" (OuterVolumeSpecName: "istio-data") pod "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" (UID: "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:10:06.282845 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.282816    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-kube-api-access-p92nv" (OuterVolumeSpecName: "kube-api-access-p92nv") pod "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" (UID: "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815"). InnerVolumeSpecName "kube-api-access-p92nv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:10:06.282962 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.282848    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" (UID: "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:10:06.283329 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.283290    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" (UID: "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 21:10:06.283606 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.283583    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-token" (OuterVolumeSpecName: "istio-token") pod "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" (UID: "d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:10:06.381597 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.381571    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-envoy\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:10:06.381705 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.381598    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p92nv\" (UniqueName: \"kubernetes.io/projected/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-kube-api-access-p92nv\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:10:06.381705 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.381613    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-data\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:10:06.381705 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.381625    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-credential-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:10:06.381705 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.381637    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-podinfo\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:10:06.381705 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.381649    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istiod-ca-cert\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:10:06.381705 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.381661    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815-istio-token\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:10:06.712724 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.712692    2576 generic.go:356] "Generic (PLEG): container finished" podID="d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" containerID="ba4538e7adc525368c7cf4d1c9a3a44ff77ad0635f45da4a4013503f41062e18" exitCode=0
Jul 16 21:10:06.713074 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.712725    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47" event={"ID":"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815","Type":"ContainerDied","Data":"ba4538e7adc525368c7cf4d1c9a3a44ff77ad0635f45da4a4013503f41062e18"}
Jul 16 21:10:06.713074 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.712742    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"
Jul 16 21:10:06.713074 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.712786    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47" event={"ID":"d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815","Type":"ContainerDied","Data":"9e31ca370a0cf84595c55a1f88732a6167e111ecccee2b22e38beaadc917d6f4"}
Jul 16 21:10:06.713074 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.712816    2576 scope.go:117] "RemoveContainer" containerID="ba4538e7adc525368c7cf4d1c9a3a44ff77ad0635f45da4a4013503f41062e18"
Jul 16 21:10:06.721987 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.721970    2576 scope.go:117] "RemoveContainer" containerID="63b687107976f082e439e3f37c51941ef5fc8198b7ee4823920b07575030e840"
Jul 16 21:10:06.729534 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.729514    2576 scope.go:117] "RemoveContainer" containerID="ba4538e7adc525368c7cf4d1c9a3a44ff77ad0635f45da4a4013503f41062e18"
Jul 16 21:10:06.729781 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:10:06.729761    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba4538e7adc525368c7cf4d1c9a3a44ff77ad0635f45da4a4013503f41062e18\": container with ID starting with ba4538e7adc525368c7cf4d1c9a3a44ff77ad0635f45da4a4013503f41062e18 not found: ID does not exist" containerID="ba4538e7adc525368c7cf4d1c9a3a44ff77ad0635f45da4a4013503f41062e18"
Jul 16 21:10:06.729839 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.729788    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba4538e7adc525368c7cf4d1c9a3a44ff77ad0635f45da4a4013503f41062e18"} err="failed to get container status \"ba4538e7adc525368c7cf4d1c9a3a44ff77ad0635f45da4a4013503f41062e18\": rpc error: code = NotFound desc = could not find container \"ba4538e7adc525368c7cf4d1c9a3a44ff77ad0635f45da4a4013503f41062e18\": container with ID starting with ba4538e7adc525368c7cf4d1c9a3a44ff77ad0635f45da4a4013503f41062e18 not found: ID does not exist"
Jul 16 21:10:06.729839 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.729804    2576 scope.go:117] "RemoveContainer" containerID="63b687107976f082e439e3f37c51941ef5fc8198b7ee4823920b07575030e840"
Jul 16 21:10:06.730038 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:10:06.730019    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63b687107976f082e439e3f37c51941ef5fc8198b7ee4823920b07575030e840\": container with ID starting with 63b687107976f082e439e3f37c51941ef5fc8198b7ee4823920b07575030e840 not found: ID does not exist" containerID="63b687107976f082e439e3f37c51941ef5fc8198b7ee4823920b07575030e840"
Jul 16 21:10:06.730087 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.730043    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63b687107976f082e439e3f37c51941ef5fc8198b7ee4823920b07575030e840"} err="failed to get container status \"63b687107976f082e439e3f37c51941ef5fc8198b7ee4823920b07575030e840\": rpc error: code = NotFound desc = could not find container \"63b687107976f082e439e3f37c51941ef5fc8198b7ee4823920b07575030e840\": container with ID starting with 63b687107976f082e439e3f37c51941ef5fc8198b7ee4823920b07575030e840 not found: ID does not exist"
Jul 16 21:10:06.736045 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.736025    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"]
Jul 16 21:10:06.746812 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:06.746793    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e65739ab-openshift-default-6fd4cd7967-8kp47"]
Jul 16 21:10:08.690003 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:08.689967    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" path="/var/lib/kubelet/pods/d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815/volumes"
Jul 16 21:10:13.350166 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.350134    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"]
Jul 16 21:10:13.350549 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.350503    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce4a7323-f128-41bf-825a-f5165904289a" containerName="revoke-keys"
Jul 16 21:10:13.350549 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.350515    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce4a7323-f128-41bf-825a-f5165904289a" containerName="revoke-keys"
Jul 16 21:10:13.350549 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.350527    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6754d967-e666-4a28-9c7a-b7a4c5087ba2" containerName="revoke-keys"
Jul 16 21:10:13.350549 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.350545    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6754d967-e666-4a28-9c7a-b7a4c5087ba2" containerName="revoke-keys"
Jul 16 21:10:13.350733 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.350561    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6754d967-e666-4a28-9c7a-b7a4c5087ba2" containerName="revoke-keys"
Jul 16 21:10:13.350733 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.350567    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6754d967-e666-4a28-9c7a-b7a4c5087ba2" containerName="revoke-keys"
Jul 16 21:10:13.350733 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.350573    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" containerName="istio-proxy"
Jul 16 21:10:13.350733 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.350579    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" containerName="istio-proxy"
Jul 16 21:10:13.350733 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.350587    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" containerName="istio-proxy"
Jul 16 21:10:13.350733 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.350592    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" containerName="istio-proxy"
Jul 16 21:10:13.350733 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.350669    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" containerName="istio-proxy"
Jul 16 21:10:13.350733 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.350679    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce4a7323-f128-41bf-825a-f5165904289a" containerName="revoke-keys"
Jul 16 21:10:13.350733 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.350691    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6754d967-e666-4a28-9c7a-b7a4c5087ba2" containerName="revoke-keys"
Jul 16 21:10:13.350733 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.350705    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6754d967-e666-4a28-9c7a-b7a4c5087ba2" containerName="revoke-keys"
Jul 16 21:10:13.353836 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.353814    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.362426 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.362404    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-90b68ec7-openshift-default-dockercfg-4gm2v\""
Jul 16 21:10:13.373951 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.373929    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"]
Jul 16 21:10:13.437520 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.437497    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-workload-certs\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.437631 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.437530    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-data\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.437631 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.437547    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-podinfo\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.437631 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.437567    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istiod-ca-cert\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.437750 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.437707    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-token\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.437792 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.437747    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w92s9\" (UniqueName: \"kubernetes.io/projected/5e3cef2c-0cc6-4855-8f97-fea412ed5381-kube-api-access-w92s9\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.437833 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.437808    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-workload-socket\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.437874 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.437850    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-credential-socket\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.437920 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.437883    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-envoy\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.539153 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.539113    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-token\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.539153 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.539155    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w92s9\" (UniqueName: \"kubernetes.io/projected/5e3cef2c-0cc6-4855-8f97-fea412ed5381-kube-api-access-w92s9\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.539355 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.539184    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-workload-socket\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.539355 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.539202    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-credential-socket\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.539355 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.539328    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-envoy\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.539560 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.539414    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-workload-certs\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.539560 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.539459    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-data\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.539560 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.539486    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-podinfo\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.539560 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.539516    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istiod-ca-cert\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.539762 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.539605    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-workload-socket\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.539762 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.539635    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-credential-socket\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.539872 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.539845    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-data\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.539953 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.539931    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-workload-certs\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.540224 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.540201    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istiod-ca-cert\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.541665 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.541640    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-envoy\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.541845 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.541828    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-podinfo\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.548129 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.548107    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-token\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.548436 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.548417    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w92s9\" (UniqueName: \"kubernetes.io/projected/5e3cef2c-0cc6-4855-8f97-fea412ed5381-kube-api-access-w92s9\") pod \"e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") " pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.665205 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.665182    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:13.804064 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.804031    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"]
Jul 16 21:10:13.807459 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:10:13.807431    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e3cef2c_0cc6_4855_8f97_fea412ed5381.slice/crio-cd744bee58b5cf7ed0e54225bbf8bcf113147fd2dc6af2023b6d99cf56220ae5 WatchSource:0}: Error finding container cd744bee58b5cf7ed0e54225bbf8bcf113147fd2dc6af2023b6d99cf56220ae5: Status 404 returned error can't find the container with id cd744bee58b5cf7ed0e54225bbf8bcf113147fd2dc6af2023b6d99cf56220ae5
Jul 16 21:10:13.809760 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.809713    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:10:13.809836 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.809809    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:10:13.809879 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:13.809854    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:10:14.753123 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:14.753089    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp" event={"ID":"5e3cef2c-0cc6-4855-8f97-fea412ed5381","Type":"ContainerStarted","Data":"8ad5f5dc19ccb165a238563f64e301c258f323c52e8172c705453166499d1323"}
Jul 16 21:10:14.753568 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:14.753141    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp" event={"ID":"5e3cef2c-0cc6-4855-8f97-fea412ed5381","Type":"ContainerStarted","Data":"cd744bee58b5cf7ed0e54225bbf8bcf113147fd2dc6af2023b6d99cf56220ae5"}
Jul 16 21:10:14.787125 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:14.787081    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp" podStartSLOduration=1.78706525 podStartE2EDuration="1.78706525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:10:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:10:14.78660835 +0000 UTC m=+2260.752542583" watchObservedRunningTime="2026-07-16 21:10:14.78706525 +0000 UTC m=+2260.752999555"
Jul 16 21:10:15.666159 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:15.666132    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:15.681315 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:15.681290    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:15.759877 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:15.759853    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:15.764753 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:15.764718    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:22.192668 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:22.192638    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:10:30.760621 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:30.760595    2576 patch_prober.go:28] interesting pod/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 16 21:10:30.760986 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:30.760655    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp" podUID="5e3cef2c-0cc6-4855-8f97-fea412ed5381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 16 21:10:30.834193 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:30.834167    2576 generic.go:356] "Generic (PLEG): container finished" podID="5e3cef2c-0cc6-4855-8f97-fea412ed5381" containerID="8ad5f5dc19ccb165a238563f64e301c258f323c52e8172c705453166499d1323" exitCode=0
Jul 16 21:10:30.834306 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:30.834243    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp" event={"ID":"5e3cef2c-0cc6-4855-8f97-fea412ed5381","Type":"ContainerDied","Data":"8ad5f5dc19ccb165a238563f64e301c258f323c52e8172c705453166499d1323"}
Jul 16 21:10:30.834597 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:30.834579    2576 scope.go:117] "RemoveContainer" containerID="8ad5f5dc19ccb165a238563f64e301c258f323c52e8172c705453166499d1323"
Jul 16 21:10:30.835517 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:30.835492    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:10:30.835613 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:30.835549    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:10:30.835613 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:30.835580    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:10:31.843382 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:31.843275    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp" event={"ID":"5e3cef2c-0cc6-4855-8f97-fea412ed5381","Type":"ContainerStarted","Data":"84adc9b03354d3503c5a7f61443821a62ffd488240244a74846f96991ec12079"}
Jul 16 21:10:32.666340 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:32.666307    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:32.666534 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:32.666515    2576 patch_prober.go:28] interesting pod/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 16 21:10:32.666591 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:32.666572    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp" podUID="5e3cef2c-0cc6-4855-8f97-fea412ed5381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 16 21:10:33.666131 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:33.666093    2576 patch_prober.go:28] interesting pod/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 16 21:10:33.666540 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:33.666149    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp" podUID="5e3cef2c-0cc6-4855-8f97-fea412ed5381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 16 21:10:34.665906 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:34.665869    2576 patch_prober.go:28] interesting pod/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 16 21:10:34.666143 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:34.665930    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp" podUID="5e3cef2c-0cc6-4855-8f97-fea412ed5381" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 16 21:10:35.670220 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:35.670188    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:35.670642 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:35.670572    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:10:35.671278 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:10:35.671258    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:11:13.746368 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:13.746320    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml"]
Jul 16 21:11:13.746862 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:13.746740    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6754d967-e666-4a28-9c7a-b7a4c5087ba2" containerName="revoke-keys"
Jul 16 21:11:13.746862 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:13.746753    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6754d967-e666-4a28-9c7a-b7a4c5087ba2" containerName="revoke-keys"
Jul 16 21:11:13.746862 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:13.746809    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d21c8dab-0ff9-4bd9-8a0f-fb0d0f75d815" containerName="istio-proxy"
Jul 16 21:11:13.746862 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:13.746819    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6754d967-e666-4a28-9c7a-b7a4c5087ba2" containerName="revoke-keys"
Jul 16 21:11:13.749041 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:13.749021    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml"
Jul 16 21:11:13.751623 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:13.751597    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:11:13.751839 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:13.751821    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:11:13.758215 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:13.758193    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml"]
Jul 16 21:11:13.844800 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:13.844771    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/099a38c3-c0e3-4356-a905-9d11845b5ebd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml\" (UID: \"099a38c3-c0e3-4356-a905-9d11845b5ebd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml"
Jul 16 21:11:13.945684 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:13.945657    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/099a38c3-c0e3-4356-a905-9d11845b5ebd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml\" (UID: \"099a38c3-c0e3-4356-a905-9d11845b5ebd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml"
Jul 16 21:11:13.946248 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:13.946228    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/099a38c3-c0e3-4356-a905-9d11845b5ebd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml\" (UID: \"099a38c3-c0e3-4356-a905-9d11845b5ebd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml"
Jul 16 21:11:14.060832 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:14.060773    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml"
Jul 16 21:11:14.397659 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:14.397633    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml"]
Jul 16 21:11:14.400070 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:11:14.400034    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod099a38c3_c0e3_4356_a905_9d11845b5ebd.slice/crio-df28edbb3623bbd52acbe6254f156a07524efc687a6c0451ee065851c3a80b9d WatchSource:0}: Error finding container df28edbb3623bbd52acbe6254f156a07524efc687a6c0451ee065851c3a80b9d: Status 404 returned error can't find the container with id df28edbb3623bbd52acbe6254f156a07524efc687a6c0451ee065851c3a80b9d
Jul 16 21:11:15.057012 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:15.056985    2576 generic.go:356] "Generic (PLEG): container finished" podID="099a38c3-c0e3-4356-a905-9d11845b5ebd" containerID="c36ea2a96506b4e1d70dfa1cff979f98d08faf9f672e5ff2b2c35554afc8440e" exitCode=6
Jul 16 21:11:15.057342 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:15.057062    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml" event={"ID":"099a38c3-c0e3-4356-a905-9d11845b5ebd","Type":"ContainerDied","Data":"c36ea2a96506b4e1d70dfa1cff979f98d08faf9f672e5ff2b2c35554afc8440e"}
Jul 16 21:11:15.057342 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:15.057153    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml" event={"ID":"099a38c3-c0e3-4356-a905-9d11845b5ebd","Type":"ContainerStarted","Data":"df28edbb3623bbd52acbe6254f156a07524efc687a6c0451ee065851c3a80b9d"}
Jul 16 21:11:15.057342 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:15.057225    2576 scope.go:117] "RemoveContainer" containerID="c36ea2a96506b4e1d70dfa1cff979f98d08faf9f672e5ff2b2c35554afc8440e"
Jul 16 21:11:16.063327 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:16.063296    2576 generic.go:356] "Generic (PLEG): container finished" podID="099a38c3-c0e3-4356-a905-9d11845b5ebd" containerID="101270cded44ee3615b37b5f0a5f473349dbc8eef3f771d824c72c24f28433cd" exitCode=6
Jul 16 21:11:16.063705 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:16.063393    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml" event={"ID":"099a38c3-c0e3-4356-a905-9d11845b5ebd","Type":"ContainerDied","Data":"101270cded44ee3615b37b5f0a5f473349dbc8eef3f771d824c72c24f28433cd"}
Jul 16 21:11:16.063705 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:16.063447    2576 scope.go:117] "RemoveContainer" containerID="c36ea2a96506b4e1d70dfa1cff979f98d08faf9f672e5ff2b2c35554afc8440e"
Jul 16 21:11:16.063705 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:16.063633    2576 scope.go:117] "RemoveContainer" containerID="101270cded44ee3615b37b5f0a5f473349dbc8eef3f771d824c72c24f28433cd"
Jul 16 21:11:16.063888 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:11:16.063827    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml_odh-ai-gateway-infra(099a38c3-c0e3-4356-a905-9d11845b5ebd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml" podUID="099a38c3-c0e3-4356-a905-9d11845b5ebd"
Jul 16 21:11:17.070193 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:17.070170    2576 scope.go:117] "RemoveContainer" containerID="101270cded44ee3615b37b5f0a5f473349dbc8eef3f771d824c72c24f28433cd"
Jul 16 21:11:17.070583 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:11:17.070377    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml_odh-ai-gateway-infra(099a38c3-c0e3-4356-a905-9d11845b5ebd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml" podUID="099a38c3-c0e3-4356-a905-9d11845b5ebd"
Jul 16 21:11:27.684595 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:27.684516    2576 scope.go:117] "RemoveContainer" containerID="101270cded44ee3615b37b5f0a5f473349dbc8eef3f771d824c72c24f28433cd"
Jul 16 21:11:28.127439 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:28.127409    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml" event={"ID":"099a38c3-c0e3-4356-a905-9d11845b5ebd","Type":"ContainerStarted","Data":"4434f0405d51cb6a9673899a05eedece7293f00e8038c76bb1f049bb8a476efe"}
Jul 16 21:11:28.150683 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:28.150636    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml" podStartSLOduration=15.150621277 podStartE2EDuration="15.150621277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:11:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:11:28.148555235 +0000 UTC m=+2334.114489469" watchObservedRunningTime="2026-07-16 21:11:28.150621277 +0000 UTC m=+2334.116555488"
Jul 16 21:11:29.134137 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:29.134104    2576 generic.go:356] "Generic (PLEG): container finished" podID="099a38c3-c0e3-4356-a905-9d11845b5ebd" containerID="4434f0405d51cb6a9673899a05eedece7293f00e8038c76bb1f049bb8a476efe" exitCode=6
Jul 16 21:11:29.134612 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:29.134181    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml" event={"ID":"099a38c3-c0e3-4356-a905-9d11845b5ebd","Type":"ContainerDied","Data":"4434f0405d51cb6a9673899a05eedece7293f00e8038c76bb1f049bb8a476efe"}
Jul 16 21:11:29.134612 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:29.134250    2576 scope.go:117] "RemoveContainer" containerID="101270cded44ee3615b37b5f0a5f473349dbc8eef3f771d824c72c24f28433cd"
Jul 16 21:11:29.134612 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:29.134530    2576 scope.go:117] "RemoveContainer" containerID="4434f0405d51cb6a9673899a05eedece7293f00e8038c76bb1f049bb8a476efe"
Jul 16 21:11:29.134756 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:11:29.134718    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml_odh-ai-gateway-infra(099a38c3-c0e3-4356-a905-9d11845b5ebd)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml" podUID="099a38c3-c0e3-4356-a905-9d11845b5ebd"
Jul 16 21:11:29.160515 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:29.160490    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml"]
Jul 16 21:11:30.268727 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:30.268708    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml"
Jul 16 21:11:30.375863 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:30.375839    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/099a38c3-c0e3-4356-a905-9d11845b5ebd-maas-api-service-ca\") pod \"099a38c3-c0e3-4356-a905-9d11845b5ebd\" (UID: \"099a38c3-c0e3-4356-a905-9d11845b5ebd\") "
Jul 16 21:11:30.376157 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:30.376135    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/099a38c3-c0e3-4356-a905-9d11845b5ebd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "099a38c3-c0e3-4356-a905-9d11845b5ebd" (UID: "099a38c3-c0e3-4356-a905-9d11845b5ebd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:11:30.476601 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:30.476537    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/099a38c3-c0e3-4356-a905-9d11845b5ebd-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:11:31.099473 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.099442    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw"]
Jul 16 21:11:31.099810 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.099797    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099a38c3-c0e3-4356-a905-9d11845b5ebd" containerName="revoke-keys"
Jul 16 21:11:31.099857 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.099811    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="099a38c3-c0e3-4356-a905-9d11845b5ebd" containerName="revoke-keys"
Jul 16 21:11:31.099857 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.099822    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099a38c3-c0e3-4356-a905-9d11845b5ebd" containerName="revoke-keys"
Jul 16 21:11:31.099857 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.099828    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="099a38c3-c0e3-4356-a905-9d11845b5ebd" containerName="revoke-keys"
Jul 16 21:11:31.099956 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.099897    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="099a38c3-c0e3-4356-a905-9d11845b5ebd" containerName="revoke-keys"
Jul 16 21:11:31.099956 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.099906    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="099a38c3-c0e3-4356-a905-9d11845b5ebd" containerName="revoke-keys"
Jul 16 21:11:31.099956 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.099913    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="099a38c3-c0e3-4356-a905-9d11845b5ebd" containerName="revoke-keys"
Jul 16 21:11:31.101974 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.101959    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw"
Jul 16 21:11:31.109977 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.109863    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw"]
Jul 16 21:11:31.146254 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.146227    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml"
Jul 16 21:11:31.146401 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.146274    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml" event={"ID":"099a38c3-c0e3-4356-a905-9d11845b5ebd","Type":"ContainerDied","Data":"df28edbb3623bbd52acbe6254f156a07524efc687a6c0451ee065851c3a80b9d"}
Jul 16 21:11:31.146401 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.146330    2576 scope.go:117] "RemoveContainer" containerID="4434f0405d51cb6a9673899a05eedece7293f00e8038c76bb1f049bb8a476efe"
Jul 16 21:11:31.163081 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.163062    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml"]
Jul 16 21:11:31.168950 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.168925    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-dmtml"]
Jul 16 21:11:31.282616 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.282591    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90a8674b-1ff8-4c23-8696-106d4c05c11d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw\" (UID: \"90a8674b-1ff8-4c23-8696-106d4c05c11d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw"
Jul 16 21:11:31.383676 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.383621    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90a8674b-1ff8-4c23-8696-106d4c05c11d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw\" (UID: \"90a8674b-1ff8-4c23-8696-106d4c05c11d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw"
Jul 16 21:11:31.384155 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.384138    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90a8674b-1ff8-4c23-8696-106d4c05c11d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw\" (UID: \"90a8674b-1ff8-4c23-8696-106d4c05c11d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw"
Jul 16 21:11:31.413180 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.413156    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw"
Jul 16 21:11:31.544394 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:31.544350    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw"]
Jul 16 21:11:31.545890 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:11:31.545854    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90a8674b_1ff8_4c23_8696_106d4c05c11d.slice/crio-4e2b8aa7036e1791e440f5cddce4660e79df25c9bb0f7ecfa1def14958962a7a WatchSource:0}: Error finding container 4e2b8aa7036e1791e440f5cddce4660e79df25c9bb0f7ecfa1def14958962a7a: Status 404 returned error can't find the container with id 4e2b8aa7036e1791e440f5cddce4660e79df25c9bb0f7ecfa1def14958962a7a
Jul 16 21:11:32.153996 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:32.153967    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw" event={"ID":"90a8674b-1ff8-4c23-8696-106d4c05c11d","Type":"ContainerStarted","Data":"354360353a89985c2716e89b15a740f44f290d801cba94d48fcc0ae9a6f651bf"}
Jul 16 21:11:32.154138 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:32.154011    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw" event={"ID":"90a8674b-1ff8-4c23-8696-106d4c05c11d","Type":"ContainerStarted","Data":"4e2b8aa7036e1791e440f5cddce4660e79df25c9bb0f7ecfa1def14958962a7a"}
Jul 16 21:11:32.176815 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:32.176761    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw" podStartSLOduration=1.176743501 podStartE2EDuration="1.176743501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:11:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:11:32.176571837 +0000 UTC m=+2338.142506071" watchObservedRunningTime="2026-07-16 21:11:32.176743501 +0000 UTC m=+2338.142677739"
Jul 16 21:11:32.697368 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:32.697331    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="099a38c3-c0e3-4356-a905-9d11845b5ebd" path="/var/lib/kubelet/pods/099a38c3-c0e3-4356-a905-9d11845b5ebd/volumes"
Jul 16 21:11:33.160724 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:33.160688    2576 generic.go:356] "Generic (PLEG): container finished" podID="90a8674b-1ff8-4c23-8696-106d4c05c11d" containerID="354360353a89985c2716e89b15a740f44f290d801cba94d48fcc0ae9a6f651bf" exitCode=6
Jul 16 21:11:33.160872 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:33.160775    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw" event={"ID":"90a8674b-1ff8-4c23-8696-106d4c05c11d","Type":"ContainerDied","Data":"354360353a89985c2716e89b15a740f44f290d801cba94d48fcc0ae9a6f651bf"}
Jul 16 21:11:33.160964 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:33.160946    2576 scope.go:117] "RemoveContainer" containerID="354360353a89985c2716e89b15a740f44f290d801cba94d48fcc0ae9a6f651bf"
Jul 16 21:11:34.168441 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:34.168405    2576 generic.go:356] "Generic (PLEG): container finished" podID="90a8674b-1ff8-4c23-8696-106d4c05c11d" containerID="d39bbca63bf3e40ad199b8fa240e1a2a63eae40f2660e1b0c77abdf8a15b44a1" exitCode=6
Jul 16 21:11:34.168838 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:34.168488    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw" event={"ID":"90a8674b-1ff8-4c23-8696-106d4c05c11d","Type":"ContainerDied","Data":"d39bbca63bf3e40ad199b8fa240e1a2a63eae40f2660e1b0c77abdf8a15b44a1"}
Jul 16 21:11:34.168838 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:34.168551    2576 scope.go:117] "RemoveContainer" containerID="354360353a89985c2716e89b15a740f44f290d801cba94d48fcc0ae9a6f651bf"
Jul 16 21:11:34.168838 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:34.168699    2576 scope.go:117] "RemoveContainer" containerID="d39bbca63bf3e40ad199b8fa240e1a2a63eae40f2660e1b0c77abdf8a15b44a1"
Jul 16 21:11:34.168967 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:11:34.168912    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw_odh-ai-gateway-infra(90a8674b-1ff8-4c23-8696-106d4c05c11d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw" podUID="90a8674b-1ff8-4c23-8696-106d4c05c11d"
Jul 16 21:11:35.175252 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:35.175168    2576 scope.go:117] "RemoveContainer" containerID="d39bbca63bf3e40ad199b8fa240e1a2a63eae40f2660e1b0c77abdf8a15b44a1"
Jul 16 21:11:35.175670 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:11:35.175409    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw_odh-ai-gateway-infra(90a8674b-1ff8-4c23-8696-106d4c05c11d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw" podUID="90a8674b-1ff8-4c23-8696-106d4c05c11d"
Jul 16 21:11:46.684621 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:46.684584    2576 scope.go:117] "RemoveContainer" containerID="d39bbca63bf3e40ad199b8fa240e1a2a63eae40f2660e1b0c77abdf8a15b44a1"
Jul 16 21:11:47.237776 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:47.237746    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw" event={"ID":"90a8674b-1ff8-4c23-8696-106d4c05c11d","Type":"ContainerStarted","Data":"148125c9ee99cade387ec50e1418a10df8e2587485be8c519e5a594ae52098a8"}
Jul 16 21:11:48.248135 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:48.248094    2576 generic.go:356] "Generic (PLEG): container finished" podID="90a8674b-1ff8-4c23-8696-106d4c05c11d" containerID="148125c9ee99cade387ec50e1418a10df8e2587485be8c519e5a594ae52098a8" exitCode=6
Jul 16 21:11:48.248635 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:48.248161    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw" event={"ID":"90a8674b-1ff8-4c23-8696-106d4c05c11d","Type":"ContainerDied","Data":"148125c9ee99cade387ec50e1418a10df8e2587485be8c519e5a594ae52098a8"}
Jul 16 21:11:48.248635 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:48.248234    2576 scope.go:117] "RemoveContainer" containerID="d39bbca63bf3e40ad199b8fa240e1a2a63eae40f2660e1b0c77abdf8a15b44a1"
Jul 16 21:11:48.248635 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:48.248580    2576 scope.go:117] "RemoveContainer" containerID="148125c9ee99cade387ec50e1418a10df8e2587485be8c519e5a594ae52098a8"
Jul 16 21:11:48.248815 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:11:48.248791    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw_odh-ai-gateway-infra(90a8674b-1ff8-4c23-8696-106d4c05c11d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw" podUID="90a8674b-1ff8-4c23-8696-106d4c05c11d"
Jul 16 21:11:48.271634 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:48.271608    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw"]
Jul 16 21:11:49.386301 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:49.386282    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw"
Jul 16 21:11:49.425573 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:49.425534    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90a8674b-1ff8-4c23-8696-106d4c05c11d-maas-api-service-ca\") pod \"90a8674b-1ff8-4c23-8696-106d4c05c11d\" (UID: \"90a8674b-1ff8-4c23-8696-106d4c05c11d\") "
Jul 16 21:11:49.425863 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:49.425842    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90a8674b-1ff8-4c23-8696-106d4c05c11d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "90a8674b-1ff8-4c23-8696-106d4c05c11d" (UID: "90a8674b-1ff8-4c23-8696-106d4c05c11d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:11:49.527295 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:49.527240    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/90a8674b-1ff8-4c23-8696-106d4c05c11d-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:11:50.261641 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:50.261608    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw"
Jul 16 21:11:50.261804 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:50.261610    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw" event={"ID":"90a8674b-1ff8-4c23-8696-106d4c05c11d","Type":"ContainerDied","Data":"4e2b8aa7036e1791e440f5cddce4660e79df25c9bb0f7ecfa1def14958962a7a"}
Jul 16 21:11:50.261804 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:50.261746    2576 scope.go:117] "RemoveContainer" containerID="148125c9ee99cade387ec50e1418a10df8e2587485be8c519e5a594ae52098a8"
Jul 16 21:11:50.284388 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:50.284344    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw"]
Jul 16 21:11:50.290213 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:50.290189    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-q4mxw"]
Jul 16 21:11:50.689171 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:11:50.689142    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90a8674b-1ff8-4c23-8696-106d4c05c11d" path="/var/lib/kubelet/pods/90a8674b-1ff8-4c23-8696-106d4c05c11d/volumes"
Jul 16 21:12:03.789424 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.789380    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc"]
Jul 16 21:12:03.789965 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.789793    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90a8674b-1ff8-4c23-8696-106d4c05c11d" containerName="revoke-keys"
Jul 16 21:12:03.789965 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.789806    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="90a8674b-1ff8-4c23-8696-106d4c05c11d" containerName="revoke-keys"
Jul 16 21:12:03.789965 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.789826    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90a8674b-1ff8-4c23-8696-106d4c05c11d" containerName="revoke-keys"
Jul 16 21:12:03.789965 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.789831    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="90a8674b-1ff8-4c23-8696-106d4c05c11d" containerName="revoke-keys"
Jul 16 21:12:03.789965 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.789840    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099a38c3-c0e3-4356-a905-9d11845b5ebd" containerName="revoke-keys"
Jul 16 21:12:03.789965 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.789846    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="099a38c3-c0e3-4356-a905-9d11845b5ebd" containerName="revoke-keys"
Jul 16 21:12:03.789965 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.789853    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90a8674b-1ff8-4c23-8696-106d4c05c11d" containerName="revoke-keys"
Jul 16 21:12:03.789965 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.789859    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="90a8674b-1ff8-4c23-8696-106d4c05c11d" containerName="revoke-keys"
Jul 16 21:12:03.789965 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.789929    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="90a8674b-1ff8-4c23-8696-106d4c05c11d" containerName="revoke-keys"
Jul 16 21:12:03.789965 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.789940    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="90a8674b-1ff8-4c23-8696-106d4c05c11d" containerName="revoke-keys"
Jul 16 21:12:03.789965 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.789946    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="90a8674b-1ff8-4c23-8696-106d4c05c11d" containerName="revoke-keys"
Jul 16 21:12:03.792906 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.792883    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc"
Jul 16 21:12:03.795437 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.795410    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:12:03.795587 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.795410    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:12:03.799812 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.799783    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc"]
Jul 16 21:12:03.827516 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.827483    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/432f3f42-7386-43b2-9eb8-5b30cdc0eb03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc\" (UID: \"432f3f42-7386-43b2-9eb8-5b30cdc0eb03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc"
Jul 16 21:12:03.928006 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.927977    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/432f3f42-7386-43b2-9eb8-5b30cdc0eb03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc\" (UID: \"432f3f42-7386-43b2-9eb8-5b30cdc0eb03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc"
Jul 16 21:12:03.928638 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:03.928614    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/432f3f42-7386-43b2-9eb8-5b30cdc0eb03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc\" (UID: \"432f3f42-7386-43b2-9eb8-5b30cdc0eb03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc"
Jul 16 21:12:04.106192 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:04.106129    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc"
Jul 16 21:12:04.241103 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:04.241076    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc"]
Jul 16 21:12:04.243174 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:12:04.243147    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod432f3f42_7386_43b2_9eb8_5b30cdc0eb03.slice/crio-3ce7830dad75727fe5771a5494edc85f3c5f0da2f27fed077540db182360c210 WatchSource:0}: Error finding container 3ce7830dad75727fe5771a5494edc85f3c5f0da2f27fed077540db182360c210: Status 404 returned error can't find the container with id 3ce7830dad75727fe5771a5494edc85f3c5f0da2f27fed077540db182360c210
Jul 16 21:12:04.333922 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:04.333897    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc" event={"ID":"432f3f42-7386-43b2-9eb8-5b30cdc0eb03","Type":"ContainerStarted","Data":"3ce7830dad75727fe5771a5494edc85f3c5f0da2f27fed077540db182360c210"}
Jul 16 21:12:05.340654 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:05.340625    2576 generic.go:356] "Generic (PLEG): container finished" podID="432f3f42-7386-43b2-9eb8-5b30cdc0eb03" containerID="a490c511a0f1cfe20386b6c0133bb044b1e6e5049e763130cf5896c63bfd1ed3" exitCode=6
Jul 16 21:12:05.340998 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:05.340723    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc" event={"ID":"432f3f42-7386-43b2-9eb8-5b30cdc0eb03","Type":"ContainerDied","Data":"a490c511a0f1cfe20386b6c0133bb044b1e6e5049e763130cf5896c63bfd1ed3"}
Jul 16 21:12:05.340998 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:05.340895    2576 scope.go:117] "RemoveContainer" containerID="a490c511a0f1cfe20386b6c0133bb044b1e6e5049e763130cf5896c63bfd1ed3"
Jul 16 21:12:06.348461 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:06.348343    2576 generic.go:356] "Generic (PLEG): container finished" podID="432f3f42-7386-43b2-9eb8-5b30cdc0eb03" containerID="decfa7cfc7d5a85ace15b58aabfaa505a4a18006f3616181a6b9e3662216b7cd" exitCode=6
Jul 16 21:12:06.348461 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:06.348441    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc" event={"ID":"432f3f42-7386-43b2-9eb8-5b30cdc0eb03","Type":"ContainerDied","Data":"decfa7cfc7d5a85ace15b58aabfaa505a4a18006f3616181a6b9e3662216b7cd"}
Jul 16 21:12:06.348937 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:06.348512    2576 scope.go:117] "RemoveContainer" containerID="a490c511a0f1cfe20386b6c0133bb044b1e6e5049e763130cf5896c63bfd1ed3"
Jul 16 21:12:06.348937 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:06.348686    2576 scope.go:117] "RemoveContainer" containerID="decfa7cfc7d5a85ace15b58aabfaa505a4a18006f3616181a6b9e3662216b7cd"
Jul 16 21:12:06.348937 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:12:06.348882    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc_odh-ai-gateway-infra(432f3f42-7386-43b2-9eb8-5b30cdc0eb03)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc" podUID="432f3f42-7386-43b2-9eb8-5b30cdc0eb03"
Jul 16 21:12:07.355257 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:07.355229    2576 scope.go:117] "RemoveContainer" containerID="decfa7cfc7d5a85ace15b58aabfaa505a4a18006f3616181a6b9e3662216b7cd"
Jul 16 21:12:07.355674 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:12:07.355436    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc_odh-ai-gateway-infra(432f3f42-7386-43b2-9eb8-5b30cdc0eb03)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc" podUID="432f3f42-7386-43b2-9eb8-5b30cdc0eb03"
Jul 16 21:12:21.684559 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:21.684525    2576 scope.go:117] "RemoveContainer" containerID="decfa7cfc7d5a85ace15b58aabfaa505a4a18006f3616181a6b9e3662216b7cd"
Jul 16 21:12:22.435000 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:22.434963    2576 generic.go:356] "Generic (PLEG): container finished" podID="432f3f42-7386-43b2-9eb8-5b30cdc0eb03" containerID="97eb23f753b6ed839dc2bbc2adea30a043937919c6fbda1015cdb0b0460e68af" exitCode=6
Jul 16 21:12:22.435166 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:22.435039    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc" event={"ID":"432f3f42-7386-43b2-9eb8-5b30cdc0eb03","Type":"ContainerDied","Data":"97eb23f753b6ed839dc2bbc2adea30a043937919c6fbda1015cdb0b0460e68af"}
Jul 16 21:12:22.435166 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:22.435143    2576 scope.go:117] "RemoveContainer" containerID="decfa7cfc7d5a85ace15b58aabfaa505a4a18006f3616181a6b9e3662216b7cd"
Jul 16 21:12:22.435533 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:22.435511    2576 scope.go:117] "RemoveContainer" containerID="97eb23f753b6ed839dc2bbc2adea30a043937919c6fbda1015cdb0b0460e68af"
Jul 16 21:12:22.436294 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:12:22.436144    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc_odh-ai-gateway-infra(432f3f42-7386-43b2-9eb8-5b30cdc0eb03)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc" podUID="432f3f42-7386-43b2-9eb8-5b30cdc0eb03"
Jul 16 21:12:23.461701 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:23.461664    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc"]
Jul 16 21:12:23.599086 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:23.599064    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc"
Jul 16 21:12:23.680381 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:23.680335    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/432f3f42-7386-43b2-9eb8-5b30cdc0eb03-maas-api-service-ca\") pod \"432f3f42-7386-43b2-9eb8-5b30cdc0eb03\" (UID: \"432f3f42-7386-43b2-9eb8-5b30cdc0eb03\") "
Jul 16 21:12:23.680679 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:23.680656    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/432f3f42-7386-43b2-9eb8-5b30cdc0eb03-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "432f3f42-7386-43b2-9eb8-5b30cdc0eb03" (UID: "432f3f42-7386-43b2-9eb8-5b30cdc0eb03"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:12:23.781228 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:23.781177    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/432f3f42-7386-43b2-9eb8-5b30cdc0eb03-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:12:24.449481 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:24.449454    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc"
Jul 16 21:12:24.449661 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:24.449486    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc" event={"ID":"432f3f42-7386-43b2-9eb8-5b30cdc0eb03","Type":"ContainerDied","Data":"3ce7830dad75727fe5771a5494edc85f3c5f0da2f27fed077540db182360c210"}
Jul 16 21:12:24.449661 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:24.449568    2576 scope.go:117] "RemoveContainer" containerID="97eb23f753b6ed839dc2bbc2adea30a043937919c6fbda1015cdb0b0460e68af"
Jul 16 21:12:24.474021 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:24.473979    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc"]
Jul 16 21:12:24.481027 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:24.481004    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-tv7lc"]
Jul 16 21:12:24.690245 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:24.690212    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="432f3f42-7386-43b2-9eb8-5b30cdc0eb03" path="/var/lib/kubelet/pods/432f3f42-7386-43b2-9eb8-5b30cdc0eb03/volumes"
Jul 16 21:12:34.640322 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:34.640292    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 21:12:34.645332 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:34.645306    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 21:12:59.061374 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:59.061282    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq"]
Jul 16 21:12:59.061856 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:59.061725    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="432f3f42-7386-43b2-9eb8-5b30cdc0eb03" containerName="revoke-keys"
Jul 16 21:12:59.061856 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:59.061742    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="432f3f42-7386-43b2-9eb8-5b30cdc0eb03" containerName="revoke-keys"
Jul 16 21:12:59.061856 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:59.061764    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="432f3f42-7386-43b2-9eb8-5b30cdc0eb03" containerName="revoke-keys"
Jul 16 21:12:59.061856 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:59.061770    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="432f3f42-7386-43b2-9eb8-5b30cdc0eb03" containerName="revoke-keys"
Jul 16 21:12:59.061856 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:59.061832    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="432f3f42-7386-43b2-9eb8-5b30cdc0eb03" containerName="revoke-keys"
Jul 16 21:12:59.061856 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:59.061841    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="432f3f42-7386-43b2-9eb8-5b30cdc0eb03" containerName="revoke-keys"
Jul 16 21:12:59.065116 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:59.065098    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq"
Jul 16 21:12:59.067555 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:59.067532    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:12:59.067680 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:59.067575    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:12:59.072279 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:59.072035    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq"]
Jul 16 21:12:59.167215 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:59.167189    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d070447-1ca3-4556-9d48-2bd01f386110-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq\" (UID: \"4d070447-1ca3-4556-9d48-2bd01f386110\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq"
Jul 16 21:12:59.268701 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:59.268672    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d070447-1ca3-4556-9d48-2bd01f386110-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq\" (UID: \"4d070447-1ca3-4556-9d48-2bd01f386110\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq"
Jul 16 21:12:59.269506 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:59.269467    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d070447-1ca3-4556-9d48-2bd01f386110-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq\" (UID: \"4d070447-1ca3-4556-9d48-2bd01f386110\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq"
Jul 16 21:12:59.378246 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:59.378187    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq"
Jul 16 21:12:59.512934 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:59.512908    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq"]
Jul 16 21:12:59.515671 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:12:59.515643    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d070447_1ca3_4556_9d48_2bd01f386110.slice/crio-9ba80dec861791126b495b7658f9399d23adf96013cd2b369e1e951bae8cc002 WatchSource:0}: Error finding container 9ba80dec861791126b495b7658f9399d23adf96013cd2b369e1e951bae8cc002: Status 404 returned error can't find the container with id 9ba80dec861791126b495b7658f9399d23adf96013cd2b369e1e951bae8cc002
Jul 16 21:12:59.634795 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:12:59.634742    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq" event={"ID":"4d070447-1ca3-4556-9d48-2bd01f386110","Type":"ContainerStarted","Data":"9ba80dec861791126b495b7658f9399d23adf96013cd2b369e1e951bae8cc002"}
Jul 16 21:13:00.641104 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:00.641063    2576 generic.go:356] "Generic (PLEG): container finished" podID="4d070447-1ca3-4556-9d48-2bd01f386110" containerID="f157abf62b098ca7a6e608b7ebbd856c066663f5e09c6a0a7e9ae2dad14261db" exitCode=7
Jul 16 21:13:00.641483 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:00.641136    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq" event={"ID":"4d070447-1ca3-4556-9d48-2bd01f386110","Type":"ContainerDied","Data":"f157abf62b098ca7a6e608b7ebbd856c066663f5e09c6a0a7e9ae2dad14261db"}
Jul 16 21:13:00.641483 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:00.641390    2576 scope.go:117] "RemoveContainer" containerID="f157abf62b098ca7a6e608b7ebbd856c066663f5e09c6a0a7e9ae2dad14261db"
Jul 16 21:13:01.648789 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:01.648759    2576 generic.go:356] "Generic (PLEG): container finished" podID="4d070447-1ca3-4556-9d48-2bd01f386110" containerID="f28c48999ad7af5eb4f1ab5a43a5fc3a7472e1779607a57d3c14eb3a032249d9" exitCode=7
Jul 16 21:13:01.649190 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:01.648830    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq" event={"ID":"4d070447-1ca3-4556-9d48-2bd01f386110","Type":"ContainerDied","Data":"f28c48999ad7af5eb4f1ab5a43a5fc3a7472e1779607a57d3c14eb3a032249d9"}
Jul 16 21:13:01.649190 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:01.648876    2576 scope.go:117] "RemoveContainer" containerID="f157abf62b098ca7a6e608b7ebbd856c066663f5e09c6a0a7e9ae2dad14261db"
Jul 16 21:13:01.649190 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:01.649025    2576 scope.go:117] "RemoveContainer" containerID="f28c48999ad7af5eb4f1ab5a43a5fc3a7472e1779607a57d3c14eb3a032249d9"
Jul 16 21:13:01.649393 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:13:01.649249    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq_odh-ai-gateway-infra(4d070447-1ca3-4556-9d48-2bd01f386110)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq" podUID="4d070447-1ca3-4556-9d48-2bd01f386110"
Jul 16 21:13:02.656207 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:02.656175    2576 scope.go:117] "RemoveContainer" containerID="f28c48999ad7af5eb4f1ab5a43a5fc3a7472e1779607a57d3c14eb3a032249d9"
Jul 16 21:13:02.656636 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:13:02.656337    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq_odh-ai-gateway-infra(4d070447-1ca3-4556-9d48-2bd01f386110)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq" podUID="4d070447-1ca3-4556-9d48-2bd01f386110"
Jul 16 21:13:13.683940 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:13.683910    2576 scope.go:117] "RemoveContainer" containerID="f28c48999ad7af5eb4f1ab5a43a5fc3a7472e1779607a57d3c14eb3a032249d9"
Jul 16 21:13:14.719326 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:14.719297    2576 generic.go:356] "Generic (PLEG): container finished" podID="4d070447-1ca3-4556-9d48-2bd01f386110" containerID="932ade6d164d415c9931f1100baf3310149821e7b1613f1b9d6c1a5b4dbb6ffa" exitCode=7
Jul 16 21:13:14.719699 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:14.719391    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq" event={"ID":"4d070447-1ca3-4556-9d48-2bd01f386110","Type":"ContainerDied","Data":"932ade6d164d415c9931f1100baf3310149821e7b1613f1b9d6c1a5b4dbb6ffa"}
Jul 16 21:13:14.719699 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:14.719453    2576 scope.go:117] "RemoveContainer" containerID="f28c48999ad7af5eb4f1ab5a43a5fc3a7472e1779607a57d3c14eb3a032249d9"
Jul 16 21:13:14.719795 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:14.719740    2576 scope.go:117] "RemoveContainer" containerID="932ade6d164d415c9931f1100baf3310149821e7b1613f1b9d6c1a5b4dbb6ffa"
Jul 16 21:13:14.719959 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:13:14.719935    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq_odh-ai-gateway-infra(4d070447-1ca3-4556-9d48-2bd01f386110)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq" podUID="4d070447-1ca3-4556-9d48-2bd01f386110"
Jul 16 21:13:15.745266 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:15.745232    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq"]
Jul 16 21:13:15.891707 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:15.891686    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq"
Jul 16 21:13:16.001545 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:16.001475    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d070447-1ca3-4556-9d48-2bd01f386110-maas-api-service-ca\") pod \"4d070447-1ca3-4556-9d48-2bd01f386110\" (UID: \"4d070447-1ca3-4556-9d48-2bd01f386110\") "
Jul 16 21:13:16.001774 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:16.001752    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d070447-1ca3-4556-9d48-2bd01f386110-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4d070447-1ca3-4556-9d48-2bd01f386110" (UID: "4d070447-1ca3-4556-9d48-2bd01f386110"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:13:16.103049 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:16.103024    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d070447-1ca3-4556-9d48-2bd01f386110-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:13:16.731994 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:16.731966    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq"
Jul 16 21:13:16.732123 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:16.732002    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq" event={"ID":"4d070447-1ca3-4556-9d48-2bd01f386110","Type":"ContainerDied","Data":"9ba80dec861791126b495b7658f9399d23adf96013cd2b369e1e951bae8cc002"}
Jul 16 21:13:16.732123 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:16.732068    2576 scope.go:117] "RemoveContainer" containerID="932ade6d164d415c9931f1100baf3310149821e7b1613f1b9d6c1a5b4dbb6ffa"
Jul 16 21:13:16.748494 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:16.748471    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq"]
Jul 16 21:13:16.754001 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:16.753976    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-d9ckq"]
Jul 16 21:13:18.690864 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:18.690830    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d070447-1ca3-4556-9d48-2bd01f386110" path="/var/lib/kubelet/pods/4d070447-1ca3-4556-9d48-2bd01f386110/volumes"
Jul 16 21:13:19.044769 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:19.044690    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"]
Jul 16 21:13:19.044923 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:19.044872    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp" podUID="5e3cef2c-0cc6-4855-8f97-fea412ed5381" containerName="istio-proxy" containerID="cri-o://84adc9b03354d3503c5a7f61443821a62ffd488240244a74846f96991ec12079" gracePeriod=30
Jul 16 21:13:20.672396 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:20.672340    2576 patch_prober.go:28] interesting pod/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 21:13:20.672836 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:20.672419    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp" podUID="5e3cef2c-0cc6-4855-8f97-fea412ed5381" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 21:13:22.108375 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:22.108330    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:13:24.295882 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.295859    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:13:24.370710 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.370681    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-envoy\") pod \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") "
Jul 16 21:13:24.370843 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.370764    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-workload-certs\") pod \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") "
Jul 16 21:13:24.370843 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.370817    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-podinfo\") pod \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") "
Jul 16 21:13:24.370966 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.370841    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-token\") pod \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") "
Jul 16 21:13:24.370966 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.370867    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w92s9\" (UniqueName: \"kubernetes.io/projected/5e3cef2c-0cc6-4855-8f97-fea412ed5381-kube-api-access-w92s9\") pod \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") "
Jul 16 21:13:24.370966 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.370911    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-data\") pod \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") "
Jul 16 21:13:24.371128 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.370996    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-workload-socket\") pod \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") "
Jul 16 21:13:24.371128 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.371010    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5e3cef2c-0cc6-4855-8f97-fea412ed5381" (UID: "5e3cef2c-0cc6-4855-8f97-fea412ed5381"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:13:24.371128 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.371043    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istiod-ca-cert\") pod \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") "
Jul 16 21:13:24.371128 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.371073    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-credential-socket\") pod \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\" (UID: \"5e3cef2c-0cc6-4855-8f97-fea412ed5381\") "
Jul 16 21:13:24.371341 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.371303    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5e3cef2c-0cc6-4855-8f97-fea412ed5381" (UID: "5e3cef2c-0cc6-4855-8f97-fea412ed5381"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:13:24.371428 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.371406    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5e3cef2c-0cc6-4855-8f97-fea412ed5381" (UID: "5e3cef2c-0cc6-4855-8f97-fea412ed5381"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:13:24.371531 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.371510    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-workload-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:13:24.371531 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.371518    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5e3cef2c-0cc6-4855-8f97-fea412ed5381" (UID: "5e3cef2c-0cc6-4855-8f97-fea412ed5381"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:13:24.371647 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.371536    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-workload-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:13:24.371815 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.371780    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-data" (OuterVolumeSpecName: "istio-data") pod "5e3cef2c-0cc6-4855-8f97-fea412ed5381" (UID: "5e3cef2c-0cc6-4855-8f97-fea412ed5381"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:13:24.372969 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.372946    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5e3cef2c-0cc6-4855-8f97-fea412ed5381" (UID: "5e3cef2c-0cc6-4855-8f97-fea412ed5381"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:13:24.373069 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.373021    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e3cef2c-0cc6-4855-8f97-fea412ed5381-kube-api-access-w92s9" (OuterVolumeSpecName: "kube-api-access-w92s9") pod "5e3cef2c-0cc6-4855-8f97-fea412ed5381" (UID: "5e3cef2c-0cc6-4855-8f97-fea412ed5381"). InnerVolumeSpecName "kube-api-access-w92s9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:13:24.373259 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.373243    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5e3cef2c-0cc6-4855-8f97-fea412ed5381" (UID: "5e3cef2c-0cc6-4855-8f97-fea412ed5381"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 21:13:24.373729 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.373715    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-token" (OuterVolumeSpecName: "istio-token") pod "5e3cef2c-0cc6-4855-8f97-fea412ed5381" (UID: "5e3cef2c-0cc6-4855-8f97-fea412ed5381"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:13:24.472066 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.472042    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-data\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:13:24.472066 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.472064    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istiod-ca-cert\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:13:24.472192 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.472073    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-credential-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:13:24.472192 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.472083    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-envoy\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:13:24.472192 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.472092    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-podinfo\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:13:24.472192 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.472100    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e3cef2c-0cc6-4855-8f97-fea412ed5381-istio-token\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:13:24.472192 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.472109    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w92s9\" (UniqueName: \"kubernetes.io/projected/5e3cef2c-0cc6-4855-8f97-fea412ed5381-kube-api-access-w92s9\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:13:24.774886 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.774855    2576 generic.go:356] "Generic (PLEG): container finished" podID="5e3cef2c-0cc6-4855-8f97-fea412ed5381" containerID="84adc9b03354d3503c5a7f61443821a62ffd488240244a74846f96991ec12079" exitCode=0
Jul 16 21:13:24.774996 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.774907    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"
Jul 16 21:13:24.774996 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.774940    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp" event={"ID":"5e3cef2c-0cc6-4855-8f97-fea412ed5381","Type":"ContainerDied","Data":"84adc9b03354d3503c5a7f61443821a62ffd488240244a74846f96991ec12079"}
Jul 16 21:13:24.775070 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.774997    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp" event={"ID":"5e3cef2c-0cc6-4855-8f97-fea412ed5381","Type":"ContainerDied","Data":"cd744bee58b5cf7ed0e54225bbf8bcf113147fd2dc6af2023b6d99cf56220ae5"}
Jul 16 21:13:24.775070 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.775024    2576 scope.go:117] "RemoveContainer" containerID="84adc9b03354d3503c5a7f61443821a62ffd488240244a74846f96991ec12079"
Jul 16 21:13:24.783522 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.783506    2576 scope.go:117] "RemoveContainer" containerID="8ad5f5dc19ccb165a238563f64e301c258f323c52e8172c705453166499d1323"
Jul 16 21:13:24.791510 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.791490    2576 scope.go:117] "RemoveContainer" containerID="84adc9b03354d3503c5a7f61443821a62ffd488240244a74846f96991ec12079"
Jul 16 21:13:24.791790 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:13:24.791771    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84adc9b03354d3503c5a7f61443821a62ffd488240244a74846f96991ec12079\": container with ID starting with 84adc9b03354d3503c5a7f61443821a62ffd488240244a74846f96991ec12079 not found: ID does not exist" containerID="84adc9b03354d3503c5a7f61443821a62ffd488240244a74846f96991ec12079"
Jul 16 21:13:24.791853 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.791799    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84adc9b03354d3503c5a7f61443821a62ffd488240244a74846f96991ec12079"} err="failed to get container status \"84adc9b03354d3503c5a7f61443821a62ffd488240244a74846f96991ec12079\": rpc error: code = NotFound desc = could not find container \"84adc9b03354d3503c5a7f61443821a62ffd488240244a74846f96991ec12079\": container with ID starting with 84adc9b03354d3503c5a7f61443821a62ffd488240244a74846f96991ec12079 not found: ID does not exist"
Jul 16 21:13:24.791853 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.791815    2576 scope.go:117] "RemoveContainer" containerID="8ad5f5dc19ccb165a238563f64e301c258f323c52e8172c705453166499d1323"
Jul 16 21:13:24.792036 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:13:24.792012    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ad5f5dc19ccb165a238563f64e301c258f323c52e8172c705453166499d1323\": container with ID starting with 8ad5f5dc19ccb165a238563f64e301c258f323c52e8172c705453166499d1323 not found: ID does not exist" containerID="8ad5f5dc19ccb165a238563f64e301c258f323c52e8172c705453166499d1323"
Jul 16 21:13:24.792086 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.792034    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ad5f5dc19ccb165a238563f64e301c258f323c52e8172c705453166499d1323"} err="failed to get container status \"8ad5f5dc19ccb165a238563f64e301c258f323c52e8172c705453166499d1323\": rpc error: code = NotFound desc = could not find container \"8ad5f5dc19ccb165a238563f64e301c258f323c52e8172c705453166499d1323\": container with ID starting with 8ad5f5dc19ccb165a238563f64e301c258f323c52e8172c705453166499d1323 not found: ID does not exist"
Jul 16 21:13:24.793278 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.793260    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"]
Jul 16 21:13:24.798573 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:24.798555    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-90b68ec7-openshift-default-84b54bf77f-w9sbp"]
Jul 16 21:13:26.690405 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:26.690351    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e3cef2c-0cc6-4855-8f97-fea412ed5381" path="/var/lib/kubelet/pods/5e3cef2c-0cc6-4855-8f97-fea412ed5381/volumes"
Jul 16 21:13:49.105566 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.105521    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t"]
Jul 16 21:13:49.106107 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.106080    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="432f3f42-7386-43b2-9eb8-5b30cdc0eb03" containerName="revoke-keys"
Jul 16 21:13:49.106197 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.106110    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="432f3f42-7386-43b2-9eb8-5b30cdc0eb03" containerName="revoke-keys"
Jul 16 21:13:49.106197 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.106129    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d070447-1ca3-4556-9d48-2bd01f386110" containerName="revoke-keys"
Jul 16 21:13:49.106197 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.106137    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d070447-1ca3-4556-9d48-2bd01f386110" containerName="revoke-keys"
Jul 16 21:13:49.106197 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.106170    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e3cef2c-0cc6-4855-8f97-fea412ed5381" containerName="istio-proxy"
Jul 16 21:13:49.106197 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.106180    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e3cef2c-0cc6-4855-8f97-fea412ed5381" containerName="istio-proxy"
Jul 16 21:13:49.106197 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.106191    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e3cef2c-0cc6-4855-8f97-fea412ed5381" containerName="istio-proxy"
Jul 16 21:13:49.106525 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.106201    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e3cef2c-0cc6-4855-8f97-fea412ed5381" containerName="istio-proxy"
Jul 16 21:13:49.106525 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.106216    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d070447-1ca3-4556-9d48-2bd01f386110" containerName="revoke-keys"
Jul 16 21:13:49.106525 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.106225    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d070447-1ca3-4556-9d48-2bd01f386110" containerName="revoke-keys"
Jul 16 21:13:49.106525 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.106326    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e3cef2c-0cc6-4855-8f97-fea412ed5381" containerName="istio-proxy"
Jul 16 21:13:49.106525 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.106340    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d070447-1ca3-4556-9d48-2bd01f386110" containerName="revoke-keys"
Jul 16 21:13:49.106525 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.106352    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="432f3f42-7386-43b2-9eb8-5b30cdc0eb03" containerName="revoke-keys"
Jul 16 21:13:49.106525 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.106382    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d070447-1ca3-4556-9d48-2bd01f386110" containerName="revoke-keys"
Jul 16 21:13:49.106525 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.106394    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d070447-1ca3-4556-9d48-2bd01f386110" containerName="revoke-keys"
Jul 16 21:13:49.109451 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.109417    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t"
Jul 16 21:13:49.111920 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.111892    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:13:49.112075 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.112053    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:13:49.115251 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.115202    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t"]
Jul 16 21:13:49.271574 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.271551    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee193d4e-4b45-4885-8765-8b13e2fe9a70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t\" (UID: \"ee193d4e-4b45-4885-8765-8b13e2fe9a70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t"
Jul 16 21:13:49.372033 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.371951    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee193d4e-4b45-4885-8765-8b13e2fe9a70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t\" (UID: \"ee193d4e-4b45-4885-8765-8b13e2fe9a70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t"
Jul 16 21:13:49.372591 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.372572    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee193d4e-4b45-4885-8765-8b13e2fe9a70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t\" (UID: \"ee193d4e-4b45-4885-8765-8b13e2fe9a70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t"
Jul 16 21:13:49.422553 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.422532    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t"
Jul 16 21:13:49.551234 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.551212    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t"]
Jul 16 21:13:49.553174 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:13:49.553145    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee193d4e_4b45_4885_8765_8b13e2fe9a70.slice/crio-055bd7904925f94f900707ef2805f8989c512c9f5bd75c358281f253ab166ee3 WatchSource:0}: Error finding container 055bd7904925f94f900707ef2805f8989c512c9f5bd75c358281f253ab166ee3: Status 404 returned error can't find the container with id 055bd7904925f94f900707ef2805f8989c512c9f5bd75c358281f253ab166ee3
Jul 16 21:13:49.895975 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.895948    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t" event={"ID":"ee193d4e-4b45-4885-8765-8b13e2fe9a70","Type":"ContainerStarted","Data":"604757192d6b0f82575a1be307151bda1049410d659df49f48e53361e03af950"}
Jul 16 21:13:49.896075 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.896033    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t" event={"ID":"ee193d4e-4b45-4885-8765-8b13e2fe9a70","Type":"ContainerStarted","Data":"055bd7904925f94f900707ef2805f8989c512c9f5bd75c358281f253ab166ee3"}
Jul 16 21:13:49.914494 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:49.914451    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t" podStartSLOduration=0.91443403 podStartE2EDuration="914.43403ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:13:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:13:49.913217643 +0000 UTC m=+2475.879151887" watchObservedRunningTime="2026-07-16 21:13:49.91443403 +0000 UTC m=+2475.880368241"
Jul 16 21:13:50.902416 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:50.902378    2576 generic.go:356] "Generic (PLEG): container finished" podID="ee193d4e-4b45-4885-8765-8b13e2fe9a70" containerID="604757192d6b0f82575a1be307151bda1049410d659df49f48e53361e03af950" exitCode=6
Jul 16 21:13:50.902893 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:50.902455    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t" event={"ID":"ee193d4e-4b45-4885-8765-8b13e2fe9a70","Type":"ContainerDied","Data":"604757192d6b0f82575a1be307151bda1049410d659df49f48e53361e03af950"}
Jul 16 21:13:50.902893 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:50.902654    2576 scope.go:117] "RemoveContainer" containerID="604757192d6b0f82575a1be307151bda1049410d659df49f48e53361e03af950"
Jul 16 21:13:51.910296 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:51.910268    2576 generic.go:356] "Generic (PLEG): container finished" podID="ee193d4e-4b45-4885-8765-8b13e2fe9a70" containerID="87748cc9f44f318fe98d7c4837e9618124e43d7f30f26bfeef8b8e965c37b115" exitCode=6
Jul 16 21:13:51.910702 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:51.910328    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t" event={"ID":"ee193d4e-4b45-4885-8765-8b13e2fe9a70","Type":"ContainerDied","Data":"87748cc9f44f318fe98d7c4837e9618124e43d7f30f26bfeef8b8e965c37b115"}
Jul 16 21:13:51.910702 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:51.910407    2576 scope.go:117] "RemoveContainer" containerID="604757192d6b0f82575a1be307151bda1049410d659df49f48e53361e03af950"
Jul 16 21:13:51.910702 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:51.910622    2576 scope.go:117] "RemoveContainer" containerID="87748cc9f44f318fe98d7c4837e9618124e43d7f30f26bfeef8b8e965c37b115"
Jul 16 21:13:51.910854 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:13:51.910800    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t_odh-ai-gateway-infra(ee193d4e-4b45-4885-8765-8b13e2fe9a70)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t" podUID="ee193d4e-4b45-4885-8765-8b13e2fe9a70"
Jul 16 21:13:52.917471 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:13:52.917424    2576 scope.go:117] "RemoveContainer" containerID="87748cc9f44f318fe98d7c4837e9618124e43d7f30f26bfeef8b8e965c37b115"
Jul 16 21:13:52.917874 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:13:52.917641    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t_odh-ai-gateway-infra(ee193d4e-4b45-4885-8765-8b13e2fe9a70)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t" podUID="ee193d4e-4b45-4885-8765-8b13e2fe9a70"
Jul 16 21:14:08.684248 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:08.684210    2576 scope.go:117] "RemoveContainer" containerID="87748cc9f44f318fe98d7c4837e9618124e43d7f30f26bfeef8b8e965c37b115"
Jul 16 21:14:08.996759 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:08.996726    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t" event={"ID":"ee193d4e-4b45-4885-8765-8b13e2fe9a70","Type":"ContainerStarted","Data":"8594fb4a43720716040616d0a4a6e601d0e9f9793787b0f418ce5f51c4869854"}
Jul 16 21:14:10.012249 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:10.012203    2576 generic.go:356] "Generic (PLEG): container finished" podID="ee193d4e-4b45-4885-8765-8b13e2fe9a70" containerID="8594fb4a43720716040616d0a4a6e601d0e9f9793787b0f418ce5f51c4869854" exitCode=6
Jul 16 21:14:10.012704 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:10.012283    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t" event={"ID":"ee193d4e-4b45-4885-8765-8b13e2fe9a70","Type":"ContainerDied","Data":"8594fb4a43720716040616d0a4a6e601d0e9f9793787b0f418ce5f51c4869854"}
Jul 16 21:14:10.012704 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:10.012411    2576 scope.go:117] "RemoveContainer" containerID="87748cc9f44f318fe98d7c4837e9618124e43d7f30f26bfeef8b8e965c37b115"
Jul 16 21:14:10.012704 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:10.012670    2576 scope.go:117] "RemoveContainer" containerID="8594fb4a43720716040616d0a4a6e601d0e9f9793787b0f418ce5f51c4869854"
Jul 16 21:14:10.012921 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:14:10.012893    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t_odh-ai-gateway-infra(ee193d4e-4b45-4885-8765-8b13e2fe9a70)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t" podUID="ee193d4e-4b45-4885-8765-8b13e2fe9a70"
Jul 16 21:14:10.023873 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:10.023843    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t"]
Jul 16 21:14:11.152638 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.152616    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t"
Jul 16 21:14:11.234230 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.234203    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns"]
Jul 16 21:14:11.234582 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.234569    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee193d4e-4b45-4885-8765-8b13e2fe9a70" containerName="revoke-keys"
Jul 16 21:14:11.234639 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.234584    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee193d4e-4b45-4885-8765-8b13e2fe9a70" containerName="revoke-keys"
Jul 16 21:14:11.234639 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.234596    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d070447-1ca3-4556-9d48-2bd01f386110" containerName="revoke-keys"
Jul 16 21:14:11.234639 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.234602    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d070447-1ca3-4556-9d48-2bd01f386110" containerName="revoke-keys"
Jul 16 21:14:11.234639 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.234613    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee193d4e-4b45-4885-8765-8b13e2fe9a70" containerName="revoke-keys"
Jul 16 21:14:11.234639 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.234618    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee193d4e-4b45-4885-8765-8b13e2fe9a70" containerName="revoke-keys"
Jul 16 21:14:11.234796 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.234675    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee193d4e-4b45-4885-8765-8b13e2fe9a70" containerName="revoke-keys"
Jul 16 21:14:11.234796 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.234684    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e3cef2c-0cc6-4855-8f97-fea412ed5381" containerName="istio-proxy"
Jul 16 21:14:11.234796 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.234693    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee193d4e-4b45-4885-8765-8b13e2fe9a70" containerName="revoke-keys"
Jul 16 21:14:11.237652 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.237637    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns"
Jul 16 21:14:11.238597 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.238581    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee193d4e-4b45-4885-8765-8b13e2fe9a70-maas-api-service-ca\") pod \"ee193d4e-4b45-4885-8765-8b13e2fe9a70\" (UID: \"ee193d4e-4b45-4885-8765-8b13e2fe9a70\") "
Jul 16 21:14:11.238901 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.238883    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee193d4e-4b45-4885-8765-8b13e2fe9a70-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ee193d4e-4b45-4885-8765-8b13e2fe9a70" (UID: "ee193d4e-4b45-4885-8765-8b13e2fe9a70"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:14:11.245478 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.245453    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns"]
Jul 16 21:14:11.339154 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.339084    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dab0dac-7220-4eb2-8e8d-6ec91e134a58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-z7wns\" (UID: \"9dab0dac-7220-4eb2-8e8d-6ec91e134a58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns"
Jul 16 21:14:11.339267 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.339199    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee193d4e-4b45-4885-8765-8b13e2fe9a70-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:14:11.439910 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.439880    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dab0dac-7220-4eb2-8e8d-6ec91e134a58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-z7wns\" (UID: \"9dab0dac-7220-4eb2-8e8d-6ec91e134a58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns"
Jul 16 21:14:11.440462 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.440444    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dab0dac-7220-4eb2-8e8d-6ec91e134a58-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-z7wns\" (UID: \"9dab0dac-7220-4eb2-8e8d-6ec91e134a58\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns"
Jul 16 21:14:11.548951 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.548927    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns"
Jul 16 21:14:11.674354 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:11.674299    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns"]
Jul 16 21:14:11.676666 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:14:11.676635    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dab0dac_7220_4eb2_8e8d_6ec91e134a58.slice/crio-0fbf8e4717e4f1cfd53232c86a909e5efd04a0340a98c5616febe288642cf4ec WatchSource:0}: Error finding container 0fbf8e4717e4f1cfd53232c86a909e5efd04a0340a98c5616febe288642cf4ec: Status 404 returned error can't find the container with id 0fbf8e4717e4f1cfd53232c86a909e5efd04a0340a98c5616febe288642cf4ec
Jul 16 21:14:12.024541 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:12.024521    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t"
Jul 16 21:14:12.024675 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:12.024521    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t" event={"ID":"ee193d4e-4b45-4885-8765-8b13e2fe9a70","Type":"ContainerDied","Data":"055bd7904925f94f900707ef2805f8989c512c9f5bd75c358281f253ab166ee3"}
Jul 16 21:14:12.024675 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:12.024654    2576 scope.go:117] "RemoveContainer" containerID="8594fb4a43720716040616d0a4a6e601d0e9f9793787b0f418ce5f51c4869854"
Jul 16 21:14:12.026045 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:12.026020    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns" event={"ID":"9dab0dac-7220-4eb2-8e8d-6ec91e134a58","Type":"ContainerStarted","Data":"2b53ff15ddd3554ed34e7f53210d554ad0fa75a17d13c7f25896911e30268c56"}
Jul 16 21:14:12.026139 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:12.026065    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns" event={"ID":"9dab0dac-7220-4eb2-8e8d-6ec91e134a58","Type":"ContainerStarted","Data":"0fbf8e4717e4f1cfd53232c86a909e5efd04a0340a98c5616febe288642cf4ec"}
Jul 16 21:14:12.045391 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:12.045325    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns" podStartSLOduration=1.045311908 podStartE2EDuration="1.045311908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:14:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:14:12.043855922 +0000 UTC m=+2498.009790156" watchObservedRunningTime="2026-07-16 21:14:12.045311908 +0000 UTC m=+2498.011246142"
Jul 16 21:14:12.060334 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:12.060315    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t"]
Jul 16 21:14:12.066471 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:12.066452    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-90b68ec7-g8v7t"]
Jul 16 21:14:12.689052 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:12.689024    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee193d4e-4b45-4885-8765-8b13e2fe9a70" path="/var/lib/kubelet/pods/ee193d4e-4b45-4885-8765-8b13e2fe9a70/volumes"
Jul 16 21:14:13.034400 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:13.034305    2576 generic.go:356] "Generic (PLEG): container finished" podID="9dab0dac-7220-4eb2-8e8d-6ec91e134a58" containerID="2b53ff15ddd3554ed34e7f53210d554ad0fa75a17d13c7f25896911e30268c56" exitCode=6
Jul 16 21:14:13.034576 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:13.034399    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns" event={"ID":"9dab0dac-7220-4eb2-8e8d-6ec91e134a58","Type":"ContainerDied","Data":"2b53ff15ddd3554ed34e7f53210d554ad0fa75a17d13c7f25896911e30268c56"}
Jul 16 21:14:13.034644 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:13.034626    2576 scope.go:117] "RemoveContainer" containerID="2b53ff15ddd3554ed34e7f53210d554ad0fa75a17d13c7f25896911e30268c56"
Jul 16 21:14:14.042028 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:14.041993    2576 generic.go:356] "Generic (PLEG): container finished" podID="9dab0dac-7220-4eb2-8e8d-6ec91e134a58" containerID="5157db37272f1b7d87c691555ed08c793beb30efbe8c2d56e83e6598d26183d5" exitCode=6
Jul 16 21:14:14.042497 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:14.042074    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns" event={"ID":"9dab0dac-7220-4eb2-8e8d-6ec91e134a58","Type":"ContainerDied","Data":"5157db37272f1b7d87c691555ed08c793beb30efbe8c2d56e83e6598d26183d5"}
Jul 16 21:14:14.042497 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:14.042140    2576 scope.go:117] "RemoveContainer" containerID="2b53ff15ddd3554ed34e7f53210d554ad0fa75a17d13c7f25896911e30268c56"
Jul 16 21:14:14.042497 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:14.042354    2576 scope.go:117] "RemoveContainer" containerID="5157db37272f1b7d87c691555ed08c793beb30efbe8c2d56e83e6598d26183d5"
Jul 16 21:14:14.042672 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:14:14.042613    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-z7wns_odh-ai-gateway-infra(9dab0dac-7220-4eb2-8e8d-6ec91e134a58)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns" podUID="9dab0dac-7220-4eb2-8e8d-6ec91e134a58"
Jul 16 21:14:15.049660 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:15.049624    2576 scope.go:117] "RemoveContainer" containerID="5157db37272f1b7d87c691555ed08c793beb30efbe8c2d56e83e6598d26183d5"
Jul 16 21:14:15.050088 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:14:15.049832    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-z7wns_odh-ai-gateway-infra(9dab0dac-7220-4eb2-8e8d-6ec91e134a58)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns" podUID="9dab0dac-7220-4eb2-8e8d-6ec91e134a58"
Jul 16 21:14:30.684318 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:30.684209    2576 scope.go:117] "RemoveContainer" containerID="5157db37272f1b7d87c691555ed08c793beb30efbe8c2d56e83e6598d26183d5"
Jul 16 21:14:31.134979 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:31.134907    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns" event={"ID":"9dab0dac-7220-4eb2-8e8d-6ec91e134a58","Type":"ContainerStarted","Data":"038f7c0af546dc4e18eee7599bd659c621a2edaf2df5c80951624a57da11b851"}
Jul 16 21:14:32.142348 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:32.142317    2576 generic.go:356] "Generic (PLEG): container finished" podID="9dab0dac-7220-4eb2-8e8d-6ec91e134a58" containerID="038f7c0af546dc4e18eee7599bd659c621a2edaf2df5c80951624a57da11b851" exitCode=6
Jul 16 21:14:32.142822 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:32.142391    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns" event={"ID":"9dab0dac-7220-4eb2-8e8d-6ec91e134a58","Type":"ContainerDied","Data":"038f7c0af546dc4e18eee7599bd659c621a2edaf2df5c80951624a57da11b851"}
Jul 16 21:14:32.142822 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:32.142450    2576 scope.go:117] "RemoveContainer" containerID="5157db37272f1b7d87c691555ed08c793beb30efbe8c2d56e83e6598d26183d5"
Jul 16 21:14:32.142822 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:32.142722    2576 scope.go:117] "RemoveContainer" containerID="038f7c0af546dc4e18eee7599bd659c621a2edaf2df5c80951624a57da11b851"
Jul 16 21:14:32.142989 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:14:32.142933    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-z7wns_odh-ai-gateway-infra(9dab0dac-7220-4eb2-8e8d-6ec91e134a58)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns" podUID="9dab0dac-7220-4eb2-8e8d-6ec91e134a58"
Jul 16 21:14:32.168567 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:32.168544    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns"]
Jul 16 21:14:33.290847 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:33.290823    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns"
Jul 16 21:14:33.415015 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:33.414952    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dab0dac-7220-4eb2-8e8d-6ec91e134a58-maas-api-service-ca\") pod \"9dab0dac-7220-4eb2-8e8d-6ec91e134a58\" (UID: \"9dab0dac-7220-4eb2-8e8d-6ec91e134a58\") "
Jul 16 21:14:33.415266 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:33.415247    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dab0dac-7220-4eb2-8e8d-6ec91e134a58-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9dab0dac-7220-4eb2-8e8d-6ec91e134a58" (UID: "9dab0dac-7220-4eb2-8e8d-6ec91e134a58"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:14:33.516488 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:33.516467    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dab0dac-7220-4eb2-8e8d-6ec91e134a58-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:14:34.155691 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:34.155666    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns"
Jul 16 21:14:34.155850 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:34.155692    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns" event={"ID":"9dab0dac-7220-4eb2-8e8d-6ec91e134a58","Type":"ContainerDied","Data":"0fbf8e4717e4f1cfd53232c86a909e5efd04a0340a98c5616febe288642cf4ec"}
Jul 16 21:14:34.155850 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:34.155754    2576 scope.go:117] "RemoveContainer" containerID="038f7c0af546dc4e18eee7599bd659c621a2edaf2df5c80951624a57da11b851"
Jul 16 21:14:34.180782 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:34.180752    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns"]
Jul 16 21:14:34.189129 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:34.189105    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-z7wns"]
Jul 16 21:14:34.691112 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:34.691083    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dab0dac-7220-4eb2-8e8d-6ec91e134a58" path="/var/lib/kubelet/pods/9dab0dac-7220-4eb2-8e8d-6ec91e134a58/volumes"
Jul 16 21:14:55.284487 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:14:55.284455    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:15:00.143705 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:00.143617    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737275-dqqm7"]
Jul 16 21:15:00.144179 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:00.144157    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee193d4e-4b45-4885-8765-8b13e2fe9a70" containerName="revoke-keys"
Jul 16 21:15:00.144258 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:00.144182    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee193d4e-4b45-4885-8765-8b13e2fe9a70" containerName="revoke-keys"
Jul 16 21:15:00.144258 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:00.144198    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dab0dac-7220-4eb2-8e8d-6ec91e134a58" containerName="revoke-keys"
Jul 16 21:15:00.144258 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:00.144206    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dab0dac-7220-4eb2-8e8d-6ec91e134a58" containerName="revoke-keys"
Jul 16 21:15:00.144258 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:00.144220    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dab0dac-7220-4eb2-8e8d-6ec91e134a58" containerName="revoke-keys"
Jul 16 21:15:00.144258 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:00.144229    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dab0dac-7220-4eb2-8e8d-6ec91e134a58" containerName="revoke-keys"
Jul 16 21:15:00.144258 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:00.144249    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dab0dac-7220-4eb2-8e8d-6ec91e134a58" containerName="revoke-keys"
Jul 16 21:15:00.144258 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:00.144257    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dab0dac-7220-4eb2-8e8d-6ec91e134a58" containerName="revoke-keys"
Jul 16 21:15:00.144645 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:00.144352    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee193d4e-4b45-4885-8765-8b13e2fe9a70" containerName="revoke-keys"
Jul 16 21:15:00.144645 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:00.144383    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dab0dac-7220-4eb2-8e8d-6ec91e134a58" containerName="revoke-keys"
Jul 16 21:15:00.144645 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:00.144396    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dab0dac-7220-4eb2-8e8d-6ec91e134a58" containerName="revoke-keys"
Jul 16 21:15:00.147205 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:00.147182    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737275-dqqm7"
Jul 16 21:15:00.150490 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:00.150471    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:15:00.157447 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:00.157421    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737275-dqqm7"
Jul 16 21:15:00.158040 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:00.158019    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737275-dqqm7"]
Jul 16 21:15:00.289892 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:00.289870    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29737275-dqqm7"]
Jul 16 21:15:00.291471 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:15:00.291447    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode400487d_b619_4469_b432_47a563cff330.slice/crio-e93246bd35deabab32409be95dbea3805c4b14c262f4ff42fcc14ceaaa91cce9 WatchSource:0}: Error finding container e93246bd35deabab32409be95dbea3805c4b14c262f4ff42fcc14ceaaa91cce9: Status 404 returned error can't find the container with id e93246bd35deabab32409be95dbea3805c4b14c262f4ff42fcc14ceaaa91cce9
Jul 16 21:15:01.293592 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:01.293557    2576 generic.go:356] "Generic (PLEG): container finished" podID="e400487d-b619-4469-b432-47a563cff330" containerID="8acc389f667656a8dcc4b65d70610c2400ada9dcbdc912de1d89c712415587ed" exitCode=0
Jul 16 21:15:01.294100 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:01.293594    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737275-dqqm7" event={"ID":"e400487d-b619-4469-b432-47a563cff330","Type":"ContainerDied","Data":"8acc389f667656a8dcc4b65d70610c2400ada9dcbdc912de1d89c712415587ed"}
Jul 16 21:15:01.294100 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:01.293645    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737275-dqqm7" event={"ID":"e400487d-b619-4469-b432-47a563cff330","Type":"ContainerStarted","Data":"e93246bd35deabab32409be95dbea3805c4b14c262f4ff42fcc14ceaaa91cce9"}
Jul 16 21:15:02.427770 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:02.427750    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737275-dqqm7"
Jul 16 21:15:03.303921 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:03.303889    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737275-dqqm7"
Jul 16 21:15:03.304070 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:03.303916    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29737275-dqqm7" event={"ID":"e400487d-b619-4469-b432-47a563cff330","Type":"ContainerDied","Data":"e93246bd35deabab32409be95dbea3805c4b14c262f4ff42fcc14ceaaa91cce9"}
Jul 16 21:15:03.304070 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:03.303967    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e93246bd35deabab32409be95dbea3805c4b14c262f4ff42fcc14ceaaa91cce9"
Jul 16 21:15:04.390761 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:04.390728    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:15:08.011062 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.011032    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"]
Jul 16 21:15:08.011465 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.011404    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e400487d-b619-4469-b432-47a563cff330" containerName="cleanup"
Jul 16 21:15:08.011465 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.011416    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e400487d-b619-4469-b432-47a563cff330" containerName="cleanup"
Jul 16 21:15:08.011554 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.011482    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e400487d-b619-4469-b432-47a563cff330" containerName="cleanup"
Jul 16 21:15:08.011554 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.011491    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dab0dac-7220-4eb2-8e8d-6ec91e134a58" containerName="revoke-keys"
Jul 16 21:15:08.014635 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.014618    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.017215 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.017199    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2a7694fd-openshift-default-dockercfg-24tsn\""
Jul 16 21:15:08.030277 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.030258    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"]
Jul 16 21:15:08.091747 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.091721    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-workload-certs\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.091856 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.091790    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-data\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.091856 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.091837    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istiod-ca-cert\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.091964 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.091868    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-credential-socket\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.092008 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.091973    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-token\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.092067 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.092028    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx69m\" (UniqueName: \"kubernetes.io/projected/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-kube-api-access-sx69m\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.092067 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.092048    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-workload-socket\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.092175 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.092077    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-envoy\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.092175 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.092105    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-podinfo\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.193421 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.193384    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-credential-socket\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.193571 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.193437    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-token\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.193571 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.193471    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx69m\" (UniqueName: \"kubernetes.io/projected/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-kube-api-access-sx69m\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.193571 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.193487    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-workload-socket\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.193571 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.193502    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-envoy\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.193571 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.193518    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-podinfo\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.193826 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.193683    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-workload-certs\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.193826 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.193770    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-data\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.193826 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.193785    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-credential-socket\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.193826 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.193803    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istiod-ca-cert\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.194012 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.193887    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-workload-socket\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.194065 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.194026    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-data\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.194121 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.194082    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-workload-certs\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.194432 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.194416    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istiod-ca-cert\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.195663 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.195643    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-envoy\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.195981 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.195963    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-podinfo\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.201715 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.201697    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-token\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.201804 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.201778    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx69m\" (UniqueName: \"kubernetes.io/projected/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-kube-api-access-sx69m\") pod \"e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") " pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.324992 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.324933    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:08.461882 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.461857    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"]
Jul 16 21:15:08.463855 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:15:08.463828    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a9af3c8_f582_4f9f_b27a_3d6a881106ec.slice/crio-70a040c17fb0f50f2fd31e284e21eaf3359711258db1b6e0350918581293851e WatchSource:0}: Error finding container 70a040c17fb0f50f2fd31e284e21eaf3359711258db1b6e0350918581293851e: Status 404 returned error can't find the container with id 70a040c17fb0f50f2fd31e284e21eaf3359711258db1b6e0350918581293851e
Jul 16 21:15:08.465921 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.465888    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:15:08.466001 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.465966    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:15:08.466051 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:08.466038    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:15:09.335549 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:09.335515    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t" event={"ID":"2a9af3c8-f582-4f9f-b27a-3d6a881106ec","Type":"ContainerStarted","Data":"eecb351ef42f4e86e247845329cc1dc7366dd39ba67d96a42e1c8cfa601c4268"}
Jul 16 21:15:09.336012 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:09.335566    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t" event={"ID":"2a9af3c8-f582-4f9f-b27a-3d6a881106ec","Type":"ContainerStarted","Data":"70a040c17fb0f50f2fd31e284e21eaf3359711258db1b6e0350918581293851e"}
Jul 16 21:15:09.358535 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:09.358487    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t" podStartSLOduration=2.35847252 podStartE2EDuration="2.35847252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:15:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:15:09.35632572 +0000 UTC m=+2555.322259975" watchObservedRunningTime="2026-07-16 21:15:09.35847252 +0000 UTC m=+2555.324406752"
Jul 16 21:15:10.326098 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:10.326063    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:10.330620 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:10.330591    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:10.340221 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:10.340196    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:10.340875 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:10.340860    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:18.407562 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:18.407530    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:15:26.036508 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:26.034679    2576 patch_prober.go:28] interesting pod/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": read tcp 10.132.0.2:48600->10.132.0.59:15021: read: connection reset by peer" start-of-body=
Jul 16 21:15:26.036508 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:26.034747    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t" podUID="2a9af3c8-f582-4f9f-b27a-3d6a881106ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": read tcp 10.132.0.2:48600->10.132.0.59:15021: read: connection reset by peer"
Jul 16 21:15:26.426099 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:26.426070    2576 generic.go:356] "Generic (PLEG): container finished" podID="2a9af3c8-f582-4f9f-b27a-3d6a881106ec" containerID="eecb351ef42f4e86e247845329cc1dc7366dd39ba67d96a42e1c8cfa601c4268" exitCode=0
Jul 16 21:15:26.426284 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:26.426139    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t" event={"ID":"2a9af3c8-f582-4f9f-b27a-3d6a881106ec","Type":"ContainerDied","Data":"eecb351ef42f4e86e247845329cc1dc7366dd39ba67d96a42e1c8cfa601c4268"}
Jul 16 21:15:26.426543 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:26.426527    2576 scope.go:117] "RemoveContainer" containerID="eecb351ef42f4e86e247845329cc1dc7366dd39ba67d96a42e1c8cfa601c4268"
Jul 16 21:15:26.427506 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:26.427478    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:15:26.427605 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:26.427583    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:15:26.427667 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:26.427632    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:15:27.433414 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:27.433355    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t" event={"ID":"2a9af3c8-f582-4f9f-b27a-3d6a881106ec","Type":"ContainerStarted","Data":"aa6b1e25eef891d462b143df366e85980a6c2d13fb3938570bbec0ee6bc4e3f1"}
Jul 16 21:15:28.325113 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:28.325072    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:28.325322 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:28.325256    2576 patch_prober.go:28] interesting pod/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 16 21:15:28.325322 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:28.325304    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t" podUID="2a9af3c8-f582-4f9f-b27a-3d6a881106ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 16 21:15:29.325883 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:29.325847    2576 patch_prober.go:28] interesting pod/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 16 21:15:29.326275 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:29.325899    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t" podUID="2a9af3c8-f582-4f9f-b27a-3d6a881106ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 16 21:15:30.325650 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:30.325620    2576 patch_prober.go:28] interesting pod/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 16 21:15:30.325826 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:30.325668    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t" podUID="2a9af3c8-f582-4f9f-b27a-3d6a881106ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 16 21:15:31.326044 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:31.326008    2576 patch_prober.go:28] interesting pod/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 16 21:15:31.326506 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:31.326073    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t" podUID="2a9af3c8-f582-4f9f-b27a-3d6a881106ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 16 21:15:32.329732 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:32.329697    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:32.330152 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:32.329969    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:32.330808 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:32.330784    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:36.791282 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:36.791248    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:15:49.011322 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:49.011287    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"]
Jul 16 21:15:49.011750 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:49.011562    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t" podUID="2a9af3c8-f582-4f9f-b27a-3d6a881106ec" containerName="istio-proxy" containerID="cri-o://aa6b1e25eef891d462b143df366e85980a6c2d13fb3938570bbec0ee6bc4e3f1" gracePeriod=30
Jul 16 21:15:53.970038 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:53.970005    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6"]
Jul 16 21:15:53.973586 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:53.973569    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6"
Jul 16 21:15:53.976046 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:53.976018    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:15:53.976170 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:53.976084    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:15:53.980851 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:53.980826    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6"]
Jul 16 21:15:54.077153 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.077121    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d4e7ca2-6620-4fd6-948a-35e0987421de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6\" (UID: \"9d4e7ca2-6620-4fd6-948a-35e0987421de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6"
Jul 16 21:15:54.177934 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.177907    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d4e7ca2-6620-4fd6-948a-35e0987421de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6\" (UID: \"9d4e7ca2-6620-4fd6-948a-35e0987421de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6"
Jul 16 21:15:54.178541 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.178523    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d4e7ca2-6620-4fd6-948a-35e0987421de-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6\" (UID: \"9d4e7ca2-6620-4fd6-948a-35e0987421de\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6"
Jul 16 21:15:54.260584 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.260559    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:54.285157 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.285135    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6"
Jul 16 21:15:54.380006 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.379975    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-podinfo\") pod \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") "
Jul 16 21:15:54.380127 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.380063    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-data\") pod \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") "
Jul 16 21:15:54.380127 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.380094    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istiod-ca-cert\") pod \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") "
Jul 16 21:15:54.380217 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.380144    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-credential-socket\") pod \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") "
Jul 16 21:15:54.380217 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.380173    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx69m\" (UniqueName: \"kubernetes.io/projected/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-kube-api-access-sx69m\") pod \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") "
Jul 16 21:15:54.380217 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.380198    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-workload-socket\") pod \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") "
Jul 16 21:15:54.380403 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.380256    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-token\") pod \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") "
Jul 16 21:15:54.380403 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.380299    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-workload-certs\") pod \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") "
Jul 16 21:15:54.380403 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.380344    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-envoy\") pod \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\" (UID: \"2a9af3c8-f582-4f9f-b27a-3d6a881106ec\") "
Jul 16 21:15:54.380787 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.380762    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2a9af3c8-f582-4f9f-b27a-3d6a881106ec" (UID: "2a9af3c8-f582-4f9f-b27a-3d6a881106ec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:15:54.380935 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.380776    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2a9af3c8-f582-4f9f-b27a-3d6a881106ec" (UID: "2a9af3c8-f582-4f9f-b27a-3d6a881106ec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:15:54.381011 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.380838    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2a9af3c8-f582-4f9f-b27a-3d6a881106ec" (UID: "2a9af3c8-f582-4f9f-b27a-3d6a881106ec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:15:54.381099 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.380901    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-data" (OuterVolumeSpecName: "istio-data") pod "2a9af3c8-f582-4f9f-b27a-3d6a881106ec" (UID: "2a9af3c8-f582-4f9f-b27a-3d6a881106ec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:15:54.381194 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.381058    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2a9af3c8-f582-4f9f-b27a-3d6a881106ec" (UID: "2a9af3c8-f582-4f9f-b27a-3d6a881106ec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:15:54.382818 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.382792    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2a9af3c8-f582-4f9f-b27a-3d6a881106ec" (UID: "2a9af3c8-f582-4f9f-b27a-3d6a881106ec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 21:15:54.384259 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.384230    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-kube-api-access-sx69m" (OuterVolumeSpecName: "kube-api-access-sx69m") pod "2a9af3c8-f582-4f9f-b27a-3d6a881106ec" (UID: "2a9af3c8-f582-4f9f-b27a-3d6a881106ec"). InnerVolumeSpecName "kube-api-access-sx69m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:15:54.385766 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.385737    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-token" (OuterVolumeSpecName: "istio-token") pod "2a9af3c8-f582-4f9f-b27a-3d6a881106ec" (UID: "2a9af3c8-f582-4f9f-b27a-3d6a881106ec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:15:54.385840 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.385765    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2a9af3c8-f582-4f9f-b27a-3d6a881106ec" (UID: "2a9af3c8-f582-4f9f-b27a-3d6a881106ec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:15:54.417182 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.416287    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6"]
Jul 16 21:15:54.419795 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:15:54.419771    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d4e7ca2_6620_4fd6_948a_35e0987421de.slice/crio-07f9c620a60cd6e0f8107137fd241586df7a8cb4834a370872e477cf47cc7157 WatchSource:0}: Error finding container 07f9c620a60cd6e0f8107137fd241586df7a8cb4834a370872e477cf47cc7157: Status 404 returned error can't find the container with id 07f9c620a60cd6e0f8107137fd241586df7a8cb4834a370872e477cf47cc7157
Jul 16 21:15:54.481024 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.481002    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-credential-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:15:54.481103 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.481032    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx69m\" (UniqueName: \"kubernetes.io/projected/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-kube-api-access-sx69m\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:15:54.481103 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.481042    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-workload-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:15:54.481103 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.481053    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-token\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:15:54.481103 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.481061    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-workload-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:15:54.481103 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.481070    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-envoy\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:15:54.481103 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.481078    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-podinfo\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:15:54.481103 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.481085    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istio-data\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:15:54.481103 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.481093    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a9af3c8-f582-4f9f-b27a-3d6a881106ec-istiod-ca-cert\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:15:54.578212 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.578154    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6" event={"ID":"9d4e7ca2-6620-4fd6-948a-35e0987421de","Type":"ContainerStarted","Data":"07f9c620a60cd6e0f8107137fd241586df7a8cb4834a370872e477cf47cc7157"}
Jul 16 21:15:54.579950 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.579923    2576 generic.go:356] "Generic (PLEG): container finished" podID="2a9af3c8-f582-4f9f-b27a-3d6a881106ec" containerID="aa6b1e25eef891d462b143df366e85980a6c2d13fb3938570bbec0ee6bc4e3f1" exitCode=0
Jul 16 21:15:54.580034 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.579969    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"
Jul 16 21:15:54.580077 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.579970    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t" event={"ID":"2a9af3c8-f582-4f9f-b27a-3d6a881106ec","Type":"ContainerDied","Data":"aa6b1e25eef891d462b143df366e85980a6c2d13fb3938570bbec0ee6bc4e3f1"}
Jul 16 21:15:54.580117 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.580104    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t" event={"ID":"2a9af3c8-f582-4f9f-b27a-3d6a881106ec","Type":"ContainerDied","Data":"70a040c17fb0f50f2fd31e284e21eaf3359711258db1b6e0350918581293851e"}
Jul 16 21:15:54.580166 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.580153    2576 scope.go:117] "RemoveContainer" containerID="aa6b1e25eef891d462b143df366e85980a6c2d13fb3938570bbec0ee6bc4e3f1"
Jul 16 21:15:54.588586 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.588566    2576 scope.go:117] "RemoveContainer" containerID="eecb351ef42f4e86e247845329cc1dc7366dd39ba67d96a42e1c8cfa601c4268"
Jul 16 21:15:54.596329 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.596310    2576 scope.go:117] "RemoveContainer" containerID="aa6b1e25eef891d462b143df366e85980a6c2d13fb3938570bbec0ee6bc4e3f1"
Jul 16 21:15:54.596602 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:15:54.596582    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa6b1e25eef891d462b143df366e85980a6c2d13fb3938570bbec0ee6bc4e3f1\": container with ID starting with aa6b1e25eef891d462b143df366e85980a6c2d13fb3938570bbec0ee6bc4e3f1 not found: ID does not exist" containerID="aa6b1e25eef891d462b143df366e85980a6c2d13fb3938570bbec0ee6bc4e3f1"
Jul 16 21:15:54.596710 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.596613    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa6b1e25eef891d462b143df366e85980a6c2d13fb3938570bbec0ee6bc4e3f1"} err="failed to get container status \"aa6b1e25eef891d462b143df366e85980a6c2d13fb3938570bbec0ee6bc4e3f1\": rpc error: code = NotFound desc = could not find container \"aa6b1e25eef891d462b143df366e85980a6c2d13fb3938570bbec0ee6bc4e3f1\": container with ID starting with aa6b1e25eef891d462b143df366e85980a6c2d13fb3938570bbec0ee6bc4e3f1 not found: ID does not exist"
Jul 16 21:15:54.596710 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.596636    2576 scope.go:117] "RemoveContainer" containerID="eecb351ef42f4e86e247845329cc1dc7366dd39ba67d96a42e1c8cfa601c4268"
Jul 16 21:15:54.596908 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:15:54.596889    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eecb351ef42f4e86e247845329cc1dc7366dd39ba67d96a42e1c8cfa601c4268\": container with ID starting with eecb351ef42f4e86e247845329cc1dc7366dd39ba67d96a42e1c8cfa601c4268 not found: ID does not exist" containerID="eecb351ef42f4e86e247845329cc1dc7366dd39ba67d96a42e1c8cfa601c4268"
Jul 16 21:15:54.596954 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.596912    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eecb351ef42f4e86e247845329cc1dc7366dd39ba67d96a42e1c8cfa601c4268"} err="failed to get container status \"eecb351ef42f4e86e247845329cc1dc7366dd39ba67d96a42e1c8cfa601c4268\": rpc error: code = NotFound desc = could not find container \"eecb351ef42f4e86e247845329cc1dc7366dd39ba67d96a42e1c8cfa601c4268\": container with ID starting with eecb351ef42f4e86e247845329cc1dc7366dd39ba67d96a42e1c8cfa601c4268 not found: ID does not exist"
Jul 16 21:15:54.602011 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.601989    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"]
Jul 16 21:15:54.608538 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.608514    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2a7694fd-openshift-default-5cbd66bf5-b4b8t"]
Jul 16 21:15:54.689692 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:54.689669    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a9af3c8-f582-4f9f-b27a-3d6a881106ec" path="/var/lib/kubelet/pods/2a9af3c8-f582-4f9f-b27a-3d6a881106ec/volumes"
Jul 16 21:15:55.587433 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:55.587396    2576 generic.go:356] "Generic (PLEG): container finished" podID="9d4e7ca2-6620-4fd6-948a-35e0987421de" containerID="0b076e17ab20e22cbcd7ab402165380368e20976dc52d04a18bc6d467ea1a3ab" exitCode=6
Jul 16 21:15:55.587779 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:55.587465    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6" event={"ID":"9d4e7ca2-6620-4fd6-948a-35e0987421de","Type":"ContainerDied","Data":"0b076e17ab20e22cbcd7ab402165380368e20976dc52d04a18bc6d467ea1a3ab"}
Jul 16 21:15:55.587779 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:55.587607    2576 scope.go:117] "RemoveContainer" containerID="0b076e17ab20e22cbcd7ab402165380368e20976dc52d04a18bc6d467ea1a3ab"
Jul 16 21:15:56.594002 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:56.593928    2576 generic.go:356] "Generic (PLEG): container finished" podID="9d4e7ca2-6620-4fd6-948a-35e0987421de" containerID="87a3b98c5ad70e690c3e694753363816d3d1728d7abd65d1386e4663caa6f193" exitCode=6
Jul 16 21:15:56.594002 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:56.593967    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6" event={"ID":"9d4e7ca2-6620-4fd6-948a-35e0987421de","Type":"ContainerDied","Data":"87a3b98c5ad70e690c3e694753363816d3d1728d7abd65d1386e4663caa6f193"}
Jul 16 21:15:56.594478 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:56.594014    2576 scope.go:117] "RemoveContainer" containerID="0b076e17ab20e22cbcd7ab402165380368e20976dc52d04a18bc6d467ea1a3ab"
Jul 16 21:15:56.594478 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:56.594258    2576 scope.go:117] "RemoveContainer" containerID="87a3b98c5ad70e690c3e694753363816d3d1728d7abd65d1386e4663caa6f193"
Jul 16 21:15:56.594584 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:15:56.594531    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6_odh-ai-gateway-infra(9d4e7ca2-6620-4fd6-948a-35e0987421de)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6" podUID="9d4e7ca2-6620-4fd6-948a-35e0987421de"
Jul 16 21:15:57.600323 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:15:57.600254    2576 scope.go:117] "RemoveContainer" containerID="87a3b98c5ad70e690c3e694753363816d3d1728d7abd65d1386e4663caa6f193"
Jul 16 21:15:57.600782 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:15:57.600439    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6_odh-ai-gateway-infra(9d4e7ca2-6620-4fd6-948a-35e0987421de)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6" podUID="9d4e7ca2-6620-4fd6-948a-35e0987421de"
Jul 16 21:16:01.791563 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:01.791523    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t"]
Jul 16 21:16:01.791979 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:01.791915    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a9af3c8-f582-4f9f-b27a-3d6a881106ec" containerName="istio-proxy"
Jul 16 21:16:01.791979 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:01.791929    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a9af3c8-f582-4f9f-b27a-3d6a881106ec" containerName="istio-proxy"
Jul 16 21:16:01.791979 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:01.791958    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a9af3c8-f582-4f9f-b27a-3d6a881106ec" containerName="istio-proxy"
Jul 16 21:16:01.791979 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:01.791964    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a9af3c8-f582-4f9f-b27a-3d6a881106ec" containerName="istio-proxy"
Jul 16 21:16:01.792124 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:01.792039    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a9af3c8-f582-4f9f-b27a-3d6a881106ec" containerName="istio-proxy"
Jul 16 21:16:01.792124 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:01.792047    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a9af3c8-f582-4f9f-b27a-3d6a881106ec" containerName="istio-proxy"
Jul 16 21:16:01.796626 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:01.796605    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t"
Jul 16 21:16:01.805390 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:01.805346    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t"]
Jul 16 21:16:01.938030 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:01.937997    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89dc6442-73ce-4a37-bcf0-fc7aafb26602-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t\" (UID: \"89dc6442-73ce-4a37-bcf0-fc7aafb26602\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t"
Jul 16 21:16:02.038765 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:02.038736    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89dc6442-73ce-4a37-bcf0-fc7aafb26602-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t\" (UID: \"89dc6442-73ce-4a37-bcf0-fc7aafb26602\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t"
Jul 16 21:16:02.039405 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:02.039385    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89dc6442-73ce-4a37-bcf0-fc7aafb26602-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t\" (UID: \"89dc6442-73ce-4a37-bcf0-fc7aafb26602\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t"
Jul 16 21:16:02.109184 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:02.109123    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t"
Jul 16 21:16:02.239467 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:02.239441    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t"]
Jul 16 21:16:02.241278 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:16:02.241254    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89dc6442_73ce_4a37_bcf0_fc7aafb26602.slice/crio-f72db503ac7a26533cbb15c59b325d2c9e446e4ff948c826cd63dbfcef2c110d WatchSource:0}: Error finding container f72db503ac7a26533cbb15c59b325d2c9e446e4ff948c826cd63dbfcef2c110d: Status 404 returned error can't find the container with id f72db503ac7a26533cbb15c59b325d2c9e446e4ff948c826cd63dbfcef2c110d
Jul 16 21:16:02.627540 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:02.627505    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t" event={"ID":"89dc6442-73ce-4a37-bcf0-fc7aafb26602","Type":"ContainerStarted","Data":"89b03cbfa1b69ed3a6c4481e41559615aa4493437facda899c56c07864180050"}
Jul 16 21:16:02.627654 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:02.627560    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t" event={"ID":"89dc6442-73ce-4a37-bcf0-fc7aafb26602","Type":"ContainerStarted","Data":"f72db503ac7a26533cbb15c59b325d2c9e446e4ff948c826cd63dbfcef2c110d"}
Jul 16 21:16:02.646437 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:02.646387    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t" podStartSLOduration=1.6463555730000001 podStartE2EDuration="1.646355573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:02.644519889 +0000 UTC m=+2608.610454121" watchObservedRunningTime="2026-07-16 21:16:02.646355573 +0000 UTC m=+2608.612289810"
Jul 16 21:16:03.634791 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:03.634748    2576 generic.go:356] "Generic (PLEG): container finished" podID="89dc6442-73ce-4a37-bcf0-fc7aafb26602" containerID="89b03cbfa1b69ed3a6c4481e41559615aa4493437facda899c56c07864180050" exitCode=6
Jul 16 21:16:03.635262 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:03.634830    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t" event={"ID":"89dc6442-73ce-4a37-bcf0-fc7aafb26602","Type":"ContainerDied","Data":"89b03cbfa1b69ed3a6c4481e41559615aa4493437facda899c56c07864180050"}
Jul 16 21:16:03.635262 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:03.634986    2576 scope.go:117] "RemoveContainer" containerID="89b03cbfa1b69ed3a6c4481e41559615aa4493437facda899c56c07864180050"
Jul 16 21:16:04.642256 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:04.642230    2576 generic.go:356] "Generic (PLEG): container finished" podID="89dc6442-73ce-4a37-bcf0-fc7aafb26602" containerID="14a2d23329e32bc41198879ae36198d93411a5d43b79c19e85a8e583114e7163" exitCode=6
Jul 16 21:16:04.642689 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:04.642325    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t" event={"ID":"89dc6442-73ce-4a37-bcf0-fc7aafb26602","Type":"ContainerDied","Data":"14a2d23329e32bc41198879ae36198d93411a5d43b79c19e85a8e583114e7163"}
Jul 16 21:16:04.642689 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:04.642420    2576 scope.go:117] "RemoveContainer" containerID="89b03cbfa1b69ed3a6c4481e41559615aa4493437facda899c56c07864180050"
Jul 16 21:16:04.642689 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:04.642484    2576 scope.go:117] "RemoveContainer" containerID="14a2d23329e32bc41198879ae36198d93411a5d43b79c19e85a8e583114e7163"
Jul 16 21:16:04.642689 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:16:04.642666    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t_odh-ai-gateway-infra(89dc6442-73ce-4a37-bcf0-fc7aafb26602)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t" podUID="89dc6442-73ce-4a37-bcf0-fc7aafb26602"
Jul 16 21:16:05.648867 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:05.648845    2576 scope.go:117] "RemoveContainer" containerID="14a2d23329e32bc41198879ae36198d93411a5d43b79c19e85a8e583114e7163"
Jul 16 21:16:05.649205 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:16:05.648988    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t_odh-ai-gateway-infra(89dc6442-73ce-4a37-bcf0-fc7aafb26602)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t" podUID="89dc6442-73ce-4a37-bcf0-fc7aafb26602"
Jul 16 21:16:09.518524 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.518490    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"]
Jul 16 21:16:09.523162 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.523141    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.526115 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.526095    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e9d9b90e-openshift-default-dockercfg-76kz8\""
Jul 16 21:16:09.535372 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.535338    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"]
Jul 16 21:16:09.695941 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.695907    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-credential-socket\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.696116 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.695954    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-workload-socket\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.696116 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.695983    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-token\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.696116 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.696023    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-podinfo\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.696116 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.696047    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istiod-ca-cert\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.696116 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.696072    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw7tk\" (UniqueName: \"kubernetes.io/projected/7fffd3d7-e78f-4da4-ba31-0c813dff9024-kube-api-access-qw7tk\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.696322 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.696163    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-envoy\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.696322 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.696214    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-workload-certs\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.696322 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.696245    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-data\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.797306 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.797239    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-envoy\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.797306 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.797273    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-workload-certs\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.797306 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.797296    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-data\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.797571 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.797318    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-credential-socket\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.797571 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.797343    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-workload-socket\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.797571 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.797391    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-token\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.797571 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.797442    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-podinfo\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.797785 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.797704    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-workload-certs\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.797785 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.797747    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-credential-socket\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.797891 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.797798    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-data\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.797891 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.797815    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istiod-ca-cert\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.797891 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.797837    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-workload-socket\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.797891 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.797861    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw7tk\" (UniqueName: \"kubernetes.io/projected/7fffd3d7-e78f-4da4-ba31-0c813dff9024-kube-api-access-qw7tk\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.798497 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.798473    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istiod-ca-cert\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.799664 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.799639    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-envoy\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.799938 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.799912    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-podinfo\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.811325 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.811301    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw7tk\" (UniqueName: \"kubernetes.io/projected/7fffd3d7-e78f-4da4-ba31-0c813dff9024-kube-api-access-qw7tk\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.811682 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.811662    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-token\") pod \"e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") " pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.833241 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.833220    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:09.992581 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.992549    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"]
Jul 16 21:16:09.993535 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:16:09.993503    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fffd3d7_e78f_4da4_ba31_0c813dff9024.slice/crio-1ee6d566730d7f854bd61518559e6c6435f27e35a10ea34748f9602eea3abe2f WatchSource:0}: Error finding container 1ee6d566730d7f854bd61518559e6c6435f27e35a10ea34748f9602eea3abe2f: Status 404 returned error can't find the container with id 1ee6d566730d7f854bd61518559e6c6435f27e35a10ea34748f9602eea3abe2f
Jul 16 21:16:09.995608 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.995569    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:16:09.995701 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.995649    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:16:09.995701 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:09.995677    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:16:10.674273 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:10.674243    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs" event={"ID":"7fffd3d7-e78f-4da4-ba31-0c813dff9024","Type":"ContainerStarted","Data":"37455bc2cd0579b60cda4bd2fd67022dbd9c3e766751020b7fee848e8f065e4c"}
Jul 16 21:16:10.674694 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:10.674290    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs" event={"ID":"7fffd3d7-e78f-4da4-ba31-0c813dff9024","Type":"ContainerStarted","Data":"1ee6d566730d7f854bd61518559e6c6435f27e35a10ea34748f9602eea3abe2f"}
Jul 16 21:16:10.697239 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:10.697197    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs" podStartSLOduration=1.697182771 podStartE2EDuration="1.697182771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:10.694733461 +0000 UTC m=+2616.660667694" watchObservedRunningTime="2026-07-16 21:16:10.697182771 +0000 UTC m=+2616.663117004"
Jul 16 21:16:10.833876 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:10.833850    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:11.839718 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:11.839692    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:12.684097 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:12.684065    2576 scope.go:117] "RemoveContainer" containerID="87a3b98c5ad70e690c3e694753363816d3d1728d7abd65d1386e4663caa6f193"
Jul 16 21:16:12.690727 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:12.690708    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:12.690807 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:12.690765    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:13.689879 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:13.689853    2576 generic.go:356] "Generic (PLEG): container finished" podID="9d4e7ca2-6620-4fd6-948a-35e0987421de" containerID="110d86d449a343e0d0a5e7c25bd859b364ac7324b4025a3c9940112b6ca921c1" exitCode=6
Jul 16 21:16:13.690301 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:13.689925    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6" event={"ID":"9d4e7ca2-6620-4fd6-948a-35e0987421de","Type":"ContainerDied","Data":"110d86d449a343e0d0a5e7c25bd859b364ac7324b4025a3c9940112b6ca921c1"}
Jul 16 21:16:13.690301 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:13.689985    2576 scope.go:117] "RemoveContainer" containerID="87a3b98c5ad70e690c3e694753363816d3d1728d7abd65d1386e4663caa6f193"
Jul 16 21:16:13.690431 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:13.690411    2576 scope.go:117] "RemoveContainer" containerID="110d86d449a343e0d0a5e7c25bd859b364ac7324b4025a3c9940112b6ca921c1"
Jul 16 21:16:13.690651 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:16:13.690623    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6_odh-ai-gateway-infra(9d4e7ca2-6620-4fd6-948a-35e0987421de)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6" podUID="9d4e7ca2-6620-4fd6-948a-35e0987421de"
Jul 16 21:16:14.719591 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:14.719556    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6"]
Jul 16 21:16:14.861872 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:14.861841    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6"
Jul 16 21:16:14.947204 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:14.947176    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d4e7ca2-6620-4fd6-948a-35e0987421de-maas-api-service-ca\") pod \"9d4e7ca2-6620-4fd6-948a-35e0987421de\" (UID: \"9d4e7ca2-6620-4fd6-948a-35e0987421de\") "
Jul 16 21:16:14.947575 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:14.947554    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d4e7ca2-6620-4fd6-948a-35e0987421de-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9d4e7ca2-6620-4fd6-948a-35e0987421de" (UID: "9d4e7ca2-6620-4fd6-948a-35e0987421de"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:16:15.048255 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:15.048199    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d4e7ca2-6620-4fd6-948a-35e0987421de-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:16:15.701331 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:15.701304    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6"
Jul 16 21:16:15.701518 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:15.701347    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6" event={"ID":"9d4e7ca2-6620-4fd6-948a-35e0987421de","Type":"ContainerDied","Data":"07f9c620a60cd6e0f8107137fd241586df7a8cb4834a370872e477cf47cc7157"}
Jul 16 21:16:15.701518 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:15.701413    2576 scope.go:117] "RemoveContainer" containerID="110d86d449a343e0d0a5e7c25bd859b364ac7324b4025a3c9940112b6ca921c1"
Jul 16 21:16:15.724631 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:15.724609    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6"]
Jul 16 21:16:15.732197 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:15.732176    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-4c4q6"]
Jul 16 21:16:16.690060 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:16.690025    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d4e7ca2-6620-4fd6-948a-35e0987421de" path="/var/lib/kubelet/pods/9d4e7ca2-6620-4fd6-948a-35e0987421de/volumes"
Jul 16 21:16:17.684331 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:17.684302    2576 scope.go:117] "RemoveContainer" containerID="14a2d23329e32bc41198879ae36198d93411a5d43b79c19e85a8e583114e7163"
Jul 16 21:16:18.716839 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:18.716811    2576 generic.go:356] "Generic (PLEG): container finished" podID="89dc6442-73ce-4a37-bcf0-fc7aafb26602" containerID="730073a7fd8a3644a6a5e85c061264baa04e70bb5b3f0bf48cc87b332b00ceac" exitCode=6
Jul 16 21:16:18.717201 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:18.716853    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t" event={"ID":"89dc6442-73ce-4a37-bcf0-fc7aafb26602","Type":"ContainerDied","Data":"730073a7fd8a3644a6a5e85c061264baa04e70bb5b3f0bf48cc87b332b00ceac"}
Jul 16 21:16:18.717201 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:18.716896    2576 scope.go:117] "RemoveContainer" containerID="14a2d23329e32bc41198879ae36198d93411a5d43b79c19e85a8e583114e7163"
Jul 16 21:16:18.717201 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:18.717158    2576 scope.go:117] "RemoveContainer" containerID="730073a7fd8a3644a6a5e85c061264baa04e70bb5b3f0bf48cc87b332b00ceac"
Jul 16 21:16:18.717339 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:16:18.717320    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t_odh-ai-gateway-infra(89dc6442-73ce-4a37-bcf0-fc7aafb26602)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t" podUID="89dc6442-73ce-4a37-bcf0-fc7aafb26602"
Jul 16 21:16:19.292582 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:19.292553    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:16:19.743117 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:19.743084    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t"]
Jul 16 21:16:19.879214 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:19.879188    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t"
Jul 16 21:16:19.988586 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:19.988563    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89dc6442-73ce-4a37-bcf0-fc7aafb26602-maas-api-service-ca\") pod \"89dc6442-73ce-4a37-bcf0-fc7aafb26602\" (UID: \"89dc6442-73ce-4a37-bcf0-fc7aafb26602\") "
Jul 16 21:16:19.988906 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:19.988884    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89dc6442-73ce-4a37-bcf0-fc7aafb26602-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "89dc6442-73ce-4a37-bcf0-fc7aafb26602" (UID: "89dc6442-73ce-4a37-bcf0-fc7aafb26602"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:16:20.089395 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:20.089314    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89dc6442-73ce-4a37-bcf0-fc7aafb26602-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:16:20.729988 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:20.729326    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t"
Jul 16 21:16:20.729988 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:20.729402    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t" event={"ID":"89dc6442-73ce-4a37-bcf0-fc7aafb26602","Type":"ContainerDied","Data":"f72db503ac7a26533cbb15c59b325d2c9e446e4ff948c826cd63dbfcef2c110d"}
Jul 16 21:16:20.729988 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:20.729481    2576 scope.go:117] "RemoveContainer" containerID="730073a7fd8a3644a6a5e85c061264baa04e70bb5b3f0bf48cc87b332b00ceac"
Jul 16 21:16:20.748656 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:20.748624    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t"]
Jul 16 21:16:20.752462 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:20.752433    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-dzj5t"]
Jul 16 21:16:22.691603 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:22.691571    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89dc6442-73ce-4a37-bcf0-fc7aafb26602" path="/var/lib/kubelet/pods/89dc6442-73ce-4a37-bcf0-fc7aafb26602/volumes"
Jul 16 21:16:24.476617 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.476581    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"]
Jul 16 21:16:24.477152 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.477132    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89dc6442-73ce-4a37-bcf0-fc7aafb26602" containerName="revoke-keys"
Jul 16 21:16:24.477197 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.477156    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="89dc6442-73ce-4a37-bcf0-fc7aafb26602" containerName="revoke-keys"
Jul 16 21:16:24.477197 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.477184    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89dc6442-73ce-4a37-bcf0-fc7aafb26602" containerName="revoke-keys"
Jul 16 21:16:24.477197 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.477192    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="89dc6442-73ce-4a37-bcf0-fc7aafb26602" containerName="revoke-keys"
Jul 16 21:16:24.477289 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.477215    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d4e7ca2-6620-4fd6-948a-35e0987421de" containerName="revoke-keys"
Jul 16 21:16:24.477289 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.477223    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d4e7ca2-6620-4fd6-948a-35e0987421de" containerName="revoke-keys"
Jul 16 21:16:24.477289 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.477235    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d4e7ca2-6620-4fd6-948a-35e0987421de" containerName="revoke-keys"
Jul 16 21:16:24.477289 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.477242    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d4e7ca2-6620-4fd6-948a-35e0987421de" containerName="revoke-keys"
Jul 16 21:16:24.477289 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.477253    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d4e7ca2-6620-4fd6-948a-35e0987421de" containerName="revoke-keys"
Jul 16 21:16:24.477289 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.477261    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d4e7ca2-6620-4fd6-948a-35e0987421de" containerName="revoke-keys"
Jul 16 21:16:24.477516 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.477373    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d4e7ca2-6620-4fd6-948a-35e0987421de" containerName="revoke-keys"
Jul 16 21:16:24.477516 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.477385    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d4e7ca2-6620-4fd6-948a-35e0987421de" containerName="revoke-keys"
Jul 16 21:16:24.477516 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.477394    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="89dc6442-73ce-4a37-bcf0-fc7aafb26602" containerName="revoke-keys"
Jul 16 21:16:24.477516 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.477404    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d4e7ca2-6620-4fd6-948a-35e0987421de" containerName="revoke-keys"
Jul 16 21:16:24.477516 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.477412    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="89dc6442-73ce-4a37-bcf0-fc7aafb26602" containerName="revoke-keys"
Jul 16 21:16:24.477516 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.477420    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="89dc6442-73ce-4a37-bcf0-fc7aafb26602" containerName="revoke-keys"
Jul 16 21:16:24.482212 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.482187    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.486251 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.486229    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e9d9b90e\"/\"default-dockercfg-qh9q6\""
Jul 16 21:16:24.486469 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.486448    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e9d9b90e\"/\"openshift-service-ca.crt\""
Jul 16 21:16:24.486792 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.486384    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e9d9b90e\"/\"kube-root-ca.crt\""
Jul 16 21:16:24.486950 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.486292    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e9d9b90e\"/\"e2e-lifecycle-model-e9d9b90e-kserve-self-signed-certs\""
Jul 16 21:16:24.489024 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.489003    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"]
Jul 16 21:16:24.529829 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.529800    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92070908-1bc9-4644-8654-56d6dfbb36d8-tls-certs\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.529941 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.529851    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-dshm\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.530015 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.529939    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7nz8\" (UniqueName: \"kubernetes.io/projected/92070908-1bc9-4644-8654-56d6dfbb36d8-kube-api-access-f7nz8\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.530072 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.530030    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-model-cache\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.530072 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.530065    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-home\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.530183 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.530091    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-tmp-dir\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.631595 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.631563    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-dshm\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.631735 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.631632    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7nz8\" (UniqueName: \"kubernetes.io/projected/92070908-1bc9-4644-8654-56d6dfbb36d8-kube-api-access-f7nz8\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.631735 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.631708    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-model-cache\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.631863 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.631742    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-home\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.631863 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.631768    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-tmp-dir\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.631863 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.631811    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92070908-1bc9-4644-8654-56d6dfbb36d8-tls-certs\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.632130 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.632102    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-model-cache\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.632214 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.632177    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-home\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.632272 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.632215    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-tmp-dir\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.634629 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.634599    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-dshm\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.635003 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.634983    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92070908-1bc9-4644-8654-56d6dfbb36d8-tls-certs\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.640035 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.640012    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7nz8\" (UniqueName: \"kubernetes.io/projected/92070908-1bc9-4644-8654-56d6dfbb36d8-kube-api-access-f7nz8\") pod \"e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") " pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.794037 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.793967    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:24.925591 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:24.925506    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"]
Jul 16 21:16:24.928994 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:16:24.928946    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92070908_1bc9_4644_8654_56d6dfbb36d8.slice/crio-1c9caf5a4e4507537b13349231b4ae45a5be7b7502b3c2064930fc6716f40c60 WatchSource:0}: Error finding container 1c9caf5a4e4507537b13349231b4ae45a5be7b7502b3c2064930fc6716f40c60: Status 404 returned error can't find the container with id 1c9caf5a4e4507537b13349231b4ae45a5be7b7502b3c2064930fc6716f40c60
Jul 16 21:16:25.776950 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:25.776905    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8" event={"ID":"92070908-1bc9-4644-8654-56d6dfbb36d8","Type":"ContainerStarted","Data":"8e626db07792804ef5af56e87fa279554b731e7a45a79bba7099e821e0a5a57b"}
Jul 16 21:16:25.777432 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:25.776967    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8" event={"ID":"92070908-1bc9-4644-8654-56d6dfbb36d8","Type":"ContainerStarted","Data":"1c9caf5a4e4507537b13349231b4ae45a5be7b7502b3c2064930fc6716f40c60"}
Jul 16 21:16:25.795894 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:25.795836    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8" podStartSLOduration=1.795814858 podStartE2EDuration="1.795814858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:16:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:16:25.79432179 +0000 UTC m=+2631.760256014" watchObservedRunningTime="2026-07-16 21:16:25.795814858 +0000 UTC m=+2631.761749092"
Jul 16 21:16:27.685024 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:27.684996    2576 patch_prober.go:28] interesting pod/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 16 21:16:27.685493 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:27.685052    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs" podUID="7fffd3d7-e78f-4da4-ba31-0c813dff9024" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 16 21:16:27.788446 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:27.788422    2576 generic.go:356] "Generic (PLEG): container finished" podID="7fffd3d7-e78f-4da4-ba31-0c813dff9024" containerID="37455bc2cd0579b60cda4bd2fd67022dbd9c3e766751020b7fee848e8f065e4c" exitCode=0
Jul 16 21:16:27.788565 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:27.788494    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs" event={"ID":"7fffd3d7-e78f-4da4-ba31-0c813dff9024","Type":"ContainerDied","Data":"37455bc2cd0579b60cda4bd2fd67022dbd9c3e766751020b7fee848e8f065e4c"}
Jul 16 21:16:27.788849 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:27.788834    2576 scope.go:117] "RemoveContainer" containerID="37455bc2cd0579b60cda4bd2fd67022dbd9c3e766751020b7fee848e8f065e4c"
Jul 16 21:16:27.789790 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:27.789764    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:16:27.789843 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:27.789828    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:16:27.789881 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:27.789864    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:16:28.795452 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:28.795416    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs" event={"ID":"7fffd3d7-e78f-4da4-ba31-0c813dff9024","Type":"ContainerStarted","Data":"0998e0f346daf6f1d04e4dc2e536be02b9a08122cda962f3dc1878c38d192b6d"}
Jul 16 21:16:28.833513 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:28.833481    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:28.833842 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:28.833812    2576 patch_prober.go:28] interesting pod/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 16 21:16:28.833919 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:28.833878    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs" podUID="7fffd3d7-e78f-4da4-ba31-0c813dff9024" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 16 21:16:29.389642 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:29.389607    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:16:29.834004 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:29.833910    2576 patch_prober.go:28] interesting pod/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 16 21:16:29.834004 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:29.833967    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs" podUID="7fffd3d7-e78f-4da4-ba31-0c813dff9024" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 16 21:16:30.833870 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:30.833828    2576 patch_prober.go:28] interesting pod/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 16 21:16:30.834075 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:30.833890    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs" podUID="7fffd3d7-e78f-4da4-ba31-0c813dff9024" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 16 21:16:31.834916 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:31.834413    2576 patch_prober.go:28] interesting pod/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 16 21:16:31.834916 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:31.834526    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs" podUID="7fffd3d7-e78f-4da4-ba31-0c813dff9024" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 16 21:16:32.883561 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:32.883526    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:32.884025 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:32.883971    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:32.884673 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:32.884653    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:16:34.794243 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:34.794215    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:34.794618 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:34.794268    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:34.806682 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:34.806655    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:34.835581 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:34.835558    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:16:49.098424 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:16:49.098385    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:17:09.692009 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:09.691976    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:17:15.657371 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:15.657337    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"]
Jul 16 21:17:30.677717 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:30.677618    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8" podUID="92070908-1bc9-4644-8654-56d6dfbb36d8" containerName="main" containerID="cri-o://8e626db07792804ef5af56e87fa279554b731e7a45a79bba7099e821e0a5a57b" gracePeriod=45
Jul 16 21:17:30.933000 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:30.932948    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:17:30.972275 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:30.972250    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92070908-1bc9-4644-8654-56d6dfbb36d8-tls-certs\") pod \"92070908-1bc9-4644-8654-56d6dfbb36d8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") "
Jul 16 21:17:30.972420 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:30.972286    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7nz8\" (UniqueName: \"kubernetes.io/projected/92070908-1bc9-4644-8654-56d6dfbb36d8-kube-api-access-f7nz8\") pod \"92070908-1bc9-4644-8654-56d6dfbb36d8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") "
Jul 16 21:17:30.972420 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:30.972327    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-home\") pod \"92070908-1bc9-4644-8654-56d6dfbb36d8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") "
Jul 16 21:17:30.972420 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:30.972374    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-dshm\") pod \"92070908-1bc9-4644-8654-56d6dfbb36d8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") "
Jul 16 21:17:30.972551 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:30.972420    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-tmp-dir\") pod \"92070908-1bc9-4644-8654-56d6dfbb36d8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") "
Jul 16 21:17:30.972551 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:30.972466    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-model-cache\") pod \"92070908-1bc9-4644-8654-56d6dfbb36d8\" (UID: \"92070908-1bc9-4644-8654-56d6dfbb36d8\") "
Jul 16 21:17:30.972869 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:30.972817    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "92070908-1bc9-4644-8654-56d6dfbb36d8" (UID: "92070908-1bc9-4644-8654-56d6dfbb36d8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:17:30.973001 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:30.972877    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-home" (OuterVolumeSpecName: "home") pod "92070908-1bc9-4644-8654-56d6dfbb36d8" (UID: "92070908-1bc9-4644-8654-56d6dfbb36d8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:17:30.973001 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:30.972888    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-model-cache" (OuterVolumeSpecName: "model-cache") pod "92070908-1bc9-4644-8654-56d6dfbb36d8" (UID: "92070908-1bc9-4644-8654-56d6dfbb36d8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:17:30.974591 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:30.974559    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-dshm" (OuterVolumeSpecName: "dshm") pod "92070908-1bc9-4644-8654-56d6dfbb36d8" (UID: "92070908-1bc9-4644-8654-56d6dfbb36d8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:17:30.974711 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:30.974682    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92070908-1bc9-4644-8654-56d6dfbb36d8-kube-api-access-f7nz8" (OuterVolumeSpecName: "kube-api-access-f7nz8") pod "92070908-1bc9-4644-8654-56d6dfbb36d8" (UID: "92070908-1bc9-4644-8654-56d6dfbb36d8"). InnerVolumeSpecName "kube-api-access-f7nz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:17:30.974782 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:30.974761    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92070908-1bc9-4644-8654-56d6dfbb36d8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "92070908-1bc9-4644-8654-56d6dfbb36d8" (UID: "92070908-1bc9-4644-8654-56d6dfbb36d8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:17:31.073287 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:31.073264    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-dshm\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:17:31.073287 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:31.073287    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-tmp-dir\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:17:31.073462 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:31.073298    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-model-cache\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:17:31.073462 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:31.073305    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92070908-1bc9-4644-8654-56d6dfbb36d8-tls-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:17:31.073462 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:31.073313    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7nz8\" (UniqueName: \"kubernetes.io/projected/92070908-1bc9-4644-8654-56d6dfbb36d8-kube-api-access-f7nz8\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:17:31.073462 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:31.073321    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92070908-1bc9-4644-8654-56d6dfbb36d8-home\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:17:31.102723 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:31.102699    2576 generic.go:356] "Generic (PLEG): container finished" podID="92070908-1bc9-4644-8654-56d6dfbb36d8" containerID="8e626db07792804ef5af56e87fa279554b731e7a45a79bba7099e821e0a5a57b" exitCode=0
Jul 16 21:17:31.102821 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:31.102735    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8" event={"ID":"92070908-1bc9-4644-8654-56d6dfbb36d8","Type":"ContainerDied","Data":"8e626db07792804ef5af56e87fa279554b731e7a45a79bba7099e821e0a5a57b"}
Jul 16 21:17:31.102821 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:31.102740    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"
Jul 16 21:17:31.102821 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:31.102762    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8" event={"ID":"92070908-1bc9-4644-8654-56d6dfbb36d8","Type":"ContainerDied","Data":"1c9caf5a4e4507537b13349231b4ae45a5be7b7502b3c2064930fc6716f40c60"}
Jul 16 21:17:31.102821 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:31.102779    2576 scope.go:117] "RemoveContainer" containerID="8e626db07792804ef5af56e87fa279554b731e7a45a79bba7099e821e0a5a57b"
Jul 16 21:17:31.112120 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:31.112104    2576 scope.go:117] "RemoveContainer" containerID="8e626db07792804ef5af56e87fa279554b731e7a45a79bba7099e821e0a5a57b"
Jul 16 21:17:31.112413 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:17:31.112352    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e626db07792804ef5af56e87fa279554b731e7a45a79bba7099e821e0a5a57b\": container with ID starting with 8e626db07792804ef5af56e87fa279554b731e7a45a79bba7099e821e0a5a57b not found: ID does not exist" containerID="8e626db07792804ef5af56e87fa279554b731e7a45a79bba7099e821e0a5a57b"
Jul 16 21:17:31.112469 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:31.112423    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e626db07792804ef5af56e87fa279554b731e7a45a79bba7099e821e0a5a57b"} err="failed to get container status \"8e626db07792804ef5af56e87fa279554b731e7a45a79bba7099e821e0a5a57b\": rpc error: code = NotFound desc = could not find container \"8e626db07792804ef5af56e87fa279554b731e7a45a79bba7099e821e0a5a57b\": container with ID starting with 8e626db07792804ef5af56e87fa279554b731e7a45a79bba7099e821e0a5a57b not found: ID does not exist"
Jul 16 21:17:31.128666 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:31.128640    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"]
Jul 16 21:17:31.135192 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:31.135175    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-e9d9b90e/e2e-lifecycle-model-e9d9b90e-kserve-7d9865fd47-c7kn8"]
Jul 16 21:17:32.691056 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:32.691018    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92070908-1bc9-4644-8654-56d6dfbb36d8" path="/var/lib/kubelet/pods/92070908-1bc9-4644-8654-56d6dfbb36d8/volumes"
Jul 16 21:17:34.673924 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:34.673892    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 21:17:34.677839 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:34.677817    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 21:17:41.869884 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:41.869850    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl"]
Jul 16 21:17:41.870406 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:41.870224    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92070908-1bc9-4644-8654-56d6dfbb36d8" containerName="main"
Jul 16 21:17:41.870406 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:41.870235    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="92070908-1bc9-4644-8654-56d6dfbb36d8" containerName="main"
Jul 16 21:17:41.870406 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:41.870259    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89dc6442-73ce-4a37-bcf0-fc7aafb26602" containerName="revoke-keys"
Jul 16 21:17:41.870406 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:41.870264    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="89dc6442-73ce-4a37-bcf0-fc7aafb26602" containerName="revoke-keys"
Jul 16 21:17:41.870406 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:41.870327    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="92070908-1bc9-4644-8654-56d6dfbb36d8" containerName="main"
Jul 16 21:17:41.874969 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:41.874952    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl"
Jul 16 21:17:41.877322 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:41.877293    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:17:41.877322 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:41.877299    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:17:41.880860 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:41.880833    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl"]
Jul 16 21:17:41.965380 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:41.965334    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a43d3d8-2540-4b72-97f1-246a58378d1c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl\" (UID: \"0a43d3d8-2540-4b72-97f1-246a58378d1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl"
Jul 16 21:17:42.066090 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:42.066064    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a43d3d8-2540-4b72-97f1-246a58378d1c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl\" (UID: \"0a43d3d8-2540-4b72-97f1-246a58378d1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl"
Jul 16 21:17:42.066650 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:42.066632    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a43d3d8-2540-4b72-97f1-246a58378d1c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl\" (UID: \"0a43d3d8-2540-4b72-97f1-246a58378d1c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl"
Jul 16 21:17:42.186913 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:42.186893    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl"
Jul 16 21:17:42.316198 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:42.316174    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl"]
Jul 16 21:17:42.317273 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:17:42.317247    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a43d3d8_2540_4b72_97f1_246a58378d1c.slice/crio-dcbcc4748581086d80d94d616991b176f53d209158bb04b093e19684ae0a1afb WatchSource:0}: Error finding container dcbcc4748581086d80d94d616991b176f53d209158bb04b093e19684ae0a1afb: Status 404 returned error can't find the container with id dcbcc4748581086d80d94d616991b176f53d209158bb04b093e19684ae0a1afb
Jul 16 21:17:43.165653 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:43.165621    2576 generic.go:356] "Generic (PLEG): container finished" podID="0a43d3d8-2540-4b72-97f1-246a58378d1c" containerID="acf8b689553bb0cda4fb5dd1fb6cb6fd385912f31887ac4c5841157fed97f5fc" exitCode=6
Jul 16 21:17:43.166018 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:43.165696    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl" event={"ID":"0a43d3d8-2540-4b72-97f1-246a58378d1c","Type":"ContainerDied","Data":"acf8b689553bb0cda4fb5dd1fb6cb6fd385912f31887ac4c5841157fed97f5fc"}
Jul 16 21:17:43.166018 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:43.165747    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl" event={"ID":"0a43d3d8-2540-4b72-97f1-246a58378d1c","Type":"ContainerStarted","Data":"dcbcc4748581086d80d94d616991b176f53d209158bb04b093e19684ae0a1afb"}
Jul 16 21:17:43.166018 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:43.165830    2576 scope.go:117] "RemoveContainer" containerID="acf8b689553bb0cda4fb5dd1fb6cb6fd385912f31887ac4c5841157fed97f5fc"
Jul 16 21:17:44.172737 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:44.172707    2576 generic.go:356] "Generic (PLEG): container finished" podID="0a43d3d8-2540-4b72-97f1-246a58378d1c" containerID="82a0dba2a4adb0b498adfe3d3416b6a941a7353e1ddd07237ba3f1181a46dd63" exitCode=6
Jul 16 21:17:44.173093 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:44.172773    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl" event={"ID":"0a43d3d8-2540-4b72-97f1-246a58378d1c","Type":"ContainerDied","Data":"82a0dba2a4adb0b498adfe3d3416b6a941a7353e1ddd07237ba3f1181a46dd63"}
Jul 16 21:17:44.173093 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:44.172819    2576 scope.go:117] "RemoveContainer" containerID="acf8b689553bb0cda4fb5dd1fb6cb6fd385912f31887ac4c5841157fed97f5fc"
Jul 16 21:17:44.173093 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:44.172968    2576 scope.go:117] "RemoveContainer" containerID="82a0dba2a4adb0b498adfe3d3416b6a941a7353e1ddd07237ba3f1181a46dd63"
Jul 16 21:17:44.173214 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:17:44.173175    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl_odh-ai-gateway-infra(0a43d3d8-2540-4b72-97f1-246a58378d1c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl" podUID="0a43d3d8-2540-4b72-97f1-246a58378d1c"
Jul 16 21:17:45.179396 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:45.179350    2576 scope.go:117] "RemoveContainer" containerID="82a0dba2a4adb0b498adfe3d3416b6a941a7353e1ddd07237ba3f1181a46dd63"
Jul 16 21:17:45.179774 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:17:45.179541    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl_odh-ai-gateway-infra(0a43d3d8-2540-4b72-97f1-246a58378d1c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl" podUID="0a43d3d8-2540-4b72-97f1-246a58378d1c"
Jul 16 21:17:58.684452 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:58.684420    2576 scope.go:117] "RemoveContainer" containerID="82a0dba2a4adb0b498adfe3d3416b6a941a7353e1ddd07237ba3f1181a46dd63"
Jul 16 21:17:59.248040 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:59.248013    2576 generic.go:356] "Generic (PLEG): container finished" podID="0a43d3d8-2540-4b72-97f1-246a58378d1c" containerID="810e509b4fd8e2da4a5488dd271ef8710e79eb706db5ed7a779d8120ef33ce1f" exitCode=6
Jul 16 21:17:59.248160 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:59.248049    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl" event={"ID":"0a43d3d8-2540-4b72-97f1-246a58378d1c","Type":"ContainerDied","Data":"810e509b4fd8e2da4a5488dd271ef8710e79eb706db5ed7a779d8120ef33ce1f"}
Jul 16 21:17:59.248160 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:59.248096    2576 scope.go:117] "RemoveContainer" containerID="82a0dba2a4adb0b498adfe3d3416b6a941a7353e1ddd07237ba3f1181a46dd63"
Jul 16 21:17:59.248432 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:17:59.248413    2576 scope.go:117] "RemoveContainer" containerID="810e509b4fd8e2da4a5488dd271ef8710e79eb706db5ed7a779d8120ef33ce1f"
Jul 16 21:17:59.248658 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:17:59.248637    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl_odh-ai-gateway-infra(0a43d3d8-2540-4b72-97f1-246a58378d1c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl" podUID="0a43d3d8-2540-4b72-97f1-246a58378d1c"
Jul 16 21:18:00.273373 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:00.273327    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl"]
Jul 16 21:18:00.408276 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:00.408253    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl"
Jul 16 21:18:00.506890 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:00.506861    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a43d3d8-2540-4b72-97f1-246a58378d1c-maas-api-service-ca\") pod \"0a43d3d8-2540-4b72-97f1-246a58378d1c\" (UID: \"0a43d3d8-2540-4b72-97f1-246a58378d1c\") "
Jul 16 21:18:00.507186 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:00.507163    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a43d3d8-2540-4b72-97f1-246a58378d1c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0a43d3d8-2540-4b72-97f1-246a58378d1c" (UID: "0a43d3d8-2540-4b72-97f1-246a58378d1c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:18:00.608146 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:00.608087    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a43d3d8-2540-4b72-97f1-246a58378d1c-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:18:01.262288 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:01.262261    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl"
Jul 16 21:18:01.262454 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:01.262300    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl" event={"ID":"0a43d3d8-2540-4b72-97f1-246a58378d1c","Type":"ContainerDied","Data":"dcbcc4748581086d80d94d616991b176f53d209158bb04b093e19684ae0a1afb"}
Jul 16 21:18:01.262454 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:01.262392    2576 scope.go:117] "RemoveContainer" containerID="810e509b4fd8e2da4a5488dd271ef8710e79eb706db5ed7a779d8120ef33ce1f"
Jul 16 21:18:01.280757 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:01.280730    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl"]
Jul 16 21:18:01.286249 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:01.286226    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-mkcjl"]
Jul 16 21:18:02.689800 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:02.689765    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a43d3d8-2540-4b72-97f1-246a58378d1c" path="/var/lib/kubelet/pods/0a43d3d8-2540-4b72-97f1-246a58378d1c/volumes"
Jul 16 21:18:23.086933 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:23.086899    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"]
Jul 16 21:18:23.087463 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:23.087115    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs" podUID="7fffd3d7-e78f-4da4-ba31-0c813dff9024" containerName="istio-proxy" containerID="cri-o://0998e0f346daf6f1d04e4dc2e536be02b9a08122cda962f3dc1878c38d192b6d" gracePeriod=30
Jul 16 21:18:28.341290 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.341265    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:18:28.403244 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.403214    2576 generic.go:356] "Generic (PLEG): container finished" podID="7fffd3d7-e78f-4da4-ba31-0c813dff9024" containerID="0998e0f346daf6f1d04e4dc2e536be02b9a08122cda962f3dc1878c38d192b6d" exitCode=0
Jul 16 21:18:28.403399 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.403255    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"
Jul 16 21:18:28.403399 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.403289    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs" event={"ID":"7fffd3d7-e78f-4da4-ba31-0c813dff9024","Type":"ContainerDied","Data":"0998e0f346daf6f1d04e4dc2e536be02b9a08122cda962f3dc1878c38d192b6d"}
Jul 16 21:18:28.403399 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.403333    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs" event={"ID":"7fffd3d7-e78f-4da4-ba31-0c813dff9024","Type":"ContainerDied","Data":"1ee6d566730d7f854bd61518559e6c6435f27e35a10ea34748f9602eea3abe2f"}
Jul 16 21:18:28.403399 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.403373    2576 scope.go:117] "RemoveContainer" containerID="0998e0f346daf6f1d04e4dc2e536be02b9a08122cda962f3dc1878c38d192b6d"
Jul 16 21:18:28.411371 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.411346    2576 scope.go:117] "RemoveContainer" containerID="37455bc2cd0579b60cda4bd2fd67022dbd9c3e766751020b7fee848e8f065e4c"
Jul 16 21:18:28.418713 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.418689    2576 scope.go:117] "RemoveContainer" containerID="0998e0f346daf6f1d04e4dc2e536be02b9a08122cda962f3dc1878c38d192b6d"
Jul 16 21:18:28.418956 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:18:28.418940    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0998e0f346daf6f1d04e4dc2e536be02b9a08122cda962f3dc1878c38d192b6d\": container with ID starting with 0998e0f346daf6f1d04e4dc2e536be02b9a08122cda962f3dc1878c38d192b6d not found: ID does not exist" containerID="0998e0f346daf6f1d04e4dc2e536be02b9a08122cda962f3dc1878c38d192b6d"
Jul 16 21:18:28.419000 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.418964    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0998e0f346daf6f1d04e4dc2e536be02b9a08122cda962f3dc1878c38d192b6d"} err="failed to get container status \"0998e0f346daf6f1d04e4dc2e536be02b9a08122cda962f3dc1878c38d192b6d\": rpc error: code = NotFound desc = could not find container \"0998e0f346daf6f1d04e4dc2e536be02b9a08122cda962f3dc1878c38d192b6d\": container with ID starting with 0998e0f346daf6f1d04e4dc2e536be02b9a08122cda962f3dc1878c38d192b6d not found: ID does not exist"
Jul 16 21:18:28.419000 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.418978    2576 scope.go:117] "RemoveContainer" containerID="37455bc2cd0579b60cda4bd2fd67022dbd9c3e766751020b7fee848e8f065e4c"
Jul 16 21:18:28.419177 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:18:28.419162    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37455bc2cd0579b60cda4bd2fd67022dbd9c3e766751020b7fee848e8f065e4c\": container with ID starting with 37455bc2cd0579b60cda4bd2fd67022dbd9c3e766751020b7fee848e8f065e4c not found: ID does not exist" containerID="37455bc2cd0579b60cda4bd2fd67022dbd9c3e766751020b7fee848e8f065e4c"
Jul 16 21:18:28.419226 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.419180    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37455bc2cd0579b60cda4bd2fd67022dbd9c3e766751020b7fee848e8f065e4c"} err="failed to get container status \"37455bc2cd0579b60cda4bd2fd67022dbd9c3e766751020b7fee848e8f065e4c\": rpc error: code = NotFound desc = could not find container \"37455bc2cd0579b60cda4bd2fd67022dbd9c3e766751020b7fee848e8f065e4c\": container with ID starting with 37455bc2cd0579b60cda4bd2fd67022dbd9c3e766751020b7fee848e8f065e4c not found: ID does not exist"
Jul 16 21:18:28.429519 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.429500    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-credential-socket\") pod \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") "
Jul 16 21:18:28.429612 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.429533    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istiod-ca-cert\") pod \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") "
Jul 16 21:18:28.429612 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.429561    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-podinfo\") pod \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") "
Jul 16 21:18:28.429612 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.429598    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-token\") pod \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") "
Jul 16 21:18:28.429770 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.429642    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-workload-certs\") pod \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") "
Jul 16 21:18:28.429770 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.429695    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qw7tk\" (UniqueName: \"kubernetes.io/projected/7fffd3d7-e78f-4da4-ba31-0c813dff9024-kube-api-access-qw7tk\") pod \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") "
Jul 16 21:18:28.429770 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.429731    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-data\") pod \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") "
Jul 16 21:18:28.429923 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.429787    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-workload-socket\") pod \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") "
Jul 16 21:18:28.429923 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.429771    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7fffd3d7-e78f-4da4-ba31-0c813dff9024" (UID: "7fffd3d7-e78f-4da4-ba31-0c813dff9024"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:18:28.429923 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.429812    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-envoy\") pod \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\" (UID: \"7fffd3d7-e78f-4da4-ba31-0c813dff9024\") "
Jul 16 21:18:28.430075 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.429958    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7fffd3d7-e78f-4da4-ba31-0c813dff9024" (UID: "7fffd3d7-e78f-4da4-ba31-0c813dff9024"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:18:28.430174 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.430154    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-credential-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:18:28.430247 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.430179    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istiod-ca-cert\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:18:28.430316 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.430295    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7fffd3d7-e78f-4da4-ba31-0c813dff9024" (UID: "7fffd3d7-e78f-4da4-ba31-0c813dff9024"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:18:28.430522 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.430372    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7fffd3d7-e78f-4da4-ba31-0c813dff9024" (UID: "7fffd3d7-e78f-4da4-ba31-0c813dff9024"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:18:28.430625 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.430600    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-data" (OuterVolumeSpecName: "istio-data") pod "7fffd3d7-e78f-4da4-ba31-0c813dff9024" (UID: "7fffd3d7-e78f-4da4-ba31-0c813dff9024"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:18:28.431870 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.431841    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7fffd3d7-e78f-4da4-ba31-0c813dff9024" (UID: "7fffd3d7-e78f-4da4-ba31-0c813dff9024"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 21:18:28.431983 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.431959    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7fffd3d7-e78f-4da4-ba31-0c813dff9024" (UID: "7fffd3d7-e78f-4da4-ba31-0c813dff9024"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:18:28.432544 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.432511    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fffd3d7-e78f-4da4-ba31-0c813dff9024-kube-api-access-qw7tk" (OuterVolumeSpecName: "kube-api-access-qw7tk") pod "7fffd3d7-e78f-4da4-ba31-0c813dff9024" (UID: "7fffd3d7-e78f-4da4-ba31-0c813dff9024"). InnerVolumeSpecName "kube-api-access-qw7tk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:18:28.432544 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.432528    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-token" (OuterVolumeSpecName: "istio-token") pod "7fffd3d7-e78f-4da4-ba31-0c813dff9024" (UID: "7fffd3d7-e78f-4da4-ba31-0c813dff9024"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:18:28.531424 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.531397    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-workload-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:18:28.531424 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.531417    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qw7tk\" (UniqueName: \"kubernetes.io/projected/7fffd3d7-e78f-4da4-ba31-0c813dff9024-kube-api-access-qw7tk\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:18:28.531551 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.531430    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-data\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:18:28.531551 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.531444    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-workload-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:18:28.531551 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.531453    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-envoy\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:18:28.531551 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.531461    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-podinfo\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:18:28.531551 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.531468    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fffd3d7-e78f-4da4-ba31-0c813dff9024-istio-token\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:18:28.721189 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.721165    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"]
Jul 16 21:18:28.725559 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:28.725539    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e9d9b90e-openshift-default-75d97796dc-82fxs"]
Jul 16 21:18:30.689427 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:30.689389    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fffd3d7-e78f-4da4-ba31-0c813dff9024" path="/var/lib/kubelet/pods/7fffd3d7-e78f-4da4-ba31-0c813dff9024/volumes"
Jul 16 21:18:32.200791 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.200754    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh"]
Jul 16 21:18:32.201155 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.201129    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a43d3d8-2540-4b72-97f1-246a58378d1c" containerName="revoke-keys"
Jul 16 21:18:32.201155 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.201141    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a43d3d8-2540-4b72-97f1-246a58378d1c" containerName="revoke-keys"
Jul 16 21:18:32.201155 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.201155    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a43d3d8-2540-4b72-97f1-246a58378d1c" containerName="revoke-keys"
Jul 16 21:18:32.201261 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.201161    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a43d3d8-2540-4b72-97f1-246a58378d1c" containerName="revoke-keys"
Jul 16 21:18:32.201261 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.201169    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fffd3d7-e78f-4da4-ba31-0c813dff9024" containerName="istio-proxy"
Jul 16 21:18:32.201261 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.201174    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fffd3d7-e78f-4da4-ba31-0c813dff9024" containerName="istio-proxy"
Jul 16 21:18:32.201261 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.201235    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a43d3d8-2540-4b72-97f1-246a58378d1c" containerName="revoke-keys"
Jul 16 21:18:32.201261 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.201243    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fffd3d7-e78f-4da4-ba31-0c813dff9024" containerName="istio-proxy"
Jul 16 21:18:32.201261 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.201251    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a43d3d8-2540-4b72-97f1-246a58378d1c" containerName="revoke-keys"
Jul 16 21:18:32.205772 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.205752    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh"
Jul 16 21:18:32.208215 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.208183    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:18:32.208329 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.208227    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:18:32.211773 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.211746    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh"]
Jul 16 21:18:32.259718 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.259696    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c49ae3d-ce20-49d2-be08-f20690126e50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh\" (UID: \"0c49ae3d-ce20-49d2-be08-f20690126e50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh"
Jul 16 21:18:32.360893 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.360874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c49ae3d-ce20-49d2-be08-f20690126e50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh\" (UID: \"0c49ae3d-ce20-49d2-be08-f20690126e50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh"
Jul 16 21:18:32.361463 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.361444    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c49ae3d-ce20-49d2-be08-f20690126e50-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh\" (UID: \"0c49ae3d-ce20-49d2-be08-f20690126e50\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh"
Jul 16 21:18:32.518879 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.518824    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh"
Jul 16 21:18:32.645983 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:32.645952    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh"]
Jul 16 21:18:32.648395 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:18:32.648338    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c49ae3d_ce20_49d2_be08_f20690126e50.slice/crio-1591a89ae7ec629ed3a849438f091b496b9843e4bcc4f57853c4c319ed7b1b26 WatchSource:0}: Error finding container 1591a89ae7ec629ed3a849438f091b496b9843e4bcc4f57853c4c319ed7b1b26: Status 404 returned error can't find the container with id 1591a89ae7ec629ed3a849438f091b496b9843e4bcc4f57853c4c319ed7b1b26
Jul 16 21:18:33.431586 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:33.431555    2576 generic.go:356] "Generic (PLEG): container finished" podID="0c49ae3d-ce20-49d2-be08-f20690126e50" containerID="ae0747b0cb550c7f24cef9864d988922c12933848763dcc5ae2f1b0d6613b950" exitCode=6
Jul 16 21:18:33.431946 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:33.431648    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh" event={"ID":"0c49ae3d-ce20-49d2-be08-f20690126e50","Type":"ContainerDied","Data":"ae0747b0cb550c7f24cef9864d988922c12933848763dcc5ae2f1b0d6613b950"}
Jul 16 21:18:33.431946 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:33.431712    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh" event={"ID":"0c49ae3d-ce20-49d2-be08-f20690126e50","Type":"ContainerStarted","Data":"1591a89ae7ec629ed3a849438f091b496b9843e4bcc4f57853c4c319ed7b1b26"}
Jul 16 21:18:33.431946 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:33.431766    2576 scope.go:117] "RemoveContainer" containerID="ae0747b0cb550c7f24cef9864d988922c12933848763dcc5ae2f1b0d6613b950"
Jul 16 21:18:33.590711 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:33.590654    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:18:34.438277 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:34.438252    2576 generic.go:356] "Generic (PLEG): container finished" podID="0c49ae3d-ce20-49d2-be08-f20690126e50" containerID="a6540c13ad50c6e80062c916024411b4a10331eaf0565dca339339f6d2798728" exitCode=6
Jul 16 21:18:34.438672 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:34.438329    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh" event={"ID":"0c49ae3d-ce20-49d2-be08-f20690126e50","Type":"ContainerDied","Data":"a6540c13ad50c6e80062c916024411b4a10331eaf0565dca339339f6d2798728"}
Jul 16 21:18:34.438672 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:34.438399    2576 scope.go:117] "RemoveContainer" containerID="ae0747b0cb550c7f24cef9864d988922c12933848763dcc5ae2f1b0d6613b950"
Jul 16 21:18:34.438672 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:34.438545    2576 scope.go:117] "RemoveContainer" containerID="a6540c13ad50c6e80062c916024411b4a10331eaf0565dca339339f6d2798728"
Jul 16 21:18:34.438814 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:18:34.438715    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh_odh-ai-gateway-infra(0c49ae3d-ce20-49d2-be08-f20690126e50)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh" podUID="0c49ae3d-ce20-49d2-be08-f20690126e50"
Jul 16 21:18:35.445514 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:35.445491    2576 scope.go:117] "RemoveContainer" containerID="a6540c13ad50c6e80062c916024411b4a10331eaf0565dca339339f6d2798728"
Jul 16 21:18:35.445905 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:18:35.445658    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh_odh-ai-gateway-infra(0c49ae3d-ce20-49d2-be08-f20690126e50)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh" podUID="0c49ae3d-ce20-49d2-be08-f20690126e50"
Jul 16 21:18:41.595630 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:41.595593    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:18:49.684256 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:49.684223    2576 scope.go:117] "RemoveContainer" containerID="a6540c13ad50c6e80062c916024411b4a10331eaf0565dca339339f6d2798728"
Jul 16 21:18:50.524187 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:50.524123    2576 generic.go:356] "Generic (PLEG): container finished" podID="0c49ae3d-ce20-49d2-be08-f20690126e50" containerID="eb43f11428c31d9bfaf8e272e1d814f7b35a008b7c9b14883c497d8fe45d17ca" exitCode=6
Jul 16 21:18:50.524303 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:50.524187    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh" event={"ID":"0c49ae3d-ce20-49d2-be08-f20690126e50","Type":"ContainerDied","Data":"eb43f11428c31d9bfaf8e272e1d814f7b35a008b7c9b14883c497d8fe45d17ca"}
Jul 16 21:18:50.524303 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:50.524250    2576 scope.go:117] "RemoveContainer" containerID="a6540c13ad50c6e80062c916024411b4a10331eaf0565dca339339f6d2798728"
Jul 16 21:18:50.524526 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:50.524512    2576 scope.go:117] "RemoveContainer" containerID="eb43f11428c31d9bfaf8e272e1d814f7b35a008b7c9b14883c497d8fe45d17ca"
Jul 16 21:18:50.524738 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:18:50.524715    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh_odh-ai-gateway-infra(0c49ae3d-ce20-49d2-be08-f20690126e50)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh" podUID="0c49ae3d-ce20-49d2-be08-f20690126e50"
Jul 16 21:18:51.553451 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:51.553404    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh"]
Jul 16 21:18:51.691329 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:51.691308    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh"
Jul 16 21:18:51.818200 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:51.818140    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c49ae3d-ce20-49d2-be08-f20690126e50-maas-api-service-ca\") pod \"0c49ae3d-ce20-49d2-be08-f20690126e50\" (UID: \"0c49ae3d-ce20-49d2-be08-f20690126e50\") "
Jul 16 21:18:51.818472 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:51.818453    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c49ae3d-ce20-49d2-be08-f20690126e50-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0c49ae3d-ce20-49d2-be08-f20690126e50" (UID: "0c49ae3d-ce20-49d2-be08-f20690126e50"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:18:51.918989 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:51.918967    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c49ae3d-ce20-49d2-be08-f20690126e50-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:18:52.538242 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:52.538215    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh"
Jul 16 21:18:52.538434 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:52.538248    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh" event={"ID":"0c49ae3d-ce20-49d2-be08-f20690126e50","Type":"ContainerDied","Data":"1591a89ae7ec629ed3a849438f091b496b9843e4bcc4f57853c4c319ed7b1b26"}
Jul 16 21:18:52.538434 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:52.538321    2576 scope.go:117] "RemoveContainer" containerID="eb43f11428c31d9bfaf8e272e1d814f7b35a008b7c9b14883c497d8fe45d17ca"
Jul 16 21:18:52.560510 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:52.560483    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh"]
Jul 16 21:18:52.567613 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:52.567587    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad09ff70-xwmqh"]
Jul 16 21:18:52.690137 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:18:52.690095    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c49ae3d-ce20-49d2-be08-f20690126e50" path="/var/lib/kubelet/pods/0c49ae3d-ce20-49d2-be08-f20690126e50/volumes"
Jul 16 21:19:18.935376 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:18.935285    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"]
Jul 16 21:19:18.935865 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:18.935845    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c49ae3d-ce20-49d2-be08-f20690126e50" containerName="revoke-keys"
Jul 16 21:19:18.935910 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:18.935870    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c49ae3d-ce20-49d2-be08-f20690126e50" containerName="revoke-keys"
Jul 16 21:19:18.935910 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:18.935885    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c49ae3d-ce20-49d2-be08-f20690126e50" containerName="revoke-keys"
Jul 16 21:19:18.935910 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:18.935894    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c49ae3d-ce20-49d2-be08-f20690126e50" containerName="revoke-keys"
Jul 16 21:19:18.936010 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:18.935909    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a43d3d8-2540-4b72-97f1-246a58378d1c" containerName="revoke-keys"
Jul 16 21:19:18.936010 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:18.935919    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a43d3d8-2540-4b72-97f1-246a58378d1c" containerName="revoke-keys"
Jul 16 21:19:18.936010 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:18.935936    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fffd3d7-e78f-4da4-ba31-0c813dff9024" containerName="istio-proxy"
Jul 16 21:19:18.936010 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:18.935944    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fffd3d7-e78f-4da4-ba31-0c813dff9024" containerName="istio-proxy"
Jul 16 21:19:18.936128 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:18.936079    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a43d3d8-2540-4b72-97f1-246a58378d1c" containerName="revoke-keys"
Jul 16 21:19:18.936128 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:18.936091    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c49ae3d-ce20-49d2-be08-f20690126e50" containerName="revoke-keys"
Jul 16 21:19:18.936128 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:18.936102    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c49ae3d-ce20-49d2-be08-f20690126e50" containerName="revoke-keys"
Jul 16 21:19:18.936128 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:18.936111    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fffd3d7-e78f-4da4-ba31-0c813dff9024" containerName="istio-proxy"
Jul 16 21:19:18.939489 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:18.939462    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:18.942283 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:18.942259    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c69b60-openshift-default-dockercfg-gxb9d\""
Jul 16 21:19:18.948286 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:18.948266    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"]
Jul 16 21:19:19.029749 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.029725    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-istio-data\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.029882 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.029763    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-credential-socket\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.029882 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.029811    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6sfj\" (UniqueName: \"kubernetes.io/projected/795b38f7-b667-4392-aace-0952157f189e-kube-api-access-c6sfj\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.029882 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.029837    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/795b38f7-b667-4392-aace-0952157f189e-istio-token\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.029882 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.029866    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-workload-socket\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.030067 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.029929    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/795b38f7-b667-4392-aace-0952157f189e-istiod-ca-cert\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.030067 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.029960    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-workload-certs\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.030067 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.029999    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/795b38f7-b667-4392-aace-0952157f189e-istio-podinfo\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.030181 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.030101    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-istio-envoy\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.131170 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.131141    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/795b38f7-b667-4392-aace-0952157f189e-istio-podinfo\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.131333 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.131183    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-istio-envoy\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.131333 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.131200    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-istio-data\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.131333 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.131230    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-credential-socket\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.131333 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.131281    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6sfj\" (UniqueName: \"kubernetes.io/projected/795b38f7-b667-4392-aace-0952157f189e-kube-api-access-c6sfj\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.131333 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.131309    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/795b38f7-b667-4392-aace-0952157f189e-istio-token\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.131905 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.131634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-workload-socket\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.131905 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.131721    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-credential-socket\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.131905 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.131729    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/795b38f7-b667-4392-aace-0952157f189e-istiod-ca-cert\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.131905 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.131799    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-workload-certs\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.131905 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.131635    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-istio-data\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.132209 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.131961    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-workload-socket\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.132209 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.132107    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-workload-certs\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.132413 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.132392    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/795b38f7-b667-4392-aace-0952157f189e-istiod-ca-cert\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.133713 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.133691    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-istio-envoy\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.134209 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.134193    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/795b38f7-b667-4392-aace-0952157f189e-istio-podinfo\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.139644 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.139615    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6sfj\" (UniqueName: \"kubernetes.io/projected/795b38f7-b667-4392-aace-0952157f189e-kube-api-access-c6sfj\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.139644 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.139623    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/795b38f7-b667-4392-aace-0952157f189e-istio-token\") pod \"e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") " pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.251308 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.251252    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:19.388729 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.388705    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"]
Jul 16 21:19:19.390428 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:19:19.390400    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod795b38f7_b667_4392_aace_0952157f189e.slice/crio-63835c78319c486cc3d5c896cbe3f175241f4f487c29e097e1166796e8a22938 WatchSource:0}: Error finding container 63835c78319c486cc3d5c896cbe3f175241f4f487c29e097e1166796e8a22938: Status 404 returned error can't find the container with id 63835c78319c486cc3d5c896cbe3f175241f4f487c29e097e1166796e8a22938
Jul 16 21:19:19.392648 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.392606    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:19:19.392748 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.392693    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:19:19.392748 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.392739    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:19:19.675040 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.675006    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp" event={"ID":"795b38f7-b667-4392-aace-0952157f189e","Type":"ContainerStarted","Data":"45f5b78f52f905f262643aeab7407fc30324a929ca0136f9e3d74cdcd103a560"}
Jul 16 21:19:19.675206 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.675055    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp" event={"ID":"795b38f7-b667-4392-aace-0952157f189e","Type":"ContainerStarted","Data":"63835c78319c486cc3d5c896cbe3f175241f4f487c29e097e1166796e8a22938"}
Jul 16 21:19:19.697234 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:19.697192    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp" podStartSLOduration=1.697177202 podStartE2EDuration="1.697177202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:19:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:19:19.69470505 +0000 UTC m=+2805.660639294" watchObservedRunningTime="2026-07-16 21:19:19.697177202 +0000 UTC m=+2805.663111539"
Jul 16 21:19:20.251672 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:20.251643    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:20.256138 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:20.256113    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:20.680515 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:20.680490    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:20.681494 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:20.681472    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:21.983984 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:21.983948    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v"]
Jul 16 21:19:21.984348 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:21.984336    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c49ae3d-ce20-49d2-be08-f20690126e50" containerName="revoke-keys"
Jul 16 21:19:21.984426 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:21.984349    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c49ae3d-ce20-49d2-be08-f20690126e50" containerName="revoke-keys"
Jul 16 21:19:21.984463 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:21.984449    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c49ae3d-ce20-49d2-be08-f20690126e50" containerName="revoke-keys"
Jul 16 21:19:21.987701 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:21.987685    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v"
Jul 16 21:19:21.990934 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:21.990908    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:19:21.991055 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:21.991020    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:19:22.007642 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:22.007615    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v"]
Jul 16 21:19:22.053604 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:22.053567    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0761ff9-39eb-43ee-91af-e8a4eb48c9f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v\" (UID: \"b0761ff9-39eb-43ee-91af-e8a4eb48c9f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v"
Jul 16 21:19:22.154089 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:22.154063    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0761ff9-39eb-43ee-91af-e8a4eb48c9f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v\" (UID: \"b0761ff9-39eb-43ee-91af-e8a4eb48c9f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v"
Jul 16 21:19:22.154670 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:22.154652    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0761ff9-39eb-43ee-91af-e8a4eb48c9f8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v\" (UID: \"b0761ff9-39eb-43ee-91af-e8a4eb48c9f8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v"
Jul 16 21:19:22.298076 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:22.298013    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v"
Jul 16 21:19:22.450078 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:22.450051    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v"]
Jul 16 21:19:22.452266 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:19:22.452237    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0761ff9_39eb_43ee_91af_e8a4eb48c9f8.slice/crio-963740cc0469532888cef66e8f17c0d39611d243b676de9a5373fdb7fceeb4d9 WatchSource:0}: Error finding container 963740cc0469532888cef66e8f17c0d39611d243b676de9a5373fdb7fceeb4d9: Status 404 returned error can't find the container with id 963740cc0469532888cef66e8f17c0d39611d243b676de9a5373fdb7fceeb4d9
Jul 16 21:19:22.693387 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:22.693331    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v" event={"ID":"b0761ff9-39eb-43ee-91af-e8a4eb48c9f8","Type":"ContainerStarted","Data":"963740cc0469532888cef66e8f17c0d39611d243b676de9a5373fdb7fceeb4d9"}
Jul 16 21:19:23.699613 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:23.699579    2576 generic.go:356] "Generic (PLEG): container finished" podID="b0761ff9-39eb-43ee-91af-e8a4eb48c9f8" containerID="2d5993f20faa7a9be9a2b5b630dea0ad60ae2552bf992bcf877d3cfb1f115ec1" exitCode=6
Jul 16 21:19:23.699982 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:23.699659    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v" event={"ID":"b0761ff9-39eb-43ee-91af-e8a4eb48c9f8","Type":"ContainerDied","Data":"2d5993f20faa7a9be9a2b5b630dea0ad60ae2552bf992bcf877d3cfb1f115ec1"}
Jul 16 21:19:23.699982 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:23.699788    2576 scope.go:117] "RemoveContainer" containerID="2d5993f20faa7a9be9a2b5b630dea0ad60ae2552bf992bcf877d3cfb1f115ec1"
Jul 16 21:19:24.705641 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:24.705614    2576 generic.go:356] "Generic (PLEG): container finished" podID="b0761ff9-39eb-43ee-91af-e8a4eb48c9f8" containerID="e89408fe92fa11f2f881fddeca35b1c3ef99bb95b35dfc7ee6cff6d51b4377a5" exitCode=6
Jul 16 21:19:24.706063 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:24.705700    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v" event={"ID":"b0761ff9-39eb-43ee-91af-e8a4eb48c9f8","Type":"ContainerDied","Data":"e89408fe92fa11f2f881fddeca35b1c3ef99bb95b35dfc7ee6cff6d51b4377a5"}
Jul 16 21:19:24.706063 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:24.705752    2576 scope.go:117] "RemoveContainer" containerID="2d5993f20faa7a9be9a2b5b630dea0ad60ae2552bf992bcf877d3cfb1f115ec1"
Jul 16 21:19:24.706063 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:24.705842    2576 scope.go:117] "RemoveContainer" containerID="e89408fe92fa11f2f881fddeca35b1c3ef99bb95b35dfc7ee6cff6d51b4377a5"
Jul 16 21:19:24.706063 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:19:24.706055    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v_odh-ai-gateway-infra(b0761ff9-39eb-43ee-91af-e8a4eb48c9f8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v" podUID="b0761ff9-39eb-43ee-91af-e8a4eb48c9f8"
Jul 16 21:19:25.712527 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:25.712503    2576 scope.go:117] "RemoveContainer" containerID="e89408fe92fa11f2f881fddeca35b1c3ef99bb95b35dfc7ee6cff6d51b4377a5"
Jul 16 21:19:25.712885 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:19:25.712666    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v_odh-ai-gateway-infra(b0761ff9-39eb-43ee-91af-e8a4eb48c9f8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v" podUID="b0761ff9-39eb-43ee-91af-e8a4eb48c9f8"
Jul 16 21:19:26.827996 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.827967    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"]
Jul 16 21:19:26.832833 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.832813    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.835421 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.835397    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-72cda6-openshift-default-dockercfg-sjpvr\""
Jul 16 21:19:26.844184 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.844160    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"]
Jul 16 21:19:26.888291 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.888269    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istiod-ca-cert\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.888429 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.888298    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-token\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.888429 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.888409    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdr2n\" (UniqueName: \"kubernetes.io/projected/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-kube-api-access-cdr2n\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.888533 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.888428    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-envoy\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.888533 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.888466    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-workload-socket\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.888615 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.888549    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-data\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.888615 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.888592    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-credential-socket\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.888700 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.888616    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-podinfo\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.888700 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.888637    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-workload-certs\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.989241 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.989208    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-podinfo\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.989241 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.989246    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-workload-certs\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.989529 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.989274    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istiod-ca-cert\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.989529 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.989443    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-token\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.989651 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.989537    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdr2n\" (UniqueName: \"kubernetes.io/projected/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-kube-api-access-cdr2n\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.989651 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.989568    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-envoy\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.989651 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.989620    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-workload-socket\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.989813 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.989658    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-data\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.989813 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.989699    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-credential-socket\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.990200 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.990038    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-workload-socket\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.990200 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.990038    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istiod-ca-cert\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.990200 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.990147    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-workload-certs\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.990200 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.990172    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-data\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.990503 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.990424    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-credential-socket\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.992119 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.992091    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-envoy\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.992248 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.992232    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-podinfo\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.998038 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.998013    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-token\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:26.998266 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:26.998247    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdr2n\" (UniqueName: \"kubernetes.io/projected/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-kube-api-access-cdr2n\") pod \"e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") " pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:27.143927 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:27.143902    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:27.280952 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:27.280924    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"]
Jul 16 21:19:27.281843 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:19:27.281814    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6bdeb6a_db9f_4a42_8c8b_cdfad8892445.slice/crio-395c1ea9eb3e77507e6e3ca41657166555c7d47421b5c12f4cb18bfb4aba05b4 WatchSource:0}: Error finding container 395c1ea9eb3e77507e6e3ca41657166555c7d47421b5c12f4cb18bfb4aba05b4: Status 404 returned error can't find the container with id 395c1ea9eb3e77507e6e3ca41657166555c7d47421b5c12f4cb18bfb4aba05b4
Jul 16 21:19:27.284275 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:27.284238    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:19:27.284381 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:27.284325    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:19:27.284436 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:27.284384    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:19:27.724235 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:27.724199    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t" event={"ID":"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445","Type":"ContainerStarted","Data":"f5809567f0adf53a16b06a6cbb939240bc817c0d7ddc781b317f4e444796e3fe"}
Jul 16 21:19:27.724395 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:27.724255    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t" event={"ID":"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445","Type":"ContainerStarted","Data":"395c1ea9eb3e77507e6e3ca41657166555c7d47421b5c12f4cb18bfb4aba05b4"}
Jul 16 21:19:27.745354 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:27.745304    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t" podStartSLOduration=1.745289957 podStartE2EDuration="1.745289957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:19:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:19:27.744534952 +0000 UTC m=+2813.710469186" watchObservedRunningTime="2026-07-16 21:19:27.745289957 +0000 UTC m=+2813.711224192"
Jul 16 21:19:28.144320 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:28.144295    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:28.149462 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:28.149439    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:28.730886 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:28.730859    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:28.731997 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:28.731979    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:28.987597 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:28.987526    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:19:36.684057 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:36.684033    2576 scope.go:117] "RemoveContainer" containerID="e89408fe92fa11f2f881fddeca35b1c3ef99bb95b35dfc7ee6cff6d51b4377a5"
Jul 16 21:19:36.722486 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:36.722460    2576 patch_prober.go:28] interesting pod/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 16 21:19:36.722592 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:36.722537    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp" podUID="795b38f7-b667-4392-aace-0952157f189e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 16 21:19:37.348128 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:19:37.348097    2576 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0761ff9_39eb_43ee_91af_e8a4eb48c9f8.slice/crio-778a86bb0b9695e72c23e313e6ef5467f171387182fda8271a12407844832325.scope\": RecentStats: unable to find data in memory cache]"
Jul 16 21:19:37.794707 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:37.794673    2576 generic.go:356] "Generic (PLEG): container finished" podID="795b38f7-b667-4392-aace-0952157f189e" containerID="45f5b78f52f905f262643aeab7407fc30324a929ca0136f9e3d74cdcd103a560" exitCode=0
Jul 16 21:19:37.795179 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:37.794750    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp" event={"ID":"795b38f7-b667-4392-aace-0952157f189e","Type":"ContainerDied","Data":"45f5b78f52f905f262643aeab7407fc30324a929ca0136f9e3d74cdcd103a560"}
Jul 16 21:19:37.795295 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:37.795276    2576 scope.go:117] "RemoveContainer" containerID="45f5b78f52f905f262643aeab7407fc30324a929ca0136f9e3d74cdcd103a560"
Jul 16 21:19:37.796345 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:37.796316    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:19:37.796533 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:37.796446    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:19:37.796533 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:37.796482    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:19:37.796895 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:37.796876    2576 generic.go:356] "Generic (PLEG): container finished" podID="b0761ff9-39eb-43ee-91af-e8a4eb48c9f8" containerID="778a86bb0b9695e72c23e313e6ef5467f171387182fda8271a12407844832325" exitCode=6
Jul 16 21:19:37.796958 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:37.796933    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v" event={"ID":"b0761ff9-39eb-43ee-91af-e8a4eb48c9f8","Type":"ContainerDied","Data":"778a86bb0b9695e72c23e313e6ef5467f171387182fda8271a12407844832325"}
Jul 16 21:19:37.797007 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:37.796973    2576 scope.go:117] "RemoveContainer" containerID="e89408fe92fa11f2f881fddeca35b1c3ef99bb95b35dfc7ee6cff6d51b4377a5"
Jul 16 21:19:37.797249 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:37.797230    2576 scope.go:117] "RemoveContainer" containerID="778a86bb0b9695e72c23e313e6ef5467f171387182fda8271a12407844832325"
Jul 16 21:19:37.797625 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:19:37.797603    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v_odh-ai-gateway-infra(b0761ff9-39eb-43ee-91af-e8a4eb48c9f8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v" podUID="b0761ff9-39eb-43ee-91af-e8a4eb48c9f8"
Jul 16 21:19:38.805376 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:38.805328    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp" event={"ID":"795b38f7-b667-4392-aace-0952157f189e","Type":"ContainerStarted","Data":"f7d1025c15317472f803cf275fd7632968d02a866b94e464a0d1a50df9aadd07"}
Jul 16 21:19:38.847660 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:38.847618    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v"]
Jul 16 21:19:39.074932 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:39.074560    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v"
Jul 16 21:19:39.102867 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:39.102836    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0761ff9-39eb-43ee-91af-e8a4eb48c9f8-maas-api-service-ca\") pod \"b0761ff9-39eb-43ee-91af-e8a4eb48c9f8\" (UID: \"b0761ff9-39eb-43ee-91af-e8a4eb48c9f8\") "
Jul 16 21:19:39.103561 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:39.103535    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0761ff9-39eb-43ee-91af-e8a4eb48c9f8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b0761ff9-39eb-43ee-91af-e8a4eb48c9f8" (UID: "b0761ff9-39eb-43ee-91af-e8a4eb48c9f8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:19:39.204257 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:39.204229    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0761ff9-39eb-43ee-91af-e8a4eb48c9f8-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:19:39.251760 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:39.251717    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:39.251954 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:39.251846    2576 patch_prober.go:28] interesting pod/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Jul 16 21:19:39.251954 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:39.251900    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp" podUID="795b38f7-b667-4392-aace-0952157f189e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Jul 16 21:19:39.812766 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:39.812731    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v" event={"ID":"b0761ff9-39eb-43ee-91af-e8a4eb48c9f8","Type":"ContainerDied","Data":"963740cc0469532888cef66e8f17c0d39611d243b676de9a5373fdb7fceeb4d9"}
Jul 16 21:19:39.813179 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:39.812795    2576 scope.go:117] "RemoveContainer" containerID="778a86bb0b9695e72c23e313e6ef5467f171387182fda8271a12407844832325"
Jul 16 21:19:39.813248 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:39.813229    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v"
Jul 16 21:19:39.835693 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:39.835663    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v"]
Jul 16 21:19:39.845902 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:39.845875    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-e65739ab-cqs6v"]
Jul 16 21:19:40.252335 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:40.252292    2576 patch_prober.go:28] interesting pod/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Jul 16 21:19:40.252540 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:40.252374    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp" podUID="795b38f7-b667-4392-aace-0952157f189e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Jul 16 21:19:40.690050 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:40.690019    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0761ff9-39eb-43ee-91af-e8a4eb48c9f8" path="/var/lib/kubelet/pods/b0761ff9-39eb-43ee-91af-e8a4eb48c9f8/volumes"
Jul 16 21:19:41.251919 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.251874    2576 patch_prober.go:28] interesting pod/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Jul 16 21:19:41.252443 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.251939    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp" podUID="795b38f7-b667-4392-aace-0952157f189e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Jul 16 21:19:41.471453 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.471412    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"]
Jul 16 21:19:41.472031 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.472007    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0761ff9-39eb-43ee-91af-e8a4eb48c9f8" containerName="revoke-keys"
Jul 16 21:19:41.472031 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.472033    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0761ff9-39eb-43ee-91af-e8a4eb48c9f8" containerName="revoke-keys"
Jul 16 21:19:41.472206 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.472065    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0761ff9-39eb-43ee-91af-e8a4eb48c9f8" containerName="revoke-keys"
Jul 16 21:19:41.472206 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.472073    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0761ff9-39eb-43ee-91af-e8a4eb48c9f8" containerName="revoke-keys"
Jul 16 21:19:41.472206 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.472176    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0761ff9-39eb-43ee-91af-e8a4eb48c9f8" containerName="revoke-keys"
Jul 16 21:19:41.472206 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.472191    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0761ff9-39eb-43ee-91af-e8a4eb48c9f8" containerName="revoke-keys"
Jul 16 21:19:41.477317 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.477288    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.483323 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.483297    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c69b60\"/\"kube-root-ca.crt\""
Jul 16 21:19:41.483610 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.483589    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c69b60\"/\"test-model-c69b60-kserve-self-signed-certs\""
Jul 16 21:19:41.483931 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.483911    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c69b60\"/\"default-dockercfg-jph28\""
Jul 16 21:19:41.484165 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.484145    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c69b60\"/\"openshift-service-ca.crt\""
Jul 16 21:19:41.486031 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.486005    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"]
Jul 16 21:19:41.524945 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.524890    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-tmp-dir\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.524945 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.524930    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-model-cache\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.525092 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.524961    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-home\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.525092 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.524985    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-dshm\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.525092 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.525009    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c88a98a-f57d-45e7-929b-586f76f3e2b6-tls-certs\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.525092 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.525051    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmg6t\" (UniqueName: \"kubernetes.io/projected/2c88a98a-f57d-45e7-929b-586f76f3e2b6-kube-api-access-bmg6t\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.626100 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.626067    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-model-cache\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.626295 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.626113    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-home\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.626295 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.626136    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-dshm\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.626295 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.626159    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c88a98a-f57d-45e7-929b-586f76f3e2b6-tls-certs\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.626295 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.626193    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmg6t\" (UniqueName: \"kubernetes.io/projected/2c88a98a-f57d-45e7-929b-586f76f3e2b6-kube-api-access-bmg6t\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.626295 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.626271    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-tmp-dir\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.626606 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.626561    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-home\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.626661 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.626607    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-model-cache\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.627056 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.627037    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-tmp-dir\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.633060 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.633010    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-dshm\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.633740 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.633716    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c88a98a-f57d-45e7-929b-586f76f3e2b6-tls-certs\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.637987 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.637946    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmg6t\" (UniqueName: \"kubernetes.io/projected/2c88a98a-f57d-45e7-929b-586f76f3e2b6-kube-api-access-bmg6t\") pod \"test-model-c69b60-kserve-77964588b9-7c8sc\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") " pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:41.791541 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:41.791418    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:42.003454 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:42.003417    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"]
Jul 16 21:19:42.007243 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:19:42.007211    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c88a98a_f57d_45e7_929b_586f76f3e2b6.slice/crio-85c1924419d0d1f90bb2e5bedf85a3f216fffd51920e55bc62691edf2b50aa7e WatchSource:0}: Error finding container 85c1924419d0d1f90bb2e5bedf85a3f216fffd51920e55bc62691edf2b50aa7e: Status 404 returned error can't find the container with id 85c1924419d0d1f90bb2e5bedf85a3f216fffd51920e55bc62691edf2b50aa7e
Jul 16 21:19:42.251836 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:42.251797    2576 patch_prober.go:28] interesting pod/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Jul 16 21:19:42.251998 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:42.251858    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp" podUID="795b38f7-b667-4392-aace-0952157f189e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Jul 16 21:19:42.837736 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:42.837694    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc" event={"ID":"2c88a98a-f57d-45e7-929b-586f76f3e2b6","Type":"ContainerStarted","Data":"f437c7879f62e3cb7f365a2dd0fb9034f5fc34d44f700a7d9dac730824a0b7d5"}
Jul 16 21:19:42.837939 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:42.837748    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc" event={"ID":"2c88a98a-f57d-45e7-929b-586f76f3e2b6","Type":"ContainerStarted","Data":"85c1924419d0d1f90bb2e5bedf85a3f216fffd51920e55bc62691edf2b50aa7e"}
Jul 16 21:19:42.857677 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:42.857621    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc" podStartSLOduration=1.857605719 podStartE2EDuration="1.857605719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:19:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:19:42.855281614 +0000 UTC m=+2828.821215849" watchObservedRunningTime="2026-07-16 21:19:42.857605719 +0000 UTC m=+2828.823539951"
Jul 16 21:19:43.255756 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:43.255721    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:43.256183 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:43.256046    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:43.256846 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:43.256825    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:19:44.588746 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:44.588712    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:19:44.711900 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:44.711480    2576 patch_prober.go:28] interesting pod/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": read tcp 10.132.0.2:47830->10.132.0.68:15021: read: connection reset by peer" start-of-body=
Jul 16 21:19:44.711900 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:44.711544    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t" podUID="f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": read tcp 10.132.0.2:47830->10.132.0.68:15021: read: connection reset by peer"
Jul 16 21:19:44.849256 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:44.849200    2576 generic.go:356] "Generic (PLEG): container finished" podID="f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" containerID="f5809567f0adf53a16b06a6cbb939240bc817c0d7ddc781b317f4e444796e3fe" exitCode=0
Jul 16 21:19:44.849387 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:44.849271    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t" event={"ID":"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445","Type":"ContainerDied","Data":"f5809567f0adf53a16b06a6cbb939240bc817c0d7ddc781b317f4e444796e3fe"}
Jul 16 21:19:44.849731 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:44.849714    2576 scope.go:117] "RemoveContainer" containerID="f5809567f0adf53a16b06a6cbb939240bc817c0d7ddc781b317f4e444796e3fe"
Jul 16 21:19:44.850692 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:44.850667    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:19:44.850743 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:44.850730    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:19:44.850791 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:44.850779    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:19:45.860063 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:45.860027    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t" event={"ID":"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445","Type":"ContainerStarted","Data":"7cdedb18818f572c8fb331537d7009a53c967ec31bef5a0cf202f899ce4f3b04"}
Jul 16 21:19:46.144911 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:46.144874    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:46.145116 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:46.144997    2576 patch_prober.go:28] interesting pod/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Jul 16 21:19:46.145116 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:46.145043    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t" podUID="f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Jul 16 21:19:47.145277 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:47.145230    2576 patch_prober.go:28] interesting pod/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Jul 16 21:19:47.145820 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:47.145301    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t" podUID="f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Jul 16 21:19:48.145153 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:48.145112    2576 patch_prober.go:28] interesting pod/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Jul 16 21:19:48.145376 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:48.145178    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t" podUID="f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Jul 16 21:19:49.144740 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:49.144703    2576 patch_prober.go:28] interesting pod/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Jul 16 21:19:49.145053 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:49.145022    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t" podUID="f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Jul 16 21:19:50.147549 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:50.147521    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:50.148039 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:50.147889    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:50.148551 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:50.148536    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:19:51.792198 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:51.792166    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:51.792619 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:51.792227    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:51.805228 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:51.805202    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:51.901697 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:51.901672    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:19:59.601722 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:19:59.601690    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:20:08.083676 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.083634    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"]
Jul 16 21:20:08.084240 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.084009    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0761ff9-39eb-43ee-91af-e8a4eb48c9f8" containerName="revoke-keys"
Jul 16 21:20:08.084240 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.084020    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0761ff9-39eb-43ee-91af-e8a4eb48c9f8" containerName="revoke-keys"
Jul 16 21:20:08.084240 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.084088    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0761ff9-39eb-43ee-91af-e8a4eb48c9f8" containerName="revoke-keys"
Jul 16 21:20:08.088634 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.088613    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.091099 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.091077    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-72cda6\"/\"kube-root-ca.crt\""
Jul 16 21:20:08.091219 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.091177    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-72cda6\"/\"openshift-service-ca.crt\""
Jul 16 21:20:08.091275 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.091237    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-72cda6\"/\"default-dockercfg-d7vvs\""
Jul 16 21:20:08.092022 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.092006    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-72cda6\"/\"test-model-72cda6-kserve-self-signed-certs\""
Jul 16 21:20:08.098794 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.098773    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"]
Jul 16 21:20:08.270083 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.270047    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-home\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.270282 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.270093    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f6b7\" (UniqueName: \"kubernetes.io/projected/784b6d1a-f100-4b8b-a8d1-cb715703098a-kube-api-access-6f6b7\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.270282 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.270179    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-dshm\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.270282 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.270228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/784b6d1a-f100-4b8b-a8d1-cb715703098a-tls-certs\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.270282 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.270261    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-tmp-dir\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.270519 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.270330    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-model-cache\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.371175 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.371112    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-model-cache\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.371175 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.371160    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-home\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.371323 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.371198    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f6b7\" (UniqueName: \"kubernetes.io/projected/784b6d1a-f100-4b8b-a8d1-cb715703098a-kube-api-access-6f6b7\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.371323 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.371235    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-dshm\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.371323 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.371265    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/784b6d1a-f100-4b8b-a8d1-cb715703098a-tls-certs\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.371323 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.371293    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-tmp-dir\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.371668 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.371640    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-model-cache\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.371774 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.371645    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-home\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.371774 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.371702    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-tmp-dir\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.373537 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.373515    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-dshm\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.373891 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.373873    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/784b6d1a-f100-4b8b-a8d1-cb715703098a-tls-certs\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.379599 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.379578    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f6b7\" (UniqueName: \"kubernetes.io/projected/784b6d1a-f100-4b8b-a8d1-cb715703098a-kube-api-access-6f6b7\") pod \"test-model-72cda6-kserve-df775bf87-rqgrc\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") " pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.398591 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.398570    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:08.532993 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.532962    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"]
Jul 16 21:20:08.534598 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:20:08.534568    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod784b6d1a_f100_4b8b_a8d1_cb715703098a.slice/crio-62048bcc86ce401e27a73ac9bdc1462905b3ca3d297a0ef5a3b464eb3e73d77b WatchSource:0}: Error finding container 62048bcc86ce401e27a73ac9bdc1462905b3ca3d297a0ef5a3b464eb3e73d77b: Status 404 returned error can't find the container with id 62048bcc86ce401e27a73ac9bdc1462905b3ca3d297a0ef5a3b464eb3e73d77b
Jul 16 21:20:08.979174 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.979138    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc" event={"ID":"784b6d1a-f100-4b8b-a8d1-cb715703098a","Type":"ContainerStarted","Data":"3c2152ea3792f01f655820ca7a8e48db24c3315e558cbffcd40463e6fc0b1d6a"}
Jul 16 21:20:08.979329 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.979181    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc" event={"ID":"784b6d1a-f100-4b8b-a8d1-cb715703098a","Type":"ContainerStarted","Data":"62048bcc86ce401e27a73ac9bdc1462905b3ca3d297a0ef5a3b464eb3e73d77b"}
Jul 16 21:20:08.995521 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:08.995475    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc" podStartSLOduration=0.995460876 podStartE2EDuration="995.460876ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:20:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:20:08.994388834 +0000 UTC m=+2854.960323065" watchObservedRunningTime="2026-07-16 21:20:08.995460876 +0000 UTC m=+2854.961395110"
Jul 16 21:20:15.699479 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:15.699445    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:20:18.399227 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:18.399193    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:18.399662 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:18.399249    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:18.411621 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:18.411595    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:19.040168 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:19.040136    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:27.757413 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:27.757314    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"]
Jul 16 21:20:28.663482 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:28.663453    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"]
Jul 16 21:20:33.101342 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:33.101311    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:20:42.777793 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:42.777724    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc" podUID="2c88a98a-f57d-45e7-929b-586f76f3e2b6" containerName="main" containerID="cri-o://f437c7879f62e3cb7f365a2dd0fb9034f5fc34d44f700a7d9dac730824a0b7d5" gracePeriod=45
Jul 16 21:20:43.026625 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.026602    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:20:43.155619 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.155591    2576 generic.go:356] "Generic (PLEG): container finished" podID="2c88a98a-f57d-45e7-929b-586f76f3e2b6" containerID="f437c7879f62e3cb7f365a2dd0fb9034f5fc34d44f700a7d9dac730824a0b7d5" exitCode=0
Jul 16 21:20:43.155773 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.155630    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"
Jul 16 21:20:43.155773 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.155681    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc" event={"ID":"2c88a98a-f57d-45e7-929b-586f76f3e2b6","Type":"ContainerDied","Data":"f437c7879f62e3cb7f365a2dd0fb9034f5fc34d44f700a7d9dac730824a0b7d5"}
Jul 16 21:20:43.155773 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.155723    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc" event={"ID":"2c88a98a-f57d-45e7-929b-586f76f3e2b6","Type":"ContainerDied","Data":"85c1924419d0d1f90bb2e5bedf85a3f216fffd51920e55bc62691edf2b50aa7e"}
Jul 16 21:20:43.155773 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.155743    2576 scope.go:117] "RemoveContainer" containerID="f437c7879f62e3cb7f365a2dd0fb9034f5fc34d44f700a7d9dac730824a0b7d5"
Jul 16 21:20:43.157924 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.157903    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-home\") pod \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") "
Jul 16 21:20:43.158052 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.157950    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmg6t\" (UniqueName: \"kubernetes.io/projected/2c88a98a-f57d-45e7-929b-586f76f3e2b6-kube-api-access-bmg6t\") pod \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") "
Jul 16 21:20:43.158052 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.157987    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-tmp-dir\") pod \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") "
Jul 16 21:20:43.158052 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.158028    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-dshm\") pod \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") "
Jul 16 21:20:43.158214 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.158065    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c88a98a-f57d-45e7-929b-586f76f3e2b6-tls-certs\") pod \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") "
Jul 16 21:20:43.158214 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.158107    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-model-cache\") pod \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\" (UID: \"2c88a98a-f57d-45e7-929b-586f76f3e2b6\") "
Jul 16 21:20:43.158214 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.158152    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-home" (OuterVolumeSpecName: "home") pod "2c88a98a-f57d-45e7-929b-586f76f3e2b6" (UID: "2c88a98a-f57d-45e7-929b-586f76f3e2b6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:43.158404 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.158252    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2c88a98a-f57d-45e7-929b-586f76f3e2b6" (UID: "2c88a98a-f57d-45e7-929b-586f76f3e2b6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:43.158486 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.158465    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-model-cache" (OuterVolumeSpecName: "model-cache") pod "2c88a98a-f57d-45e7-929b-586f76f3e2b6" (UID: "2c88a98a-f57d-45e7-929b-586f76f3e2b6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:43.158553 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.158484    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-tmp-dir\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:20:43.158553 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.158500    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-home\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:20:43.162402 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.161109    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-dshm" (OuterVolumeSpecName: "dshm") pod "2c88a98a-f57d-45e7-929b-586f76f3e2b6" (UID: "2c88a98a-f57d-45e7-929b-586f76f3e2b6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:43.167702 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.167620    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c88a98a-f57d-45e7-929b-586f76f3e2b6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2c88a98a-f57d-45e7-929b-586f76f3e2b6" (UID: "2c88a98a-f57d-45e7-929b-586f76f3e2b6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:20:43.167702 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.167665    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c88a98a-f57d-45e7-929b-586f76f3e2b6-kube-api-access-bmg6t" (OuterVolumeSpecName: "kube-api-access-bmg6t") pod "2c88a98a-f57d-45e7-929b-586f76f3e2b6" (UID: "2c88a98a-f57d-45e7-929b-586f76f3e2b6"). InnerVolumeSpecName "kube-api-access-bmg6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:20:43.171688 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.171670    2576 scope.go:117] "RemoveContainer" containerID="f437c7879f62e3cb7f365a2dd0fb9034f5fc34d44f700a7d9dac730824a0b7d5"
Jul 16 21:20:43.171948 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:20:43.171929    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f437c7879f62e3cb7f365a2dd0fb9034f5fc34d44f700a7d9dac730824a0b7d5\": container with ID starting with f437c7879f62e3cb7f365a2dd0fb9034f5fc34d44f700a7d9dac730824a0b7d5 not found: ID does not exist" containerID="f437c7879f62e3cb7f365a2dd0fb9034f5fc34d44f700a7d9dac730824a0b7d5"
Jul 16 21:20:43.172023 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.171959    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f437c7879f62e3cb7f365a2dd0fb9034f5fc34d44f700a7d9dac730824a0b7d5"} err="failed to get container status \"f437c7879f62e3cb7f365a2dd0fb9034f5fc34d44f700a7d9dac730824a0b7d5\": rpc error: code = NotFound desc = could not find container \"f437c7879f62e3cb7f365a2dd0fb9034f5fc34d44f700a7d9dac730824a0b7d5\": container with ID starting with f437c7879f62e3cb7f365a2dd0fb9034f5fc34d44f700a7d9dac730824a0b7d5 not found: ID does not exist"
Jul 16 21:20:43.259191 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.259166    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c88a98a-f57d-45e7-929b-586f76f3e2b6-tls-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:20:43.259191 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.259189    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-model-cache\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:20:43.259314 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.259203    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmg6t\" (UniqueName: \"kubernetes.io/projected/2c88a98a-f57d-45e7-929b-586f76f3e2b6-kube-api-access-bmg6t\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:20:43.259314 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.259216    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2c88a98a-f57d-45e7-929b-586f76f3e2b6-dshm\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:20:43.476835 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.476808    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"]
Jul 16 21:20:43.482531 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.482509    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-c69b60/test-model-c69b60-kserve-77964588b9-7c8sc"]
Jul 16 21:20:43.684184 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.684125    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc" podUID="784b6d1a-f100-4b8b-a8d1-cb715703098a" containerName="main" containerID="cri-o://3c2152ea3792f01f655820ca7a8e48db24c3315e558cbffcd40463e6fc0b1d6a" gracePeriod=45
Jul 16 21:20:43.943511 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.943484    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:43.995734 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:43.995704    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:20:44.066579 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.066511    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-model-cache\") pod \"784b6d1a-f100-4b8b-a8d1-cb715703098a\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") "
Jul 16 21:20:44.066739 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.066585    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/784b6d1a-f100-4b8b-a8d1-cb715703098a-tls-certs\") pod \"784b6d1a-f100-4b8b-a8d1-cb715703098a\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") "
Jul 16 21:20:44.066739 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.066607    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6f6b7\" (UniqueName: \"kubernetes.io/projected/784b6d1a-f100-4b8b-a8d1-cb715703098a-kube-api-access-6f6b7\") pod \"784b6d1a-f100-4b8b-a8d1-cb715703098a\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") "
Jul 16 21:20:44.066739 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.066645    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-home\") pod \"784b6d1a-f100-4b8b-a8d1-cb715703098a\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") "
Jul 16 21:20:44.066739 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.066663    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-dshm\") pod \"784b6d1a-f100-4b8b-a8d1-cb715703098a\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") "
Jul 16 21:20:44.066739 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.066726    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-tmp-dir\") pod \"784b6d1a-f100-4b8b-a8d1-cb715703098a\" (UID: \"784b6d1a-f100-4b8b-a8d1-cb715703098a\") "
Jul 16 21:20:44.066995 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.066802    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-model-cache" (OuterVolumeSpecName: "model-cache") pod "784b6d1a-f100-4b8b-a8d1-cb715703098a" (UID: "784b6d1a-f100-4b8b-a8d1-cb715703098a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:44.066995 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.066911    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-home" (OuterVolumeSpecName: "home") pod "784b6d1a-f100-4b8b-a8d1-cb715703098a" (UID: "784b6d1a-f100-4b8b-a8d1-cb715703098a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:44.067098 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.067042    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "784b6d1a-f100-4b8b-a8d1-cb715703098a" (UID: "784b6d1a-f100-4b8b-a8d1-cb715703098a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:44.067210 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.067191    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-model-cache\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:20:44.067275 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.067218    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-home\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:20:44.068972 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.068946    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/784b6d1a-f100-4b8b-a8d1-cb715703098a-kube-api-access-6f6b7" (OuterVolumeSpecName: "kube-api-access-6f6b7") pod "784b6d1a-f100-4b8b-a8d1-cb715703098a" (UID: "784b6d1a-f100-4b8b-a8d1-cb715703098a"). InnerVolumeSpecName "kube-api-access-6f6b7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:20:44.069213 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.069188    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/784b6d1a-f100-4b8b-a8d1-cb715703098a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "784b6d1a-f100-4b8b-a8d1-cb715703098a" (UID: "784b6d1a-f100-4b8b-a8d1-cb715703098a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:20:44.069213 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.069195    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-dshm" (OuterVolumeSpecName: "dshm") pod "784b6d1a-f100-4b8b-a8d1-cb715703098a" (UID: "784b6d1a-f100-4b8b-a8d1-cb715703098a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:44.162656 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.162625    2576 generic.go:356] "Generic (PLEG): container finished" podID="784b6d1a-f100-4b8b-a8d1-cb715703098a" containerID="3c2152ea3792f01f655820ca7a8e48db24c3315e558cbffcd40463e6fc0b1d6a" exitCode=0
Jul 16 21:20:44.162789 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.162672    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"
Jul 16 21:20:44.162789 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.162681    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc" event={"ID":"784b6d1a-f100-4b8b-a8d1-cb715703098a","Type":"ContainerDied","Data":"3c2152ea3792f01f655820ca7a8e48db24c3315e558cbffcd40463e6fc0b1d6a"}
Jul 16 21:20:44.162789 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.162710    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc" event={"ID":"784b6d1a-f100-4b8b-a8d1-cb715703098a","Type":"ContainerDied","Data":"62048bcc86ce401e27a73ac9bdc1462905b3ca3d297a0ef5a3b464eb3e73d77b"}
Jul 16 21:20:44.162789 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.162727    2576 scope.go:117] "RemoveContainer" containerID="3c2152ea3792f01f655820ca7a8e48db24c3315e558cbffcd40463e6fc0b1d6a"
Jul 16 21:20:44.167841 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.167815    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-dshm\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:20:44.167841 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.167836    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/784b6d1a-f100-4b8b-a8d1-cb715703098a-tmp-dir\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:20:44.167983 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.167850    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/784b6d1a-f100-4b8b-a8d1-cb715703098a-tls-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:20:44.167983 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.167862    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6f6b7\" (UniqueName: \"kubernetes.io/projected/784b6d1a-f100-4b8b-a8d1-cb715703098a-kube-api-access-6f6b7\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:20:44.171403 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.171383    2576 scope.go:117] "RemoveContainer" containerID="3c2152ea3792f01f655820ca7a8e48db24c3315e558cbffcd40463e6fc0b1d6a"
Jul 16 21:20:44.171635 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:20:44.171616    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c2152ea3792f01f655820ca7a8e48db24c3315e558cbffcd40463e6fc0b1d6a\": container with ID starting with 3c2152ea3792f01f655820ca7a8e48db24c3315e558cbffcd40463e6fc0b1d6a not found: ID does not exist" containerID="3c2152ea3792f01f655820ca7a8e48db24c3315e558cbffcd40463e6fc0b1d6a"
Jul 16 21:20:44.171677 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.171644    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c2152ea3792f01f655820ca7a8e48db24c3315e558cbffcd40463e6fc0b1d6a"} err="failed to get container status \"3c2152ea3792f01f655820ca7a8e48db24c3315e558cbffcd40463e6fc0b1d6a\": rpc error: code = NotFound desc = could not find container \"3c2152ea3792f01f655820ca7a8e48db24c3315e558cbffcd40463e6fc0b1d6a\": container with ID starting with 3c2152ea3792f01f655820ca7a8e48db24c3315e558cbffcd40463e6fc0b1d6a not found: ID does not exist"
Jul 16 21:20:44.182675 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.182655    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"]
Jul 16 21:20:44.187439 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.187421    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-72cda6/test-model-72cda6-kserve-df775bf87-rqgrc"]
Jul 16 21:20:44.690501 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.690472    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c88a98a-f57d-45e7-929b-586f76f3e2b6" path="/var/lib/kubelet/pods/2c88a98a-f57d-45e7-929b-586f76f3e2b6/volumes"
Jul 16 21:20:44.690872 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:44.690859    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="784b6d1a-f100-4b8b-a8d1-cb715703098a" path="/var/lib/kubelet/pods/784b6d1a-f100-4b8b-a8d1-cb715703098a/volumes"
Jul 16 21:20:54.558619 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:54.558580    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"]
Jul 16 21:20:54.559053 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:54.558818    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp" podUID="795b38f7-b667-4392-aace-0952157f189e" containerName="istio-proxy" containerID="cri-o://f7d1025c15317472f803cf275fd7632968d02a866b94e464a0d1a50df9aadd07" gracePeriod=30
Jul 16 21:20:58.258131 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:58.258096    2576 patch_prober.go:28] interesting pod/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 21:20:58.258628 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:58.258173    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp" podUID="795b38f7-b667-4392-aace-0952157f189e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 21:20:59.813949 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.813916    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:20:59.900577 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.900556    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6sfj\" (UniqueName: \"kubernetes.io/projected/795b38f7-b667-4392-aace-0952157f189e-kube-api-access-c6sfj\") pod \"795b38f7-b667-4392-aace-0952157f189e\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") "
Jul 16 21:20:59.900716 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.900588    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-workload-socket\") pod \"795b38f7-b667-4392-aace-0952157f189e\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") "
Jul 16 21:20:59.900716 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.900619    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-workload-certs\") pod \"795b38f7-b667-4392-aace-0952157f189e\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") "
Jul 16 21:20:59.900716 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.900642    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-istio-envoy\") pod \"795b38f7-b667-4392-aace-0952157f189e\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") "
Jul 16 21:20:59.900716 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.900660    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/795b38f7-b667-4392-aace-0952157f189e-istio-token\") pod \"795b38f7-b667-4392-aace-0952157f189e\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") "
Jul 16 21:20:59.900716 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.900691    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/795b38f7-b667-4392-aace-0952157f189e-istiod-ca-cert\") pod \"795b38f7-b667-4392-aace-0952157f189e\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") "
Jul 16 21:20:59.900982 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.900722    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-istio-data\") pod \"795b38f7-b667-4392-aace-0952157f189e\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") "
Jul 16 21:20:59.900982 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.900808    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/795b38f7-b667-4392-aace-0952157f189e-istio-podinfo\") pod \"795b38f7-b667-4392-aace-0952157f189e\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") "
Jul 16 21:20:59.900982 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.900867    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-credential-socket\") pod \"795b38f7-b667-4392-aace-0952157f189e\" (UID: \"795b38f7-b667-4392-aace-0952157f189e\") "
Jul 16 21:20:59.900982 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.900896    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "795b38f7-b667-4392-aace-0952157f189e" (UID: "795b38f7-b667-4392-aace-0952157f189e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:59.900982 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.900936    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "795b38f7-b667-4392-aace-0952157f189e" (UID: "795b38f7-b667-4392-aace-0952157f189e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:59.901199 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.901065    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/795b38f7-b667-4392-aace-0952157f189e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "795b38f7-b667-4392-aace-0952157f189e" (UID: "795b38f7-b667-4392-aace-0952157f189e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:20:59.901244 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.901225    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-workload-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:20:59.901309 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.901243    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-workload-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:20:59.901309 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.901256    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/795b38f7-b667-4392-aace-0952157f189e-istiod-ca-cert\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:20:59.901309 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.901225    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "795b38f7-b667-4392-aace-0952157f189e" (UID: "795b38f7-b667-4392-aace-0952157f189e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:59.901723 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.901694    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-istio-data" (OuterVolumeSpecName: "istio-data") pod "795b38f7-b667-4392-aace-0952157f189e" (UID: "795b38f7-b667-4392-aace-0952157f189e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:20:59.902771 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.902747    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/795b38f7-b667-4392-aace-0952157f189e-kube-api-access-c6sfj" (OuterVolumeSpecName: "kube-api-access-c6sfj") pod "795b38f7-b667-4392-aace-0952157f189e" (UID: "795b38f7-b667-4392-aace-0952157f189e"). InnerVolumeSpecName "kube-api-access-c6sfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:20:59.902893 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.902795    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/795b38f7-b667-4392-aace-0952157f189e-istio-token" (OuterVolumeSpecName: "istio-token") pod "795b38f7-b667-4392-aace-0952157f189e" (UID: "795b38f7-b667-4392-aace-0952157f189e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:20:59.903325 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.903301    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/795b38f7-b667-4392-aace-0952157f189e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "795b38f7-b667-4392-aace-0952157f189e" (UID: "795b38f7-b667-4392-aace-0952157f189e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 21:20:59.903448 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:20:59.903382    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "795b38f7-b667-4392-aace-0952157f189e" (UID: "795b38f7-b667-4392-aace-0952157f189e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:21:00.001924 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.001898    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6sfj\" (UniqueName: \"kubernetes.io/projected/795b38f7-b667-4392-aace-0952157f189e-kube-api-access-c6sfj\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:21:00.001924 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.001922    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-istio-envoy\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:21:00.002054 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.001932    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/795b38f7-b667-4392-aace-0952157f189e-istio-token\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:21:00.002054 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.001939    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-istio-data\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:21:00.002054 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.001947    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/795b38f7-b667-4392-aace-0952157f189e-istio-podinfo\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:21:00.002054 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.001954    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/795b38f7-b667-4392-aace-0952157f189e-credential-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:21:00.244832 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.244772    2576 generic.go:356] "Generic (PLEG): container finished" podID="795b38f7-b667-4392-aace-0952157f189e" containerID="f7d1025c15317472f803cf275fd7632968d02a866b94e464a0d1a50df9aadd07" exitCode=0
Jul 16 21:21:00.244832 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.244828    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"
Jul 16 21:21:00.244967 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.244856    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp" event={"ID":"795b38f7-b667-4392-aace-0952157f189e","Type":"ContainerDied","Data":"f7d1025c15317472f803cf275fd7632968d02a866b94e464a0d1a50df9aadd07"}
Jul 16 21:21:00.244967 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.244910    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp" event={"ID":"795b38f7-b667-4392-aace-0952157f189e","Type":"ContainerDied","Data":"63835c78319c486cc3d5c896cbe3f175241f4f487c29e097e1166796e8a22938"}
Jul 16 21:21:00.244967 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.244934    2576 scope.go:117] "RemoveContainer" containerID="f7d1025c15317472f803cf275fd7632968d02a866b94e464a0d1a50df9aadd07"
Jul 16 21:21:00.254636 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.254620    2576 scope.go:117] "RemoveContainer" containerID="45f5b78f52f905f262643aeab7407fc30324a929ca0136f9e3d74cdcd103a560"
Jul 16 21:21:00.262467 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.262451    2576 scope.go:117] "RemoveContainer" containerID="f7d1025c15317472f803cf275fd7632968d02a866b94e464a0d1a50df9aadd07"
Jul 16 21:21:00.262705 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:21:00.262688    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7d1025c15317472f803cf275fd7632968d02a866b94e464a0d1a50df9aadd07\": container with ID starting with f7d1025c15317472f803cf275fd7632968d02a866b94e464a0d1a50df9aadd07 not found: ID does not exist" containerID="f7d1025c15317472f803cf275fd7632968d02a866b94e464a0d1a50df9aadd07"
Jul 16 21:21:00.262757 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.262712    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7d1025c15317472f803cf275fd7632968d02a866b94e464a0d1a50df9aadd07"} err="failed to get container status \"f7d1025c15317472f803cf275fd7632968d02a866b94e464a0d1a50df9aadd07\": rpc error: code = NotFound desc = could not find container \"f7d1025c15317472f803cf275fd7632968d02a866b94e464a0d1a50df9aadd07\": container with ID starting with f7d1025c15317472f803cf275fd7632968d02a866b94e464a0d1a50df9aadd07 not found: ID does not exist"
Jul 16 21:21:00.262757 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.262728    2576 scope.go:117] "RemoveContainer" containerID="45f5b78f52f905f262643aeab7407fc30324a929ca0136f9e3d74cdcd103a560"
Jul 16 21:21:00.262931 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:21:00.262917    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45f5b78f52f905f262643aeab7407fc30324a929ca0136f9e3d74cdcd103a560\": container with ID starting with 45f5b78f52f905f262643aeab7407fc30324a929ca0136f9e3d74cdcd103a560 not found: ID does not exist" containerID="45f5b78f52f905f262643aeab7407fc30324a929ca0136f9e3d74cdcd103a560"
Jul 16 21:21:00.262976 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.262934    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45f5b78f52f905f262643aeab7407fc30324a929ca0136f9e3d74cdcd103a560"} err="failed to get container status \"45f5b78f52f905f262643aeab7407fc30324a929ca0136f9e3d74cdcd103a560\": rpc error: code = NotFound desc = could not find container \"45f5b78f52f905f262643aeab7407fc30324a929ca0136f9e3d74cdcd103a560\": container with ID starting with 45f5b78f52f905f262643aeab7407fc30324a929ca0136f9e3d74cdcd103a560 not found: ID does not exist"
Jul 16 21:21:00.268849 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.268827    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"]
Jul 16 21:21:00.273453 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.273426    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c69b60-openshift-default-f7d8bcdd9-j5dcp"]
Jul 16 21:21:00.690489 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:00.690461    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="795b38f7-b667-4392-aace-0952157f189e" path="/var/lib/kubelet/pods/795b38f7-b667-4392-aace-0952157f189e/volumes"
Jul 16 21:21:08.394225 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:08.394193    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:21:20.717474 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:20.717432    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"]
Jul 16 21:21:20.717903 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:20.717666    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t" podUID="f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" containerName="istio-proxy" containerID="cri-o://7cdedb18818f572c8fb331537d7009a53c967ec31bef5a0cf202f899ce4f3b04" gracePeriod=30
Jul 16 21:21:21.596490 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.596461    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"]
Jul 16 21:21:21.596893 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.596874    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="795b38f7-b667-4392-aace-0952157f189e" containerName="istio-proxy"
Jul 16 21:21:21.596988 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.596896    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="795b38f7-b667-4392-aace-0952157f189e" containerName="istio-proxy"
Jul 16 21:21:21.596988 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.596907    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="795b38f7-b667-4392-aace-0952157f189e" containerName="istio-proxy"
Jul 16 21:21:21.596988 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.596915    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="795b38f7-b667-4392-aace-0952157f189e" containerName="istio-proxy"
Jul 16 21:21:21.596988 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.596929    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c88a98a-f57d-45e7-929b-586f76f3e2b6" containerName="main"
Jul 16 21:21:21.596988 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.596937    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c88a98a-f57d-45e7-929b-586f76f3e2b6" containerName="main"
Jul 16 21:21:21.596988 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.596957    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="784b6d1a-f100-4b8b-a8d1-cb715703098a" containerName="main"
Jul 16 21:21:21.596988 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.596964    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="784b6d1a-f100-4b8b-a8d1-cb715703098a" containerName="main"
Jul 16 21:21:21.597322 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.597072    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c88a98a-f57d-45e7-929b-586f76f3e2b6" containerName="main"
Jul 16 21:21:21.597322 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.597082    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="784b6d1a-f100-4b8b-a8d1-cb715703098a" containerName="main"
Jul 16 21:21:21.597322 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.597093    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="795b38f7-b667-4392-aace-0952157f189e" containerName="istio-proxy"
Jul 16 21:21:21.600255 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.600235    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.602457 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.602439    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ed076e-openshift-default-dockercfg-4x2dw\""
Jul 16 21:21:21.613703 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.613674    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"]
Jul 16 21:21:21.783091 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.783061    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-token\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.783091 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.783094    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzsh4\" (UniqueName: \"kubernetes.io/projected/3d64f8f8-4eec-4564-9bba-a8a4529131b8-kube-api-access-mzsh4\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.783617 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.783129    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-workload-socket\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.783617 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.783212    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-data\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.783617 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.783274    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istiod-ca-cert\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.783617 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.783322    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-workload-certs\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.783617 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.783397    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-credential-socket\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.783617 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.783452    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-podinfo\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.783617 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.783479    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-envoy\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.884370 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.884294    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-token\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.884370 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.884325    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzsh4\" (UniqueName: \"kubernetes.io/projected/3d64f8f8-4eec-4564-9bba-a8a4529131b8-kube-api-access-mzsh4\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.884370 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.884344    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-workload-socket\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.884621 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.884380    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-data\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.884621 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.884406    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istiod-ca-cert\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.884621 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.884432    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-workload-certs\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.884621 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.884449    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-credential-socket\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.884621 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.884488    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-podinfo\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.884621 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.884513    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-envoy\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.884907 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.884754    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-workload-socket\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.884907 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.884874    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-data\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.885010 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.884922    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-credential-socket\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.885082 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.885067    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-workload-certs\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.885268 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.885242    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istiod-ca-cert\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.886661 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.886643    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-envoy\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.886970 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.886954    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-podinfo\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.892526 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.892499    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-token\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.892605 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.892562    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzsh4\" (UniqueName: \"kubernetes.io/projected/3d64f8f8-4eec-4564-9bba-a8a4529131b8-kube-api-access-mzsh4\") pod \"e2e-api-a-ed076e-openshift-default-79677c5df-hvslr\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") " pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:21.910386 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:21.910354    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:22.055804 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:22.055774    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"]
Jul 16 21:21:22.056760 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:21:22.056733    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d64f8f8_4eec_4564_9bba_a8a4529131b8.slice/crio-19c06c95681b2819d5a166cecb7f5cc373095f3eb2835d99cb45e24ca56bf163 WatchSource:0}: Error finding container 19c06c95681b2819d5a166cecb7f5cc373095f3eb2835d99cb45e24ca56bf163: Status 404 returned error can't find the container with id 19c06c95681b2819d5a166cecb7f5cc373095f3eb2835d99cb45e24ca56bf163
Jul 16 21:21:22.059008 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:22.058971    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:21:22.059112 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:22.059057    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:21:22.059112 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:22.059093    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:21:22.366845 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:22.366798    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr" event={"ID":"3d64f8f8-4eec-4564-9bba-a8a4529131b8","Type":"ContainerStarted","Data":"2fb63aef773a8cb284a058fd88fdccfe4ed3916cec6fcbb737eaac3b1187e01a"}
Jul 16 21:21:22.367044 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:22.366884    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr" event={"ID":"3d64f8f8-4eec-4564-9bba-a8a4529131b8","Type":"ContainerStarted","Data":"19c06c95681b2819d5a166cecb7f5cc373095f3eb2835d99cb45e24ca56bf163"}
Jul 16 21:21:22.388585 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:22.388508    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr" podStartSLOduration=1.388490067 podStartE2EDuration="1.388490067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:21:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:21:22.387287433 +0000 UTC m=+2928.353221666" watchObservedRunningTime="2026-07-16 21:21:22.388490067 +0000 UTC m=+2928.354424302"
Jul 16 21:21:22.910560 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:22.910519    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:23.916075 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:23.916049    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:24.376065 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:24.375983    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:24.377012 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:24.376992    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:25.967149 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:25.967126    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:21:26.021756 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.021693    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-envoy\") pod \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") "
Jul 16 21:21:26.021868 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.021768    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-podinfo\") pod \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") "
Jul 16 21:21:26.021868 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.021810    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-data\") pod \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") "
Jul 16 21:21:26.021868 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.021858    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdr2n\" (UniqueName: \"kubernetes.io/projected/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-kube-api-access-cdr2n\") pod \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") "
Jul 16 21:21:26.022034 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.021898    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-workload-socket\") pod \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") "
Jul 16 21:21:26.022034 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.021941    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-token\") pod \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") "
Jul 16 21:21:26.022034 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.021997    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-credential-socket\") pod \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") "
Jul 16 21:21:26.022034 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.022025    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-workload-certs\") pod \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") "
Jul 16 21:21:26.022253 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.022059    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istiod-ca-cert\") pod \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\" (UID: \"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445\") "
Jul 16 21:21:26.022308 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.022286    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" (UID: "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:21:26.022465 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.022417    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-credential-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:21:26.022665 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.022628    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" (UID: "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:21:26.022796 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.022765    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-data" (OuterVolumeSpecName: "istio-data") pod "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" (UID: "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:21:26.022870 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.022678    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" (UID: "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:21:26.022991 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.022627    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" (UID: "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:21:26.024216 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.024197    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" (UID: "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:21:26.024675 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.024648    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" (UID: "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 21:21:26.024772 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.024673    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-kube-api-access-cdr2n" (OuterVolumeSpecName: "kube-api-access-cdr2n") pod "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" (UID: "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445"). InnerVolumeSpecName "kube-api-access-cdr2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:21:26.024772 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.024747    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-token" (OuterVolumeSpecName: "istio-token") pod "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" (UID: "f6bdeb6a-db9f-4a42-8c8b-cdfad8892445"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:21:26.123120 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.123094    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-token\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:21:26.123120 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.123117    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-workload-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:21:26.123264 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.123131    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istiod-ca-cert\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:21:26.123264 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.123142    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-envoy\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:21:26.123264 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.123153    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-podinfo\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:21:26.123264 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.123164    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-istio-data\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:21:26.123264 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.123176    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdr2n\" (UniqueName: \"kubernetes.io/projected/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-kube-api-access-cdr2n\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:21:26.123264 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.123191    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445-workload-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:21:26.387801 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.387737    2576 generic.go:356] "Generic (PLEG): container finished" podID="f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" containerID="7cdedb18818f572c8fb331537d7009a53c967ec31bef5a0cf202f899ce4f3b04" exitCode=0
Jul 16 21:21:26.387801 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.387786    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"
Jul 16 21:21:26.387941 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.387821    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t" event={"ID":"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445","Type":"ContainerDied","Data":"7cdedb18818f572c8fb331537d7009a53c967ec31bef5a0cf202f899ce4f3b04"}
Jul 16 21:21:26.387941 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.387875    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t" event={"ID":"f6bdeb6a-db9f-4a42-8c8b-cdfad8892445","Type":"ContainerDied","Data":"395c1ea9eb3e77507e6e3ca41657166555c7d47421b5c12f4cb18bfb4aba05b4"}
Jul 16 21:21:26.387941 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.387899    2576 scope.go:117] "RemoveContainer" containerID="7cdedb18818f572c8fb331537d7009a53c967ec31bef5a0cf202f899ce4f3b04"
Jul 16 21:21:26.397222 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.397206    2576 scope.go:117] "RemoveContainer" containerID="f5809567f0adf53a16b06a6cbb939240bc817c0d7ddc781b317f4e444796e3fe"
Jul 16 21:21:26.405044 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.405028    2576 scope.go:117] "RemoveContainer" containerID="7cdedb18818f572c8fb331537d7009a53c967ec31bef5a0cf202f899ce4f3b04"
Jul 16 21:21:26.405284 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:21:26.405268    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cdedb18818f572c8fb331537d7009a53c967ec31bef5a0cf202f899ce4f3b04\": container with ID starting with 7cdedb18818f572c8fb331537d7009a53c967ec31bef5a0cf202f899ce4f3b04 not found: ID does not exist" containerID="7cdedb18818f572c8fb331537d7009a53c967ec31bef5a0cf202f899ce4f3b04"
Jul 16 21:21:26.405346 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.405289    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cdedb18818f572c8fb331537d7009a53c967ec31bef5a0cf202f899ce4f3b04"} err="failed to get container status \"7cdedb18818f572c8fb331537d7009a53c967ec31bef5a0cf202f899ce4f3b04\": rpc error: code = NotFound desc = could not find container \"7cdedb18818f572c8fb331537d7009a53c967ec31bef5a0cf202f899ce4f3b04\": container with ID starting with 7cdedb18818f572c8fb331537d7009a53c967ec31bef5a0cf202f899ce4f3b04 not found: ID does not exist"
Jul 16 21:21:26.405346 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.405304    2576 scope.go:117] "RemoveContainer" containerID="f5809567f0adf53a16b06a6cbb939240bc817c0d7ddc781b317f4e444796e3fe"
Jul 16 21:21:26.405647 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:21:26.405630    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5809567f0adf53a16b06a6cbb939240bc817c0d7ddc781b317f4e444796e3fe\": container with ID starting with f5809567f0adf53a16b06a6cbb939240bc817c0d7ddc781b317f4e444796e3fe not found: ID does not exist" containerID="f5809567f0adf53a16b06a6cbb939240bc817c0d7ddc781b317f4e444796e3fe"
Jul 16 21:21:26.405706 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.405652    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5809567f0adf53a16b06a6cbb939240bc817c0d7ddc781b317f4e444796e3fe"} err="failed to get container status \"f5809567f0adf53a16b06a6cbb939240bc817c0d7ddc781b317f4e444796e3fe\": rpc error: code = NotFound desc = could not find container \"f5809567f0adf53a16b06a6cbb939240bc817c0d7ddc781b317f4e444796e3fe\": container with ID starting with f5809567f0adf53a16b06a6cbb939240bc817c0d7ddc781b317f4e444796e3fe not found: ID does not exist"
Jul 16 21:21:26.408617 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.408597    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"]
Jul 16 21:21:26.412568 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.412550    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-72cda6-openshift-default-bc9df467c-r4v4t"]
Jul 16 21:21:26.689393 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:26.689345    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" path="/var/lib/kubelet/pods/f6bdeb6a-db9f-4a42-8c8b-cdfad8892445/volumes"
Jul 16 21:21:29.090338 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.090299    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:21:29.570686 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.570652    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"]
Jul 16 21:21:29.571056 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.571042    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" containerName="istio-proxy"
Jul 16 21:21:29.571107 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.571057    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" containerName="istio-proxy"
Jul 16 21:21:29.571107 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.571076    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" containerName="istio-proxy"
Jul 16 21:21:29.571107 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.571082    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" containerName="istio-proxy"
Jul 16 21:21:29.571202 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.571155    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" containerName="istio-proxy"
Jul 16 21:21:29.571202 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.571165    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="795b38f7-b667-4392-aace-0952157f189e" containerName="istio-proxy"
Jul 16 21:21:29.575667 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.575647    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.580094 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.580072    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-df0daf-openshift-default-dockercfg-skgw5\""
Jul 16 21:21:29.588074 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.588049    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"]
Jul 16 21:21:29.649280 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.649258    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-envoy\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.649401 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.649287    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcqh6\" (UniqueName: \"kubernetes.io/projected/fc9604ce-aa3e-4398-84e6-bbd62b74943f-kube-api-access-kcqh6\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.649401 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.649310    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-podinfo\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.649401 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.649330    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-workload-socket\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.649542 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.649424    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-workload-certs\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.649542 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.649450    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-token\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.649542 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.649494    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-credential-socket\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.649542 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.649525    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istiod-ca-cert\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.649670 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.649543    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-data\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.750692 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.750654    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcqh6\" (UniqueName: \"kubernetes.io/projected/fc9604ce-aa3e-4398-84e6-bbd62b74943f-kube-api-access-kcqh6\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.750692 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.750696    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-podinfo\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.750947 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.750719    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-workload-socket\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.750947 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.750751    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-workload-certs\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.750947 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.750777    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-token\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.750947 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.750875    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-credential-socket\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.751219 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.750957    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istiod-ca-cert\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.751219 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.750994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-data\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.751219 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.751074    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-envoy\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.751219 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.751126    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-workload-socket\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.751219 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.751187    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-workload-certs\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.751522 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.751286    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-credential-socket\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.751522 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.751397    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-data\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.751776 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.751753    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istiod-ca-cert\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.753337 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.753315    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-podinfo\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.753446 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.753315    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-envoy\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.761096 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.761070    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcqh6\" (UniqueName: \"kubernetes.io/projected/fc9604ce-aa3e-4398-84e6-bbd62b74943f-kube-api-access-kcqh6\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.761207 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.761077    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-token\") pod \"e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") " pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:29.885763 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:29.885706    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:30.050059 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:30.050037    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"]
Jul 16 21:21:30.052493 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:21:30.052465    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc9604ce_aa3e_4398_84e6_bbd62b74943f.slice/crio-906c733ec49072c874d6a75f74e7249a6a8652f61e3ab366d1eda10896259017 WatchSource:0}: Error finding container 906c733ec49072c874d6a75f74e7249a6a8652f61e3ab366d1eda10896259017: Status 404 returned error can't find the container with id 906c733ec49072c874d6a75f74e7249a6a8652f61e3ab366d1eda10896259017
Jul 16 21:21:30.054556 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:30.054524    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:21:30.054667 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:30.054585    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:21:30.054667 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:30.054613    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:21:30.410571 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:30.410538    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps" event={"ID":"fc9604ce-aa3e-4398-84e6-bbd62b74943f","Type":"ContainerStarted","Data":"60f232ea140da5073d69ad6c11882249cf69c2e9475050accce5263efe478ed3"}
Jul 16 21:21:30.410953 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:30.410591    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps" event={"ID":"fc9604ce-aa3e-4398-84e6-bbd62b74943f","Type":"ContainerStarted","Data":"906c733ec49072c874d6a75f74e7249a6a8652f61e3ab366d1eda10896259017"}
Jul 16 21:21:30.886476 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:30.886382    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:31.892131 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:31.892100    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:31.916683 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:31.916634    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps" podStartSLOduration=2.916619688 podStartE2EDuration="2.916619688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:21:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:21:30.434758138 +0000 UTC m=+2936.400692372" watchObservedRunningTime="2026-07-16 21:21:31.916619688 +0000 UTC m=+2937.882553920"
Jul 16 21:21:32.422073 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:32.422046    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:32.422965 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:32.422946    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:37.346384 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:21:37.346344    2576 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d64f8f8_4eec_4564_9bba_a8a4529131b8.slice/crio-2fb63aef773a8cb284a058fd88fdccfe4ed3916cec6fcbb737eaac3b1187e01a.scope\": RecentStats: unable to find data in memory cache]"
Jul 16 21:21:37.346704 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:21:37.346384    2576 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d64f8f8_4eec_4564_9bba_a8a4529131b8.slice/crio-2fb63aef773a8cb284a058fd88fdccfe4ed3916cec6fcbb737eaac3b1187e01a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d64f8f8_4eec_4564_9bba_a8a4529131b8.slice/crio-conmon-2fb63aef773a8cb284a058fd88fdccfe4ed3916cec6fcbb737eaac3b1187e01a.scope\": RecentStats: unable to find data in memory cache]"
Jul 16 21:21:37.449777 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:37.449748    2576 generic.go:356] "Generic (PLEG): container finished" podID="3d64f8f8-4eec-4564-9bba-a8a4529131b8" containerID="2fb63aef773a8cb284a058fd88fdccfe4ed3916cec6fcbb737eaac3b1187e01a" exitCode=0
Jul 16 21:21:37.449908 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:37.449822    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr" event={"ID":"3d64f8f8-4eec-4564-9bba-a8a4529131b8","Type":"ContainerDied","Data":"2fb63aef773a8cb284a058fd88fdccfe4ed3916cec6fcbb737eaac3b1187e01a"}
Jul 16 21:21:37.450198 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:37.450182    2576 scope.go:117] "RemoveContainer" containerID="2fb63aef773a8cb284a058fd88fdccfe4ed3916cec6fcbb737eaac3b1187e01a"
Jul 16 21:21:37.451151 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:37.451125    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:21:37.451216 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:37.451203    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:21:37.451257 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:37.451233    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:21:38.456473 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:38.456437    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr" event={"ID":"3d64f8f8-4eec-4564-9bba-a8a4529131b8","Type":"ContainerStarted","Data":"0e10a6036eb3a5714c45ecec4515c241f1117f73a66ad91a13e5eeff15b5b72a"}
Jul 16 21:21:38.911583 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:38.911551    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:38.912662 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:38.912637    2576 patch_prober.go:28] interesting pod/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Jul 16 21:21:38.912800 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:38.912692    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr" podUID="3d64f8f8-4eec-4564-9bba-a8a4529131b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Jul 16 21:21:39.376763 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:39.376679    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:39.911076 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:39.911042    2576 patch_prober.go:28] interesting pod/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Jul 16 21:21:39.911501 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:39.911102    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr" podUID="3d64f8f8-4eec-4564-9bba-a8a4529131b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Jul 16 21:21:40.390806 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:40.390776    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:21:40.910921 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:40.910892    2576 patch_prober.go:28] interesting pod/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Jul 16 21:21:40.911072 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:40.910940    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr" podUID="3d64f8f8-4eec-4564-9bba-a8a4529131b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Jul 16 21:21:41.911215 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:41.911185    2576 patch_prober.go:28] interesting pod/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Jul 16 21:21:41.911617 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:41.911244    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr" podUID="3d64f8f8-4eec-4564-9bba-a8a4529131b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Jul 16 21:21:42.911526 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:42.911488    2576 patch_prober.go:28] interesting pod/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Jul 16 21:21:42.911950 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:42.911539    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr" podUID="3d64f8f8-4eec-4564-9bba-a8a4529131b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Jul 16 21:21:43.915166 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:43.915131    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:43.916257 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:43.916239    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:21:48.463527 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:48.463483    2576 patch_prober.go:28] interesting pod/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 16 21:21:48.463919 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:48.463554    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps" podUID="fc9604ce-aa3e-4398-84e6-bbd62b74943f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 16 21:21:49.519066 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:49.519034    2576 generic.go:356] "Generic (PLEG): container finished" podID="fc9604ce-aa3e-4398-84e6-bbd62b74943f" containerID="60f232ea140da5073d69ad6c11882249cf69c2e9475050accce5263efe478ed3" exitCode=0
Jul 16 21:21:49.519472 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:49.519105    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps" event={"ID":"fc9604ce-aa3e-4398-84e6-bbd62b74943f","Type":"ContainerDied","Data":"60f232ea140da5073d69ad6c11882249cf69c2e9475050accce5263efe478ed3"}
Jul 16 21:21:49.519519 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:49.519501    2576 scope.go:117] "RemoveContainer" containerID="60f232ea140da5073d69ad6c11882249cf69c2e9475050accce5263efe478ed3"
Jul 16 21:21:49.520477 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:49.520451    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:21:49.520546 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:49.520526    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:21:49.520587 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:49.520557    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:21:50.526225 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:50.526192    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps" event={"ID":"fc9604ce-aa3e-4398-84e6-bbd62b74943f","Type":"ContainerStarted","Data":"8630afb8dd2bf486ac7ba373e0cc3128e42f75953f9601b8ae1ad16abef5a5c2"}
Jul 16 21:21:50.886207 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:50.886111    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:50.886380 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:50.886318    2576 patch_prober.go:28] interesting pod/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Jul 16 21:21:50.886430 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:50.886389    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps" podUID="fc9604ce-aa3e-4398-84e6-bbd62b74943f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Jul 16 21:21:51.886189 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:51.886156    2576 patch_prober.go:28] interesting pod/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Jul 16 21:21:51.886658 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:51.886210    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps" podUID="fc9604ce-aa3e-4398-84e6-bbd62b74943f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Jul 16 21:21:52.886622 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:52.886591    2576 patch_prober.go:28] interesting pod/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Jul 16 21:21:52.886985 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:52.886636    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps" podUID="fc9604ce-aa3e-4398-84e6-bbd62b74943f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Jul 16 21:21:53.886036 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:53.886001    2576 patch_prober.go:28] interesting pod/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Jul 16 21:21:53.886224 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:53.886053    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps" podUID="fc9604ce-aa3e-4398-84e6-bbd62b74943f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Jul 16 21:21:54.889953 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:54.889922    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:54.890333 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:54.890182    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:21:54.890905 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:21:54.890885    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:22:01.194573 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:01.194485    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:22:10.607133 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:10.607099    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"]
Jul 16 21:22:10.607561 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:10.607337    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr" podUID="3d64f8f8-4eec-4564-9bba-a8a4529131b8" containerName="istio-proxy" containerID="cri-o://0e10a6036eb3a5714c45ecec4515c241f1117f73a66ad91a13e5eeff15b5b72a" gracePeriod=30
Jul 16 21:22:13.917238 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:13.917205    2576 patch_prober.go:28] interesting pod/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 21:22:13.917634 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:13.917268    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr" podUID="3d64f8f8-4eec-4564-9bba-a8a4529131b8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 21:22:15.853291 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.853266    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:22:15.952186 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.952164    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istiod-ca-cert\") pod \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") "
Jul 16 21:22:15.952333 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.952202    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-podinfo\") pod \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") "
Jul 16 21:22:15.952333 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.952245    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-workload-certs\") pod \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") "
Jul 16 21:22:15.952333 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.952279    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzsh4\" (UniqueName: \"kubernetes.io/projected/3d64f8f8-4eec-4564-9bba-a8a4529131b8-kube-api-access-mzsh4\") pod \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") "
Jul 16 21:22:15.952333 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.952306    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-credential-socket\") pod \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") "
Jul 16 21:22:15.952571 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.952387    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-data\") pod \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") "
Jul 16 21:22:15.952571 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.952422    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-envoy\") pod \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") "
Jul 16 21:22:15.952571 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.952463    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-workload-socket\") pod \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") "
Jul 16 21:22:15.952571 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.952500    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-token\") pod \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\" (UID: \"3d64f8f8-4eec-4564-9bba-a8a4529131b8\") "
Jul 16 21:22:15.952571 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.952501    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3d64f8f8-4eec-4564-9bba-a8a4529131b8" (UID: "3d64f8f8-4eec-4564-9bba-a8a4529131b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:15.952854 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.952809    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3d64f8f8-4eec-4564-9bba-a8a4529131b8" (UID: "3d64f8f8-4eec-4564-9bba-a8a4529131b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:15.952854 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.952831    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-workload-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:15.953034 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.953012    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3d64f8f8-4eec-4564-9bba-a8a4529131b8" (UID: "3d64f8f8-4eec-4564-9bba-a8a4529131b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:15.953110 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.952576    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3d64f8f8-4eec-4564-9bba-a8a4529131b8" (UID: "3d64f8f8-4eec-4564-9bba-a8a4529131b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:22:15.953330 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.953303    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "3d64f8f8-4eec-4564-9bba-a8a4529131b8" (UID: "3d64f8f8-4eec-4564-9bba-a8a4529131b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:15.954645 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.954617    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d64f8f8-4eec-4564-9bba-a8a4529131b8-kube-api-access-mzsh4" (OuterVolumeSpecName: "kube-api-access-mzsh4") pod "3d64f8f8-4eec-4564-9bba-a8a4529131b8" (UID: "3d64f8f8-4eec-4564-9bba-a8a4529131b8"). InnerVolumeSpecName "kube-api-access-mzsh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:22:15.954788 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.954772    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3d64f8f8-4eec-4564-9bba-a8a4529131b8" (UID: "3d64f8f8-4eec-4564-9bba-a8a4529131b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 21:22:15.954869 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.954845    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3d64f8f8-4eec-4564-9bba-a8a4529131b8" (UID: "3d64f8f8-4eec-4564-9bba-a8a4529131b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:15.954911 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:15.954867    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "3d64f8f8-4eec-4564-9bba-a8a4529131b8" (UID: "3d64f8f8-4eec-4564-9bba-a8a4529131b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:22:16.053926 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.053901    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-credential-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:16.053926 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.053924    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-data\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:16.054058 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.053933    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-envoy\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:16.054058 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.053941    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d64f8f8-4eec-4564-9bba-a8a4529131b8-workload-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:16.054058 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.053951    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-token\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:16.054058 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.053959    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istiod-ca-cert\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:16.054058 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.053967    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d64f8f8-4eec-4564-9bba-a8a4529131b8-istio-podinfo\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:16.054058 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.053974    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzsh4\" (UniqueName: \"kubernetes.io/projected/3d64f8f8-4eec-4564-9bba-a8a4529131b8-kube-api-access-mzsh4\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:16.663168 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.663137    2576 generic.go:356] "Generic (PLEG): container finished" podID="3d64f8f8-4eec-4564-9bba-a8a4529131b8" containerID="0e10a6036eb3a5714c45ecec4515c241f1117f73a66ad91a13e5eeff15b5b72a" exitCode=0
Jul 16 21:22:16.663300 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.663195    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"
Jul 16 21:22:16.663300 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.663225    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr" event={"ID":"3d64f8f8-4eec-4564-9bba-a8a4529131b8","Type":"ContainerDied","Data":"0e10a6036eb3a5714c45ecec4515c241f1117f73a66ad91a13e5eeff15b5b72a"}
Jul 16 21:22:16.663300 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.663275    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr" event={"ID":"3d64f8f8-4eec-4564-9bba-a8a4529131b8","Type":"ContainerDied","Data":"19c06c95681b2819d5a166cecb7f5cc373095f3eb2835d99cb45e24ca56bf163"}
Jul 16 21:22:16.663300 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.663296    2576 scope.go:117] "RemoveContainer" containerID="0e10a6036eb3a5714c45ecec4515c241f1117f73a66ad91a13e5eeff15b5b72a"
Jul 16 21:22:16.673095 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.673080    2576 scope.go:117] "RemoveContainer" containerID="2fb63aef773a8cb284a058fd88fdccfe4ed3916cec6fcbb737eaac3b1187e01a"
Jul 16 21:22:16.680751 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.680732    2576 scope.go:117] "RemoveContainer" containerID="0e10a6036eb3a5714c45ecec4515c241f1117f73a66ad91a13e5eeff15b5b72a"
Jul 16 21:22:16.680983 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:22:16.680964    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e10a6036eb3a5714c45ecec4515c241f1117f73a66ad91a13e5eeff15b5b72a\": container with ID starting with 0e10a6036eb3a5714c45ecec4515c241f1117f73a66ad91a13e5eeff15b5b72a not found: ID does not exist" containerID="0e10a6036eb3a5714c45ecec4515c241f1117f73a66ad91a13e5eeff15b5b72a"
Jul 16 21:22:16.681025 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.680990    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e10a6036eb3a5714c45ecec4515c241f1117f73a66ad91a13e5eeff15b5b72a"} err="failed to get container status \"0e10a6036eb3a5714c45ecec4515c241f1117f73a66ad91a13e5eeff15b5b72a\": rpc error: code = NotFound desc = could not find container \"0e10a6036eb3a5714c45ecec4515c241f1117f73a66ad91a13e5eeff15b5b72a\": container with ID starting with 0e10a6036eb3a5714c45ecec4515c241f1117f73a66ad91a13e5eeff15b5b72a not found: ID does not exist"
Jul 16 21:22:16.681025 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.681008    2576 scope.go:117] "RemoveContainer" containerID="2fb63aef773a8cb284a058fd88fdccfe4ed3916cec6fcbb737eaac3b1187e01a"
Jul 16 21:22:16.681216 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:22:16.681200    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fb63aef773a8cb284a058fd88fdccfe4ed3916cec6fcbb737eaac3b1187e01a\": container with ID starting with 2fb63aef773a8cb284a058fd88fdccfe4ed3916cec6fcbb737eaac3b1187e01a not found: ID does not exist" containerID="2fb63aef773a8cb284a058fd88fdccfe4ed3916cec6fcbb737eaac3b1187e01a"
Jul 16 21:22:16.681259 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.681222    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fb63aef773a8cb284a058fd88fdccfe4ed3916cec6fcbb737eaac3b1187e01a"} err="failed to get container status \"2fb63aef773a8cb284a058fd88fdccfe4ed3916cec6fcbb737eaac3b1187e01a\": rpc error: code = NotFound desc = could not find container \"2fb63aef773a8cb284a058fd88fdccfe4ed3916cec6fcbb737eaac3b1187e01a\": container with ID starting with 2fb63aef773a8cb284a058fd88fdccfe4ed3916cec6fcbb737eaac3b1187e01a not found: ID does not exist"
Jul 16 21:22:16.689419 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.689400    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"]
Jul 16 21:22:16.690068 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:16.690049    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ed076e-openshift-default-79677c5df-hvslr"]
Jul 16 21:22:18.689960 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:18.689918    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d64f8f8-4eec-4564-9bba-a8a4529131b8" path="/var/lib/kubelet/pods/3d64f8f8-4eec-4564-9bba-a8a4529131b8/volumes"
Jul 16 21:22:24.306353 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:24.306321    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46"]
Jul 16 21:22:24.306759 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:24.306710    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d64f8f8-4eec-4564-9bba-a8a4529131b8" containerName="istio-proxy"
Jul 16 21:22:24.306759 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:24.306721    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d64f8f8-4eec-4564-9bba-a8a4529131b8" containerName="istio-proxy"
Jul 16 21:22:24.306847 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:24.306788    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d64f8f8-4eec-4564-9bba-a8a4529131b8" containerName="istio-proxy"
Jul 16 21:22:24.306847 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:24.306797    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6bdeb6a-db9f-4a42-8c8b-cdfad8892445" containerName="istio-proxy"
Jul 16 21:22:24.309949 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:24.309929    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46"
Jul 16 21:22:24.312224 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:24.312204    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:22:24.312339 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:24.312213    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:22:24.316436 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:24.316413    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33e414e8-8beb-49e4-a64d-4ee2d9f974e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46\" (UID: \"33e414e8-8beb-49e4-a64d-4ee2d9f974e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46"
Jul 16 21:22:24.316867 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:24.316843    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46"]
Jul 16 21:22:24.417228 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:24.417204    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33e414e8-8beb-49e4-a64d-4ee2d9f974e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46\" (UID: \"33e414e8-8beb-49e4-a64d-4ee2d9f974e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46"
Jul 16 21:22:24.417847 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:24.417830    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33e414e8-8beb-49e4-a64d-4ee2d9f974e6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46\" (UID: \"33e414e8-8beb-49e4-a64d-4ee2d9f974e6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46"
Jul 16 21:22:24.621501 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:24.621443    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46"
Jul 16 21:22:24.699812 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:24.699785    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:22:24.968152 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:24.968129    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46"]
Jul 16 21:22:24.970798 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:22:24.970771    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33e414e8_8beb_49e4_a64d_4ee2d9f974e6.slice/crio-a751f3030f56d75cf8a469b3e09be14639889fc527ad88df8f9c55ee86cf1ecd WatchSource:0}: Error finding container a751f3030f56d75cf8a469b3e09be14639889fc527ad88df8f9c55ee86cf1ecd: Status 404 returned error can't find the container with id a751f3030f56d75cf8a469b3e09be14639889fc527ad88df8f9c55ee86cf1ecd
Jul 16 21:22:25.710550 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:25.710522    2576 generic.go:356] "Generic (PLEG): container finished" podID="33e414e8-8beb-49e4-a64d-4ee2d9f974e6" containerID="09aba442f3b396dae608a05d1515261054586b62d2c6241b08216822990e384d" exitCode=6
Jul 16 21:22:25.710893 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:25.710613    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46" event={"ID":"33e414e8-8beb-49e4-a64d-4ee2d9f974e6","Type":"ContainerDied","Data":"09aba442f3b396dae608a05d1515261054586b62d2c6241b08216822990e384d"}
Jul 16 21:22:25.710893 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:25.710676    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46" event={"ID":"33e414e8-8beb-49e4-a64d-4ee2d9f974e6","Type":"ContainerStarted","Data":"a751f3030f56d75cf8a469b3e09be14639889fc527ad88df8f9c55ee86cf1ecd"}
Jul 16 21:22:25.710893 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:25.710746    2576 scope.go:117] "RemoveContainer" containerID="09aba442f3b396dae608a05d1515261054586b62d2c6241b08216822990e384d"
Jul 16 21:22:26.717283 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:26.717198    2576 generic.go:356] "Generic (PLEG): container finished" podID="33e414e8-8beb-49e4-a64d-4ee2d9f974e6" containerID="b153b338b4916c29c05ea7e7652a0eeadeb226c824e7b40252141f62c4395b9f" exitCode=6
Jul 16 21:22:26.717720 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:26.717280    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46" event={"ID":"33e414e8-8beb-49e4-a64d-4ee2d9f974e6","Type":"ContainerDied","Data":"b153b338b4916c29c05ea7e7652a0eeadeb226c824e7b40252141f62c4395b9f"}
Jul 16 21:22:26.717720 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:26.717338    2576 scope.go:117] "RemoveContainer" containerID="09aba442f3b396dae608a05d1515261054586b62d2c6241b08216822990e384d"
Jul 16 21:22:26.717720 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:26.717529    2576 scope.go:117] "RemoveContainer" containerID="b153b338b4916c29c05ea7e7652a0eeadeb226c824e7b40252141f62c4395b9f"
Jul 16 21:22:26.717845 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:22:26.717751    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46_odh-ai-gateway-infra(33e414e8-8beb-49e4-a64d-4ee2d9f974e6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46" podUID="33e414e8-8beb-49e4-a64d-4ee2d9f974e6"
Jul 16 21:22:27.724678 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:27.724646    2576 scope.go:117] "RemoveContainer" containerID="b153b338b4916c29c05ea7e7652a0eeadeb226c824e7b40252141f62c4395b9f"
Jul 16 21:22:27.725052 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:22:27.724810    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46_odh-ai-gateway-infra(33e414e8-8beb-49e4-a64d-4ee2d9f974e6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46" podUID="33e414e8-8beb-49e4-a64d-4ee2d9f974e6"
Jul 16 21:22:34.706288 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:34.706255    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 21:22:34.714314 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:34.714290    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 21:22:36.741906 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:36.741868    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"]
Jul 16 21:22:36.742512 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:36.742419    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps" podUID="fc9604ce-aa3e-4398-84e6-bbd62b74943f" containerName="istio-proxy" containerID="cri-o://8630afb8dd2bf486ac7ba373e0cc3128e42f75953f9601b8ae1ad16abef5a5c2" gracePeriod=30
Jul 16 21:22:37.612798 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.612766    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"]
Jul 16 21:22:37.613144 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.613131    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d64f8f8-4eec-4564-9bba-a8a4529131b8" containerName="istio-proxy"
Jul 16 21:22:37.613203 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.613145    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d64f8f8-4eec-4564-9bba-a8a4529131b8" containerName="istio-proxy"
Jul 16 21:22:37.613242 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.613221    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d64f8f8-4eec-4564-9bba-a8a4529131b8" containerName="istio-proxy"
Jul 16 21:22:37.617318 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.617300    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.626873 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.626853    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-9331e3-openshift-default-dockercfg-sjxmd\""
Jul 16 21:22:37.636963 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.636942    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"]
Jul 16 21:22:37.713766 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.713733    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19df913d-1cd0-480f-aeec-77c1bbecbe66-istiod-ca-cert\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.713766 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.713768    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-workload-certs\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.714123 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.713786    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-envoy\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.714123 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.713842    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-workload-socket\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.714123 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.713882    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-token\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.714123 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.713948    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfdx5\" (UniqueName: \"kubernetes.io/projected/19df913d-1cd0-480f-aeec-77c1bbecbe66-kube-api-access-pfdx5\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.714123 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.713993    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-credential-socket\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.714123 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.714022    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-data\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.714123 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.714071    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-podinfo\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.815383 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.815327    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-podinfo\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.815844 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.815398    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19df913d-1cd0-480f-aeec-77c1bbecbe66-istiod-ca-cert\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.815844 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.815417    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-workload-certs\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.815844 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.815431    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-envoy\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.815844 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.815449    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-workload-socket\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.815844 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.815472    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-token\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.815844 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.815533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfdx5\" (UniqueName: \"kubernetes.io/projected/19df913d-1cd0-480f-aeec-77c1bbecbe66-kube-api-access-pfdx5\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.815844 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.815593    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-credential-socket\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.815844 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.815634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-data\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.816267 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.815968    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-workload-certs\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.816267 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.816106    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-workload-socket\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.816267 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.816107    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-data\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.816449 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.816300    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19df913d-1cd0-480f-aeec-77c1bbecbe66-istiod-ca-cert\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.816449 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.816303    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-credential-socket\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.817939 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.817912    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-envoy\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.818055 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.818023    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-podinfo\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.824090 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.824064    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-token\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.824589 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.824571    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfdx5\" (UniqueName: \"kubernetes.io/projected/19df913d-1cd0-480f-aeec-77c1bbecbe66-kube-api-access-pfdx5\") pod \"e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") " pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:37.927277 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:37.927242    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:38.072112 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:38.072083    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"]
Jul 16 21:22:38.074715 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:22:38.074682    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19df913d_1cd0_480f_aeec_77c1bbecbe66.slice/crio-ca33c2baa971130eccb0c0fe499ab5f9284764345fd7be9e091da55df1065ecd WatchSource:0}: Error finding container ca33c2baa971130eccb0c0fe499ab5f9284764345fd7be9e091da55df1065ecd: Status 404 returned error can't find the container with id ca33c2baa971130eccb0c0fe499ab5f9284764345fd7be9e091da55df1065ecd
Jul 16 21:22:38.076936 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:38.076906    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:22:38.077001 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:38.076973    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:22:38.077043 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:38.077000    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:22:38.782652 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:38.782618    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp" event={"ID":"19df913d-1cd0-480f-aeec-77c1bbecbe66","Type":"ContainerStarted","Data":"f36bf619323690be6bb3ecd469af0825d96da365b9496c27fe87a014ef34ffd8"}
Jul 16 21:22:38.782827 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:38.782666    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp" event={"ID":"19df913d-1cd0-480f-aeec-77c1bbecbe66","Type":"ContainerStarted","Data":"ca33c2baa971130eccb0c0fe499ab5f9284764345fd7be9e091da55df1065ecd"}
Jul 16 21:22:38.806314 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:38.806247    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp" podStartSLOduration=1.806227598 podStartE2EDuration="1.806227598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:22:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:22:38.803353103 +0000 UTC m=+3004.769287337" watchObservedRunningTime="2026-07-16 21:22:38.806227598 +0000 UTC m=+3004.772161831"
Jul 16 21:22:38.928282 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:38.928254    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:39.892421 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:39.892385    2576 patch_prober.go:28] interesting pod/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 21:22:39.892593 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:39.892435    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps" podUID="fc9604ce-aa3e-4398-84e6-bbd62b74943f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 21:22:39.932902 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:39.932876    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:40.792001 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:40.791974    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:40.792811 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:40.792794    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:41.683999 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:41.683969    2576 scope.go:117] "RemoveContainer" containerID="b153b338b4916c29c05ea7e7652a0eeadeb226c824e7b40252141f62c4395b9f"
Jul 16 21:22:41.997042 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:41.997017    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:22:42.054113 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.054089    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-workload-certs\") pod \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") "
Jul 16 21:22:42.054249 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.054138    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcqh6\" (UniqueName: \"kubernetes.io/projected/fc9604ce-aa3e-4398-84e6-bbd62b74943f-kube-api-access-kcqh6\") pod \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") "
Jul 16 21:22:42.054249 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.054169    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-credential-socket\") pod \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") "
Jul 16 21:22:42.054249 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.054189    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-workload-socket\") pod \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") "
Jul 16 21:22:42.054249 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.054204    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-token\") pod \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") "
Jul 16 21:22:42.054484 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.054252    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-podinfo\") pod \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") "
Jul 16 21:22:42.054484 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.054287    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-envoy\") pod \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") "
Jul 16 21:22:42.054484 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.054319    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-data\") pod \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") "
Jul 16 21:22:42.054484 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.054386    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istiod-ca-cert\") pod \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\" (UID: \"fc9604ce-aa3e-4398-84e6-bbd62b74943f\") "
Jul 16 21:22:42.054484 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.054391    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fc9604ce-aa3e-4398-84e6-bbd62b74943f" (UID: "fc9604ce-aa3e-4398-84e6-bbd62b74943f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:42.054484 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.054480    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fc9604ce-aa3e-4398-84e6-bbd62b74943f" (UID: "fc9604ce-aa3e-4398-84e6-bbd62b74943f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:42.054777 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.054573    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fc9604ce-aa3e-4398-84e6-bbd62b74943f" (UID: "fc9604ce-aa3e-4398-84e6-bbd62b74943f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:42.054838 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.054780    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-workload-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:42.054838 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.054799    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-credential-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:42.054838 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.054808    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fc9604ce-aa3e-4398-84e6-bbd62b74943f" (UID: "fc9604ce-aa3e-4398-84e6-bbd62b74943f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:22:42.054838 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.054816    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-workload-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:42.055238 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.055212    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-data" (OuterVolumeSpecName: "istio-data") pod "fc9604ce-aa3e-4398-84e6-bbd62b74943f" (UID: "fc9604ce-aa3e-4398-84e6-bbd62b74943f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:42.056493 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.056459    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-token" (OuterVolumeSpecName: "istio-token") pod "fc9604ce-aa3e-4398-84e6-bbd62b74943f" (UID: "fc9604ce-aa3e-4398-84e6-bbd62b74943f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:22:42.056775 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.056749    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fc9604ce-aa3e-4398-84e6-bbd62b74943f" (UID: "fc9604ce-aa3e-4398-84e6-bbd62b74943f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:22:42.057149 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.057125    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc9604ce-aa3e-4398-84e6-bbd62b74943f-kube-api-access-kcqh6" (OuterVolumeSpecName: "kube-api-access-kcqh6") pod "fc9604ce-aa3e-4398-84e6-bbd62b74943f" (UID: "fc9604ce-aa3e-4398-84e6-bbd62b74943f"). InnerVolumeSpecName "kube-api-access-kcqh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:22:42.057257 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.057226    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fc9604ce-aa3e-4398-84e6-bbd62b74943f" (UID: "fc9604ce-aa3e-4398-84e6-bbd62b74943f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 21:22:42.155399 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.155348    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcqh6\" (UniqueName: \"kubernetes.io/projected/fc9604ce-aa3e-4398-84e6-bbd62b74943f-kube-api-access-kcqh6\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:42.155493 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.155400    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-token\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:42.155493 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.155413    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-podinfo\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:42.155493 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.155420    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-envoy\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:42.155493 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.155428    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istio-data\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:42.155493 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.155437    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc9604ce-aa3e-4398-84e6-bbd62b74943f-istiod-ca-cert\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:42.802508 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.802480    2576 generic.go:356] "Generic (PLEG): container finished" podID="fc9604ce-aa3e-4398-84e6-bbd62b74943f" containerID="8630afb8dd2bf486ac7ba373e0cc3128e42f75953f9601b8ae1ad16abef5a5c2" exitCode=0
Jul 16 21:22:42.802906 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.802530    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"
Jul 16 21:22:42.802906 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.802564    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps" event={"ID":"fc9604ce-aa3e-4398-84e6-bbd62b74943f","Type":"ContainerDied","Data":"8630afb8dd2bf486ac7ba373e0cc3128e42f75953f9601b8ae1ad16abef5a5c2"}
Jul 16 21:22:42.802906 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.802628    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps" event={"ID":"fc9604ce-aa3e-4398-84e6-bbd62b74943f","Type":"ContainerDied","Data":"906c733ec49072c874d6a75f74e7249a6a8652f61e3ab366d1eda10896259017"}
Jul 16 21:22:42.802906 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.802663    2576 scope.go:117] "RemoveContainer" containerID="8630afb8dd2bf486ac7ba373e0cc3128e42f75953f9601b8ae1ad16abef5a5c2"
Jul 16 21:22:42.804589 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.804569    2576 generic.go:356] "Generic (PLEG): container finished" podID="33e414e8-8beb-49e4-a64d-4ee2d9f974e6" containerID="51a20584967ae959139d899a357f3c417f435e6f4c12a7e7d061ce7dc77aab08" exitCode=6
Jul 16 21:22:42.804746 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.804724    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46" event={"ID":"33e414e8-8beb-49e4-a64d-4ee2d9f974e6","Type":"ContainerDied","Data":"51a20584967ae959139d899a357f3c417f435e6f4c12a7e7d061ce7dc77aab08"}
Jul 16 21:22:42.805051 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.805032    2576 scope.go:117] "RemoveContainer" containerID="51a20584967ae959139d899a357f3c417f435e6f4c12a7e7d061ce7dc77aab08"
Jul 16 21:22:42.805217 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:22:42.805196    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46_odh-ai-gateway-infra(33e414e8-8beb-49e4-a64d-4ee2d9f974e6)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46" podUID="33e414e8-8beb-49e4-a64d-4ee2d9f974e6"
Jul 16 21:22:42.811459 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.811443    2576 scope.go:117] "RemoveContainer" containerID="60f232ea140da5073d69ad6c11882249cf69c2e9475050accce5263efe478ed3"
Jul 16 21:22:42.818971 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.818957    2576 scope.go:117] "RemoveContainer" containerID="8630afb8dd2bf486ac7ba373e0cc3128e42f75953f9601b8ae1ad16abef5a5c2"
Jul 16 21:22:42.819224 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:22:42.819206    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8630afb8dd2bf486ac7ba373e0cc3128e42f75953f9601b8ae1ad16abef5a5c2\": container with ID starting with 8630afb8dd2bf486ac7ba373e0cc3128e42f75953f9601b8ae1ad16abef5a5c2 not found: ID does not exist" containerID="8630afb8dd2bf486ac7ba373e0cc3128e42f75953f9601b8ae1ad16abef5a5c2"
Jul 16 21:22:42.819296 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.819231    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8630afb8dd2bf486ac7ba373e0cc3128e42f75953f9601b8ae1ad16abef5a5c2"} err="failed to get container status \"8630afb8dd2bf486ac7ba373e0cc3128e42f75953f9601b8ae1ad16abef5a5c2\": rpc error: code = NotFound desc = could not find container \"8630afb8dd2bf486ac7ba373e0cc3128e42f75953f9601b8ae1ad16abef5a5c2\": container with ID starting with 8630afb8dd2bf486ac7ba373e0cc3128e42f75953f9601b8ae1ad16abef5a5c2 not found: ID does not exist"
Jul 16 21:22:42.819296 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.819245    2576 scope.go:117] "RemoveContainer" containerID="60f232ea140da5073d69ad6c11882249cf69c2e9475050accce5263efe478ed3"
Jul 16 21:22:42.819501 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:22:42.819485    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60f232ea140da5073d69ad6c11882249cf69c2e9475050accce5263efe478ed3\": container with ID starting with 60f232ea140da5073d69ad6c11882249cf69c2e9475050accce5263efe478ed3 not found: ID does not exist" containerID="60f232ea140da5073d69ad6c11882249cf69c2e9475050accce5263efe478ed3"
Jul 16 21:22:42.819540 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.819505    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60f232ea140da5073d69ad6c11882249cf69c2e9475050accce5263efe478ed3"} err="failed to get container status \"60f232ea140da5073d69ad6c11882249cf69c2e9475050accce5263efe478ed3\": rpc error: code = NotFound desc = could not find container \"60f232ea140da5073d69ad6c11882249cf69c2e9475050accce5263efe478ed3\": container with ID starting with 60f232ea140da5073d69ad6c11882249cf69c2e9475050accce5263efe478ed3 not found: ID does not exist"
Jul 16 21:22:42.819540 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.819517    2576 scope.go:117] "RemoveContainer" containerID="b153b338b4916c29c05ea7e7652a0eeadeb226c824e7b40252141f62c4395b9f"
Jul 16 21:22:42.842843 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.842823    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"]
Jul 16 21:22:42.853462 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:42.853432    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-df0daf-openshift-default-695485d5ff-jxjps"]
Jul 16 21:22:43.840871 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:43.840840    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46"]
Jul 16 21:22:43.974012 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:43.973992    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46"
Jul 16 21:22:44.069295 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:44.069265    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33e414e8-8beb-49e4-a64d-4ee2d9f974e6-maas-api-service-ca\") pod \"33e414e8-8beb-49e4-a64d-4ee2d9f974e6\" (UID: \"33e414e8-8beb-49e4-a64d-4ee2d9f974e6\") "
Jul 16 21:22:44.069598 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:44.069577    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33e414e8-8beb-49e4-a64d-4ee2d9f974e6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "33e414e8-8beb-49e4-a64d-4ee2d9f974e6" (UID: "33e414e8-8beb-49e4-a64d-4ee2d9f974e6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:22:44.170734 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:44.170709    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33e414e8-8beb-49e4-a64d-4ee2d9f974e6-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:22:44.689798 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:44.689759    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc9604ce-aa3e-4398-84e6-bbd62b74943f" path="/var/lib/kubelet/pods/fc9604ce-aa3e-4398-84e6-bbd62b74943f/volumes"
Jul 16 21:22:44.817501 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:44.817481    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46"
Jul 16 21:22:44.817633 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:44.817508    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46" event={"ID":"33e414e8-8beb-49e4-a64d-4ee2d9f974e6","Type":"ContainerDied","Data":"a751f3030f56d75cf8a469b3e09be14639889fc527ad88df8f9c55ee86cf1ecd"}
Jul 16 21:22:44.817633 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:44.817558    2576 scope.go:117] "RemoveContainer" containerID="51a20584967ae959139d899a357f3c417f435e6f4c12a7e7d061ce7dc77aab08"
Jul 16 21:22:44.837120 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:44.837098    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46"]
Jul 16 21:22:44.842318 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:44.842293    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-7hj46"]
Jul 16 21:22:45.491695 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:45.491666    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:22:45.993992 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:45.993949    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"]
Jul 16 21:22:45.994527 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:45.994509    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33e414e8-8beb-49e4-a64d-4ee2d9f974e6" containerName="revoke-keys"
Jul 16 21:22:45.994623 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:45.994530    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="33e414e8-8beb-49e4-a64d-4ee2d9f974e6" containerName="revoke-keys"
Jul 16 21:22:45.994623 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:45.994541    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33e414e8-8beb-49e4-a64d-4ee2d9f974e6" containerName="revoke-keys"
Jul 16 21:22:45.994623 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:45.994549    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="33e414e8-8beb-49e4-a64d-4ee2d9f974e6" containerName="revoke-keys"
Jul 16 21:22:45.994623 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:45.994558    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc9604ce-aa3e-4398-84e6-bbd62b74943f" containerName="istio-proxy"
Jul 16 21:22:45.994623 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:45.994566    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc9604ce-aa3e-4398-84e6-bbd62b74943f" containerName="istio-proxy"
Jul 16 21:22:45.994870 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:45.994670    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc9604ce-aa3e-4398-84e6-bbd62b74943f" containerName="istio-proxy"
Jul 16 21:22:45.994870 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:45.994686    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="33e414e8-8beb-49e4-a64d-4ee2d9f974e6" containerName="revoke-keys"
Jul 16 21:22:45.994870 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:45.994698    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="33e414e8-8beb-49e4-a64d-4ee2d9f974e6" containerName="revoke-keys"
Jul 16 21:22:45.997870 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:45.997850    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.008076 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.008058    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-fb5e77-openshift-default-dockercfg-fqxmr\""
Jul 16 21:22:46.025355 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.025333    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"]
Jul 16 21:22:46.085674 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.085639    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istiod-ca-cert\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.085846 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.085696    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-envoy\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.085846 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.085769    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-credential-socket\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.085846 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.085831    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-podinfo\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.086022 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.085901    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-workload-socket\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.086022 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.085925    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-workload-certs\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.086022 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.085954    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-token\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.086022 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.086009    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-data\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.086214 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.086086    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89h6w\" (UniqueName: \"kubernetes.io/projected/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-kube-api-access-89h6w\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.186754 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.186723    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89h6w\" (UniqueName: \"kubernetes.io/projected/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-kube-api-access-89h6w\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.186934 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.186764    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istiod-ca-cert\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.186934 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.186788    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-envoy\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.186934 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.186814    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-credential-socket\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.186934 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.186847    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-podinfo\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.186934 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.186898    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-workload-socket\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.186934 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.186922    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-workload-certs\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.187309 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.186944    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-token\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.187309 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.186979    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-data\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.187309 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.187229    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-workload-socket\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.187507 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.187417    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-credential-socket\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.187507 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.187443    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-data\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.187587 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.187523    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-workload-certs\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.187587 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.187560    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istiod-ca-cert\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.189083 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.189062    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-envoy\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.189316 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.189297    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-podinfo\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.194667 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.194647    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-token\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.195318 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.195303    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89h6w\" (UniqueName: \"kubernetes.io/projected/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-kube-api-access-89h6w\") pod \"e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") " pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.307903 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.307822    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:46.450700 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.450668    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"]
Jul 16 21:22:46.451516 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:22:46.451487    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fe4d706_6abd_4aea_9663_36c2f91bf3c6.slice/crio-e61b29c68ef4bf402943c72706b2807f1c1a909241cedc45046ae49633e2237c WatchSource:0}: Error finding container e61b29c68ef4bf402943c72706b2807f1c1a909241cedc45046ae49633e2237c: Status 404 returned error can't find the container with id e61b29c68ef4bf402943c72706b2807f1c1a909241cedc45046ae49633e2237c
Jul 16 21:22:46.454151 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.454119    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:22:46.454232 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.454183    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:22:46.454232 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.454212    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:22:46.690803 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.690769    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33e414e8-8beb-49e4-a64d-4ee2d9f974e6" path="/var/lib/kubelet/pods/33e414e8-8beb-49e4-a64d-4ee2d9f974e6/volumes"
Jul 16 21:22:46.832777 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.832735    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9" event={"ID":"4fe4d706-6abd-4aea-9663-36c2f91bf3c6","Type":"ContainerStarted","Data":"1eef2bde8652a401edbeca8b7861ab7e2a881159feafc7fd77ae56873026698c"}
Jul 16 21:22:46.832967 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.832804    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9" event={"ID":"4fe4d706-6abd-4aea-9663-36c2f91bf3c6","Type":"ContainerStarted","Data":"e61b29c68ef4bf402943c72706b2807f1c1a909241cedc45046ae49633e2237c"}
Jul 16 21:22:46.865944 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:46.865892    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9" podStartSLOduration=1.865876581 podStartE2EDuration="1.865876581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:22:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:22:46.864732931 +0000 UTC m=+3012.830667165" watchObservedRunningTime="2026-07-16 21:22:46.865876581 +0000 UTC m=+3012.831810813"
Jul 16 21:22:47.308433 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:47.308404    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:47.313217 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:47.313188    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:47.840662 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:47.840593    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:47.845237 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:47.845010    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:22:53.877384 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:53.877280    2576 generic.go:356] "Generic (PLEG): container finished" podID="19df913d-1cd0-480f-aeec-77c1bbecbe66" containerID="f36bf619323690be6bb3ecd469af0825d96da365b9496c27fe87a014ef34ffd8" exitCode=0
Jul 16 21:22:53.877806 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:53.877355    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp" event={"ID":"19df913d-1cd0-480f-aeec-77c1bbecbe66","Type":"ContainerDied","Data":"f36bf619323690be6bb3ecd469af0825d96da365b9496c27fe87a014ef34ffd8"}
Jul 16 21:22:53.877806 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:53.877783    2576 scope.go:117] "RemoveContainer" containerID="f36bf619323690be6bb3ecd469af0825d96da365b9496c27fe87a014ef34ffd8"
Jul 16 21:22:53.878743 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:53.878720    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:22:53.878790 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:53.878781    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:22:53.878826 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:53.878815    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:22:54.886868 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:54.886826    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp" event={"ID":"19df913d-1cd0-480f-aeec-77c1bbecbe66","Type":"ContainerStarted","Data":"082d256dfe2f7e062c30e01e7c1df4175ae21c848ff0e3bb44719b17e53c5255"}
Jul 16 21:22:54.927603 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:54.927576    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:54.929483 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:54.929455    2576 patch_prober.go:28] interesting pod/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Jul 16 21:22:54.929697 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:54.929668    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp" podUID="19df913d-1cd0-480f-aeec-77c1bbecbe66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Jul 16 21:22:55.792420 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:55.792385    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:55.927624 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:55.927590    2576 patch_prober.go:28] interesting pod/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Jul 16 21:22:55.928071 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:55.927647    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp" podUID="19df913d-1cd0-480f-aeec-77c1bbecbe66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Jul 16 21:22:56.929027 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:56.928589    2576 patch_prober.go:28] interesting pod/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Jul 16 21:22:56.929027 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:56.928655    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp" podUID="19df913d-1cd0-480f-aeec-77c1bbecbe66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Jul 16 21:22:57.928452 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:57.928412    2576 patch_prober.go:28] interesting pod/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Jul 16 21:22:57.928658 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:57.928478    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp" podUID="19df913d-1cd0-480f-aeec-77c1bbecbe66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Jul 16 21:22:58.928320 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:58.928288    2576 patch_prober.go:28] interesting pod/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Jul 16 21:22:58.928811 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:58.928340    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp" podUID="19df913d-1cd0-480f-aeec-77c1bbecbe66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Jul 16 21:22:59.931868 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:59.931838    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:22:59.932821 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:22:59.932804    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:23:00.917738 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:00.917702    2576 generic.go:356] "Generic (PLEG): container finished" podID="4fe4d706-6abd-4aea-9663-36c2f91bf3c6" containerID="1eef2bde8652a401edbeca8b7861ab7e2a881159feafc7fd77ae56873026698c" exitCode=0
Jul 16 21:23:00.917944 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:00.917775    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9" event={"ID":"4fe4d706-6abd-4aea-9663-36c2f91bf3c6","Type":"ContainerDied","Data":"1eef2bde8652a401edbeca8b7861ab7e2a881159feafc7fd77ae56873026698c"}
Jul 16 21:23:00.918272 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:00.918255    2576 scope.go:117] "RemoveContainer" containerID="1eef2bde8652a401edbeca8b7861ab7e2a881159feafc7fd77ae56873026698c"
Jul 16 21:23:00.919207 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:00.919181    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:23:00.919259 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:00.919244    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:23:00.919294 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:00.919277    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 16 21:23:00.996265 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:00.996238    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:23:01.924569 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:01.924535    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9" event={"ID":"4fe4d706-6abd-4aea-9663-36c2f91bf3c6","Type":"ContainerStarted","Data":"8638bf493d0298f13a8403545fa9ece937e3421754ed48a632cfc79645515a4a"}
Jul 16 21:23:02.308487 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:02.308401    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:23:02.309443 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:02.309410    2576 patch_prober.go:28] interesting pod/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Jul 16 21:23:02.309585 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:02.309476    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9" podUID="4fe4d706-6abd-4aea-9663-36c2f91bf3c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Jul 16 21:23:02.840749 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:02.840705    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:23:03.308812 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:03.308778    2576 patch_prober.go:28] interesting pod/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Jul 16 21:23:03.309209 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:03.308844    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9" podUID="4fe4d706-6abd-4aea-9663-36c2f91bf3c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Jul 16 21:23:04.185127 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.185093    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"]
Jul 16 21:23:04.185540 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.185525    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33e414e8-8beb-49e4-a64d-4ee2d9f974e6" containerName="revoke-keys"
Jul 16 21:23:04.185540 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.185541    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="33e414e8-8beb-49e4-a64d-4ee2d9f974e6" containerName="revoke-keys"
Jul 16 21:23:04.185651 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.185551    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc9604ce-aa3e-4398-84e6-bbd62b74943f" containerName="istio-proxy"
Jul 16 21:23:04.185651 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.185556    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc9604ce-aa3e-4398-84e6-bbd62b74943f" containerName="istio-proxy"
Jul 16 21:23:04.185651 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.185616    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="33e414e8-8beb-49e4-a64d-4ee2d9f974e6" containerName="revoke-keys"
Jul 16 21:23:04.185651 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.185627    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc9604ce-aa3e-4398-84e6-bbd62b74943f" containerName="istio-proxy"
Jul 16 21:23:04.187736 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.187721    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.191288 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.191270    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9331e3\"/\"auth-test-model-9331e3-kserve-self-signed-certs\""
Jul 16 21:23:04.191597 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.191274    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9331e3\"/\"default-dockercfg-b5mjn\""
Jul 16 21:23:04.191854 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.191420    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9331e3\"/\"openshift-service-ca.crt\""
Jul 16 21:23:04.192034 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.192019    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9331e3\"/\"kube-root-ca.crt\""
Jul 16 21:23:04.207503 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.207481    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"]
Jul 16 21:23:04.271285 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.271252    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-model-cache\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.271462 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.271298    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-home\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.271462 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.271384    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-dshm\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.271462 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.271421    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbdf63db-b977-4cf5-b64f-9e48ce401a17-tls-certs\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.271462 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.271452    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62pq2\" (UniqueName: \"kubernetes.io/projected/cbdf63db-b977-4cf5-b64f-9e48ce401a17-kube-api-access-62pq2\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.271605 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.271479    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-tmp-dir\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.308913 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.308890    2576 patch_prober.go:28] interesting pod/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Jul 16 21:23:04.309227 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.308946    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9" podUID="4fe4d706-6abd-4aea-9663-36c2f91bf3c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Jul 16 21:23:04.372258 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.372225    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62pq2\" (UniqueName: \"kubernetes.io/projected/cbdf63db-b977-4cf5-b64f-9e48ce401a17-kube-api-access-62pq2\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.372686 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.372263    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-tmp-dir\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.372686 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.372342    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-model-cache\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.372686 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.372397    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-home\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.372686 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.372433    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-dshm\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.372686 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.372459    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbdf63db-b977-4cf5-b64f-9e48ce401a17-tls-certs\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.372932 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.372725    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-model-cache\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.372932 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.372821    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-home\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.372932 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.372912    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-tmp-dir\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.374876 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.374856    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbdf63db-b977-4cf5-b64f-9e48ce401a17-tls-certs\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.375742 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.375727    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-dshm\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.380270 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.380252    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62pq2\" (UniqueName: \"kubernetes.io/projected/cbdf63db-b977-4cf5-b64f-9e48ce401a17-kube-api-access-62pq2\") pod \"auth-test-model-9331e3-kserve-8b4f846c9-nxw9b\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") " pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.498102 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.498035    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:04.661247 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.661219    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"]
Jul 16 21:23:04.663513 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:23:04.663483    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcbdf63db_b977_4cf5_b64f_9e48ce401a17.slice/crio-bb14f4ecc1fe30dc151f88c78fd9577372f474c1c666f5beeccc71017c7fb334 WatchSource:0}: Error finding container bb14f4ecc1fe30dc151f88c78fd9577372f474c1c666f5beeccc71017c7fb334: Status 404 returned error can't find the container with id bb14f4ecc1fe30dc151f88c78fd9577372f474c1c666f5beeccc71017c7fb334
Jul 16 21:23:04.948468 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.947610    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b" event={"ID":"cbdf63db-b977-4cf5-b64f-9e48ce401a17","Type":"ContainerStarted","Data":"cf1493bdc8ad1d78aa18511cd391066617a529fbe4797a898c3d2d0821130de2"}
Jul 16 21:23:04.948468 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.947662    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b" event={"ID":"cbdf63db-b977-4cf5-b64f-9e48ce401a17","Type":"ContainerStarted","Data":"bb14f4ecc1fe30dc151f88c78fd9577372f474c1c666f5beeccc71017c7fb334"}
Jul 16 21:23:04.967558 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:04.967512    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b" podStartSLOduration=0.967498225 podStartE2EDuration="967.498225ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:23:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:23:04.966483763 +0000 UTC m=+3030.932417998" watchObservedRunningTime="2026-07-16 21:23:04.967498225 +0000 UTC m=+3030.933432459"
Jul 16 21:23:05.309067 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:05.308984    2576 patch_prober.go:28] interesting pod/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Jul 16 21:23:05.309067 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:05.309029    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9" podUID="4fe4d706-6abd-4aea-9663-36c2f91bf3c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Jul 16 21:23:06.312538 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:06.312506    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:23:06.313460 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:06.313432    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:23:14.498831 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:14.498800    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:14.499441 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:14.499091    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:14.511119 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:14.511096    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:14.787224 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:14.787138    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:23:15.007301 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:15.007273    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:23:29.484940 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.484860    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"]
Jul 16 21:23:29.488820 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.488789    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.491998 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.491972    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fb5e77\"/\"openshift-service-ca.crt\""
Jul 16 21:23:29.491998 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.491994    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fb5e77\"/\"auth-test-model-fb5e77-kserve-self-signed-certs\""
Jul 16 21:23:29.492182 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.491982    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fb5e77\"/\"kube-root-ca.crt\""
Jul 16 21:23:29.492182 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.491974    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fb5e77\"/\"default-dockercfg-m9vmz\""
Jul 16 21:23:29.499261 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.499240    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"]
Jul 16 21:23:29.565708 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.565689    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-home\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.565816 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.565734    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-dshm\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.565816 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.565770    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-model-cache\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.565816 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.565788    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-tmp-dir\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.565816 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.565802    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5hk7\" (UniqueName: \"kubernetes.io/projected/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-kube-api-access-g5hk7\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.565957 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.565828    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-tls-certs\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.667152 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.667123    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-dshm\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.667314 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.667173    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-model-cache\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.667314 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.667203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-tmp-dir\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.667314 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.667224    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5hk7\" (UniqueName: \"kubernetes.io/projected/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-kube-api-access-g5hk7\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.667314 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.667270    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-tls-certs\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.667570 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.667347    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-home\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.667696 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.667662    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-model-cache\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.667764 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.667700    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-home\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.667764 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.667739    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-tmp-dir\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.669560 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.669537    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-dshm\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.669835 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.669818    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-tls-certs\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.675139 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.675095    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5hk7\" (UniqueName: \"kubernetes.io/projected/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-kube-api-access-g5hk7\") pod \"auth-test-model-fb5e77-kserve-68fc99959f-w5wlh\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") " pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:29.800648 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:29.800576    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:30.140717 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:30.140691    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"]
Jul 16 21:23:30.142567 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:23:30.142541    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd6c40ca_a39b_4e0c_b8e6_0ce2ce4916fd.slice/crio-2610f732395cf71a5ab7e67f5209122b707af6efa568dbe824bacb92de975083 WatchSource:0}: Error finding container 2610f732395cf71a5ab7e67f5209122b707af6efa568dbe824bacb92de975083: Status 404 returned error can't find the container with id 2610f732395cf71a5ab7e67f5209122b707af6efa568dbe824bacb92de975083
Jul 16 21:23:31.084424 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:31.084389    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh" event={"ID":"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd","Type":"ContainerStarted","Data":"af3218fa37b603b68aa9e3f57442596e196c0a87572a82bf03bf50b3fde3deb0"}
Jul 16 21:23:31.084830 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:31.084430    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh" event={"ID":"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd","Type":"ContainerStarted","Data":"2610f732395cf71a5ab7e67f5209122b707af6efa568dbe824bacb92de975083"}
Jul 16 21:23:31.106487 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:31.106441    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh" podStartSLOduration=2.106426701 podStartE2EDuration="2.106426701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:23:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:23:31.105480112 +0000 UTC m=+3057.071414346" watchObservedRunningTime="2026-07-16 21:23:31.106426701 +0000 UTC m=+3057.072360955"
Jul 16 21:23:33.783736 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:33.783703    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:23:39.801646 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:39.801605    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:39.801646 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:39.801654    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:39.814394 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:39.814349    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:40.144675 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:40.144647    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:23:47.696895 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:23:47.696860    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:24:05.286475 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:24:05.286432    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:24:22.901305 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:24:22.901275    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:24:39.692093 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:24:39.692057    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:24:56.600343 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:24:56.600309    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:25:15.099102 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:15.099025    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:25:25.398445 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:25.398415    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"]
Jul 16 21:25:26.210257 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:26.210228    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"]
Jul 16 21:25:30.499768 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:30.499734    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:25:40.418205 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.418158    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b" podUID="cbdf63db-b977-4cf5-b64f-9e48ce401a17" containerName="main" containerID="cri-o://cf1493bdc8ad1d78aa18511cd391066617a529fbe4797a898c3d2d0821130de2" gracePeriod=45
Jul 16 21:25:40.665192 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.665169    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:25:40.732935 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.732861    2576 generic.go:356] "Generic (PLEG): container finished" podID="cbdf63db-b977-4cf5-b64f-9e48ce401a17" containerID="cf1493bdc8ad1d78aa18511cd391066617a529fbe4797a898c3d2d0821130de2" exitCode=0
Jul 16 21:25:40.732935 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.732897    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"
Jul 16 21:25:40.733094 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.732954    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b" event={"ID":"cbdf63db-b977-4cf5-b64f-9e48ce401a17","Type":"ContainerDied","Data":"cf1493bdc8ad1d78aa18511cd391066617a529fbe4797a898c3d2d0821130de2"}
Jul 16 21:25:40.733094 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.732991    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b" event={"ID":"cbdf63db-b977-4cf5-b64f-9e48ce401a17","Type":"ContainerDied","Data":"bb14f4ecc1fe30dc151f88c78fd9577372f474c1c666f5beeccc71017c7fb334"}
Jul 16 21:25:40.733094 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.733008    2576 scope.go:117] "RemoveContainer" containerID="cf1493bdc8ad1d78aa18511cd391066617a529fbe4797a898c3d2d0821130de2"
Jul 16 21:25:40.738272 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.738254    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-dshm\") pod \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") "
Jul 16 21:25:40.738421 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.738322    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62pq2\" (UniqueName: \"kubernetes.io/projected/cbdf63db-b977-4cf5-b64f-9e48ce401a17-kube-api-access-62pq2\") pod \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") "
Jul 16 21:25:40.738421 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.738355    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbdf63db-b977-4cf5-b64f-9e48ce401a17-tls-certs\") pod \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") "
Jul 16 21:25:40.738421 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.738402    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-tmp-dir\") pod \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") "
Jul 16 21:25:40.738665 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.738447    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-home\") pod \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") "
Jul 16 21:25:40.738665 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.738487    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-model-cache\") pod \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\" (UID: \"cbdf63db-b977-4cf5-b64f-9e48ce401a17\") "
Jul 16 21:25:40.738776 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.738692    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cbdf63db-b977-4cf5-b64f-9e48ce401a17" (UID: "cbdf63db-b977-4cf5-b64f-9e48ce401a17"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:25:40.738776 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.738729    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-home" (OuterVolumeSpecName: "home") pod "cbdf63db-b977-4cf5-b64f-9e48ce401a17" (UID: "cbdf63db-b977-4cf5-b64f-9e48ce401a17"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:25:40.738896 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.738874    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-model-cache" (OuterVolumeSpecName: "model-cache") pod "cbdf63db-b977-4cf5-b64f-9e48ce401a17" (UID: "cbdf63db-b977-4cf5-b64f-9e48ce401a17"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:25:40.738896 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.738878    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-tmp-dir\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:25:40.739012 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.738901    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-home\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:25:40.741037 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.741017    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-dshm" (OuterVolumeSpecName: "dshm") pod "cbdf63db-b977-4cf5-b64f-9e48ce401a17" (UID: "cbdf63db-b977-4cf5-b64f-9e48ce401a17"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:25:40.741150 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.741056    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbdf63db-b977-4cf5-b64f-9e48ce401a17-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cbdf63db-b977-4cf5-b64f-9e48ce401a17" (UID: "cbdf63db-b977-4cf5-b64f-9e48ce401a17"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:25:40.741150 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.741091    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbdf63db-b977-4cf5-b64f-9e48ce401a17-kube-api-access-62pq2" (OuterVolumeSpecName: "kube-api-access-62pq2") pod "cbdf63db-b977-4cf5-b64f-9e48ce401a17" (UID: "cbdf63db-b977-4cf5-b64f-9e48ce401a17"). InnerVolumeSpecName "kube-api-access-62pq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:25:40.742686 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.742670    2576 scope.go:117] "RemoveContainer" containerID="cf1493bdc8ad1d78aa18511cd391066617a529fbe4797a898c3d2d0821130de2"
Jul 16 21:25:40.742935 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:25:40.742915    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf1493bdc8ad1d78aa18511cd391066617a529fbe4797a898c3d2d0821130de2\": container with ID starting with cf1493bdc8ad1d78aa18511cd391066617a529fbe4797a898c3d2d0821130de2 not found: ID does not exist" containerID="cf1493bdc8ad1d78aa18511cd391066617a529fbe4797a898c3d2d0821130de2"
Jul 16 21:25:40.742993 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.742946    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf1493bdc8ad1d78aa18511cd391066617a529fbe4797a898c3d2d0821130de2"} err="failed to get container status \"cf1493bdc8ad1d78aa18511cd391066617a529fbe4797a898c3d2d0821130de2\": rpc error: code = NotFound desc = could not find container \"cf1493bdc8ad1d78aa18511cd391066617a529fbe4797a898c3d2d0821130de2\": container with ID starting with cf1493bdc8ad1d78aa18511cd391066617a529fbe4797a898c3d2d0821130de2 not found: ID does not exist"
Jul 16 21:25:40.839577 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.839552    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-model-cache\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:25:40.839577 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.839574    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbdf63db-b977-4cf5-b64f-9e48ce401a17-dshm\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:25:40.839714 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.839588    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62pq2\" (UniqueName: \"kubernetes.io/projected/cbdf63db-b977-4cf5-b64f-9e48ce401a17-kube-api-access-62pq2\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:25:40.839714 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:40.839601    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbdf63db-b977-4cf5-b64f-9e48ce401a17-tls-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:25:41.055285 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.055262    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"]
Jul 16 21:25:41.059520 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.059496    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-9331e3/auth-test-model-9331e3-kserve-8b4f846c9-nxw9b"]
Jul 16 21:25:41.230240 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.230213    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh" podUID="cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd" containerName="main" containerID="cri-o://af3218fa37b603b68aa9e3f57442596e196c0a87572a82bf03bf50b3fde3deb0" gracePeriod=45
Jul 16 21:25:41.481030 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.481000    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:25:41.544619 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.544593    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-tmp-dir\") pod \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") "
Jul 16 21:25:41.544744 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.544636    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-home\") pod \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") "
Jul 16 21:25:41.544744 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.544660    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5hk7\" (UniqueName: \"kubernetes.io/projected/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-kube-api-access-g5hk7\") pod \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") "
Jul 16 21:25:41.544744 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.544700    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-model-cache\") pod \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") "
Jul 16 21:25:41.544744 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.544726    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-dshm\") pod \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") "
Jul 16 21:25:41.544915 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.544743    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-tls-certs\") pod \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\" (UID: \"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd\") "
Jul 16 21:25:41.544915 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.544872    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-home" (OuterVolumeSpecName: "home") pod "cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd" (UID: "cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:25:41.544915 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.544886    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd" (UID: "cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:25:41.545088 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.544983    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-model-cache" (OuterVolumeSpecName: "model-cache") pod "cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd" (UID: "cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:25:41.545088 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.545071    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-home\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:25:41.545088 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.545082    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-model-cache\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:25:41.545218 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.545093    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-tmp-dir\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:25:41.546944 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.546922    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd" (UID: "cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:25:41.547050 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.546969    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-dshm" (OuterVolumeSpecName: "dshm") pod "cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd" (UID: "cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:25:41.547099 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.547054    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-kube-api-access-g5hk7" (OuterVolumeSpecName: "kube-api-access-g5hk7") pod "cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd" (UID: "cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd"). InnerVolumeSpecName "kube-api-access-g5hk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:25:41.645473 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.645453    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-dshm\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:25:41.645473 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.645473    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-tls-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:25:41.645603 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.645483    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5hk7\" (UniqueName: \"kubernetes.io/projected/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd-kube-api-access-g5hk7\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:25:41.740225 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.740199    2576 generic.go:356] "Generic (PLEG): container finished" podID="cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd" containerID="af3218fa37b603b68aa9e3f57442596e196c0a87572a82bf03bf50b3fde3deb0" exitCode=0
Jul 16 21:25:41.740379 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.740251    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"
Jul 16 21:25:41.740379 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.740293    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh" event={"ID":"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd","Type":"ContainerDied","Data":"af3218fa37b603b68aa9e3f57442596e196c0a87572a82bf03bf50b3fde3deb0"}
Jul 16 21:25:41.740379 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.740333    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh" event={"ID":"cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd","Type":"ContainerDied","Data":"2610f732395cf71a5ab7e67f5209122b707af6efa568dbe824bacb92de975083"}
Jul 16 21:25:41.740379 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.740349    2576 scope.go:117] "RemoveContainer" containerID="af3218fa37b603b68aa9e3f57442596e196c0a87572a82bf03bf50b3fde3deb0"
Jul 16 21:25:41.751905 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.751887    2576 scope.go:117] "RemoveContainer" containerID="af3218fa37b603b68aa9e3f57442596e196c0a87572a82bf03bf50b3fde3deb0"
Jul 16 21:25:41.752218 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:25:41.752194    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af3218fa37b603b68aa9e3f57442596e196c0a87572a82bf03bf50b3fde3deb0\": container with ID starting with af3218fa37b603b68aa9e3f57442596e196c0a87572a82bf03bf50b3fde3deb0 not found: ID does not exist" containerID="af3218fa37b603b68aa9e3f57442596e196c0a87572a82bf03bf50b3fde3deb0"
Jul 16 21:25:41.752309 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.752226    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af3218fa37b603b68aa9e3f57442596e196c0a87572a82bf03bf50b3fde3deb0"} err="failed to get container status \"af3218fa37b603b68aa9e3f57442596e196c0a87572a82bf03bf50b3fde3deb0\": rpc error: code = NotFound desc = could not find container \"af3218fa37b603b68aa9e3f57442596e196c0a87572a82bf03bf50b3fde3deb0\": container with ID starting with af3218fa37b603b68aa9e3f57442596e196c0a87572a82bf03bf50b3fde3deb0 not found: ID does not exist"
Jul 16 21:25:41.767568 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.767544    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"]
Jul 16 21:25:41.774424 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:41.774404    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/auth-test-model-fb5e77-kserve-68fc99959f-w5wlh"]
Jul 16 21:25:42.690270 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:42.690236    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbdf63db-b977-4cf5-b64f-9e48ce401a17" path="/var/lib/kubelet/pods/cbdf63db-b977-4cf5-b64f-9e48ce401a17/volumes"
Jul 16 21:25:42.690818 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:42.690796    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd" path="/var/lib/kubelet/pods/cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd/volumes"
Jul 16 21:25:46.091999 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:25:46.086781    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:26:12.280146 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.280113    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"]
Jul 16 21:26:12.280662 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.280519    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd" containerName="main"
Jul 16 21:26:12.280662 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.280530    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd" containerName="main"
Jul 16 21:26:12.280662 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.280540    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbdf63db-b977-4cf5-b64f-9e48ce401a17" containerName="main"
Jul 16 21:26:12.280662 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.280546    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbdf63db-b977-4cf5-b64f-9e48ce401a17" containerName="main"
Jul 16 21:26:12.280662 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.280613    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbdf63db-b977-4cf5-b64f-9e48ce401a17" containerName="main"
Jul 16 21:26:12.280662 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.280624    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd6c40ca-a39b-4e0c-b8e6-0ce2ce4916fd" containerName="main"
Jul 16 21:26:12.283793 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.283771    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.286924 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.286900    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fb5e77\"/\"kube-root-ca.crt\""
Jul 16 21:26:12.287767 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.287747    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fb5e77\"/\"default-dockercfg-m9vmz\""
Jul 16 21:26:12.287858 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.287791    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fb5e77\"/\"sub-test-model-fb5e77-kserve-self-signed-certs\""
Jul 16 21:26:12.287923 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.287865    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fb5e77\"/\"openshift-service-ca.crt\""
Jul 16 21:26:12.301651 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.301629    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"]
Jul 16 21:26:12.394377 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.394327    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-home\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.394502 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.394389    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6fd6\" (UniqueName: \"kubernetes.io/projected/934a0a57-6757-4e18-9081-d3142cd68251-kube-api-access-c6fd6\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.394502 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.394462    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-tmp-dir\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.394592 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.394529    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/934a0a57-6757-4e18-9081-d3142cd68251-tls-certs\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.394592 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.394569    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-dshm\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.394689 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.394591    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-model-cache\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.495241 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.495210    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/934a0a57-6757-4e18-9081-d3142cd68251-tls-certs\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.495431 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.495264    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-dshm\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.495431 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.495296    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-model-cache\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.495431 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.495392    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-home\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.495582 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.495435    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6fd6\" (UniqueName: \"kubernetes.io/projected/934a0a57-6757-4e18-9081-d3142cd68251-kube-api-access-c6fd6\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.495663 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.495633    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-tmp-dir\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.495851 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.495826    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-home\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.495982 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.495896    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-model-cache\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.496053 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.496027    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-tmp-dir\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.497784 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.497754    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-dshm\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.497888 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.497847    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/934a0a57-6757-4e18-9081-d3142cd68251-tls-certs\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.504137 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.504119    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6fd6\" (UniqueName: \"kubernetes.io/projected/934a0a57-6757-4e18-9081-d3142cd68251-kube-api-access-c6fd6\") pod \"sub-test-model-fb5e77-kserve-76b5958b79-2fqsc\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") " pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.593607 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.593548    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:12.725905 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.725877    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"]
Jul 16 21:26:12.727233 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:26:12.727206    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod934a0a57_6757_4e18_9081_d3142cd68251.slice/crio-7b17dce475eaf22b53e516eb40dfb94f81553ee5419916e514495ede7289fa7d WatchSource:0}: Error finding container 7b17dce475eaf22b53e516eb40dfb94f81553ee5419916e514495ede7289fa7d: Status 404 returned error can't find the container with id 7b17dce475eaf22b53e516eb40dfb94f81553ee5419916e514495ede7289fa7d
Jul 16 21:26:12.901321 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.901292    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc" event={"ID":"934a0a57-6757-4e18-9081-d3142cd68251","Type":"ContainerStarted","Data":"d34ae44937a3e1306ea15aad3a0a1e5077c0416bfc370e88da02b3aee694ef10"}
Jul 16 21:26:12.901495 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.901332    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc" event={"ID":"934a0a57-6757-4e18-9081-d3142cd68251","Type":"ContainerStarted","Data":"7b17dce475eaf22b53e516eb40dfb94f81553ee5419916e514495ede7289fa7d"}
Jul 16 21:26:12.919462 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:12.919419    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc" podStartSLOduration=0.919404761 podStartE2EDuration="919.404761ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:26:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:26:12.917858827 +0000 UTC m=+3218.883793083" watchObservedRunningTime="2026-07-16 21:26:12.919404761 +0000 UTC m=+3218.885338994"
Jul 16 21:26:16.404527 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:16.404497    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:26:22.594219 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:22.594183    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:22.594219 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:22.594228    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:22.611450 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:22.611425    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:22.960110 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:22.960088    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:26:39.312161 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:39.312130    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:26:57.193884 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:26:57.193852    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:27:03.908326 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:03.908289    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"]
Jul 16 21:27:07.680083 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.680053    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"]
Jul 16 21:27:07.683129 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.683109    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.686327 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.686304    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9331e3\"/\"openshift-service-ca.crt\""
Jul 16 21:27:07.686557 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.686354    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-9331e3\"/\"kube-root-ca.crt\""
Jul 16 21:27:07.686557 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.686374    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9331e3\"/\"rate-test-model-9331e3-kserve-self-signed-certs\""
Jul 16 21:27:07.686557 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.686313    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-9331e3\"/\"default-dockercfg-b5mjn\""
Jul 16 21:27:07.694415 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.694392    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"]
Jul 16 21:27:07.744108 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.744084    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-tmp-dir\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.744209 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.744111    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-dshm\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.744209 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.744145    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f773a436-190a-435e-a88b-092cf87102b5-tls-certs\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.744209 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.744173    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksts6\" (UniqueName: \"kubernetes.io/projected/f773a436-190a-435e-a88b-092cf87102b5-kube-api-access-ksts6\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.744352 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.744334    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-model-cache\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.744433 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.744393    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-home\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.845615 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.845591    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-model-cache\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.845723 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.845626    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-home\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.845723 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.845659    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-tmp-dir\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.845723 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.845676    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-dshm\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.845910 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.845734    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f773a436-190a-435e-a88b-092cf87102b5-tls-certs\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.845910 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.845766    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksts6\" (UniqueName: \"kubernetes.io/projected/f773a436-190a-435e-a88b-092cf87102b5-kube-api-access-ksts6\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.846057 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.846036    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-model-cache\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.846460 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.846419    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-tmp-dir\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.846460 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.846436    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-home\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.848068 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.848043    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-dshm\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.848924 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.848902    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f773a436-190a-435e-a88b-092cf87102b5-tls-certs\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.855569 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.855545    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksts6\" (UniqueName: \"kubernetes.io/projected/f773a436-190a-435e-a88b-092cf87102b5-kube-api-access-ksts6\") pod \"rate-test-model-9331e3-kserve-685fb7b87f-z7dcr\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") " pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:07.993005 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:07.992938    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:08.127333 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:08.127304    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"]
Jul 16 21:27:08.128819 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:27:08.128793    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf773a436_190a_435e_a88b_092cf87102b5.slice/crio-96cd98d526f53a9774d045a0b06078cc0dc704f09437c110841f6e72dab7a1af WatchSource:0}: Error finding container 96cd98d526f53a9774d045a0b06078cc0dc704f09437c110841f6e72dab7a1af: Status 404 returned error can't find the container with id 96cd98d526f53a9774d045a0b06078cc0dc704f09437c110841f6e72dab7a1af
Jul 16 21:27:08.169415 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:08.169394    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr" event={"ID":"f773a436-190a-435e-a88b-092cf87102b5","Type":"ContainerStarted","Data":"96cd98d526f53a9774d045a0b06078cc0dc704f09437c110841f6e72dab7a1af"}
Jul 16 21:27:09.181896 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:09.181854    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr" event={"ID":"f773a436-190a-435e-a88b-092cf87102b5","Type":"ContainerStarted","Data":"54e8e0ad1ced30d362fccaf9fabd1004095dcb777b743584cd3d73a755c45701"}
Jul 16 21:27:09.199470 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:09.199421    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr" podStartSLOduration=2.199409027 podStartE2EDuration="2.199409027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:27:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:27:09.197666883 +0000 UTC m=+3275.163601116" watchObservedRunningTime="2026-07-16 21:27:09.199409027 +0000 UTC m=+3275.165343259"
Jul 16 21:27:15.496454 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:15.496419    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:27:17.993310 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:17.993275    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:17.993310 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:17.993318    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:18.005522 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:18.005495    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:18.239659 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:18.239635    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:27:18.927650 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:18.927617    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc" podUID="934a0a57-6757-4e18-9081-d3142cd68251" containerName="main" containerID="cri-o://d34ae44937a3e1306ea15aad3a0a1e5077c0416bfc370e88da02b3aee694ef10" gracePeriod=45
Jul 16 21:27:19.177431 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.177408    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:27:19.234601 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.234526    2576 generic.go:356] "Generic (PLEG): container finished" podID="934a0a57-6757-4e18-9081-d3142cd68251" containerID="d34ae44937a3e1306ea15aad3a0a1e5077c0416bfc370e88da02b3aee694ef10" exitCode=0
Jul 16 21:27:19.234601 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.234576    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"
Jul 16 21:27:19.234779 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.234613    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc" event={"ID":"934a0a57-6757-4e18-9081-d3142cd68251","Type":"ContainerDied","Data":"d34ae44937a3e1306ea15aad3a0a1e5077c0416bfc370e88da02b3aee694ef10"}
Jul 16 21:27:19.234779 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.234653    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc" event={"ID":"934a0a57-6757-4e18-9081-d3142cd68251","Type":"ContainerDied","Data":"7b17dce475eaf22b53e516eb40dfb94f81553ee5419916e514495ede7289fa7d"}
Jul 16 21:27:19.234779 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.234671    2576 scope.go:117] "RemoveContainer" containerID="d34ae44937a3e1306ea15aad3a0a1e5077c0416bfc370e88da02b3aee694ef10"
Jul 16 21:27:19.242985 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.242956    2576 scope.go:117] "RemoveContainer" containerID="d34ae44937a3e1306ea15aad3a0a1e5077c0416bfc370e88da02b3aee694ef10"
Jul 16 21:27:19.243242 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:27:19.243221    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d34ae44937a3e1306ea15aad3a0a1e5077c0416bfc370e88da02b3aee694ef10\": container with ID starting with d34ae44937a3e1306ea15aad3a0a1e5077c0416bfc370e88da02b3aee694ef10 not found: ID does not exist" containerID="d34ae44937a3e1306ea15aad3a0a1e5077c0416bfc370e88da02b3aee694ef10"
Jul 16 21:27:19.243307 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.243248    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d34ae44937a3e1306ea15aad3a0a1e5077c0416bfc370e88da02b3aee694ef10"} err="failed to get container status \"d34ae44937a3e1306ea15aad3a0a1e5077c0416bfc370e88da02b3aee694ef10\": rpc error: code = NotFound desc = could not find container \"d34ae44937a3e1306ea15aad3a0a1e5077c0416bfc370e88da02b3aee694ef10\": container with ID starting with d34ae44937a3e1306ea15aad3a0a1e5077c0416bfc370e88da02b3aee694ef10 not found: ID does not exist"
Jul 16 21:27:19.345166 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.345144    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-model-cache\") pod \"934a0a57-6757-4e18-9081-d3142cd68251\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") "
Jul 16 21:27:19.345271 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.345177    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6fd6\" (UniqueName: \"kubernetes.io/projected/934a0a57-6757-4e18-9081-d3142cd68251-kube-api-access-c6fd6\") pod \"934a0a57-6757-4e18-9081-d3142cd68251\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") "
Jul 16 21:27:19.345271 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.345222    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/934a0a57-6757-4e18-9081-d3142cd68251-tls-certs\") pod \"934a0a57-6757-4e18-9081-d3142cd68251\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") "
Jul 16 21:27:19.345271 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.345242    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-home\") pod \"934a0a57-6757-4e18-9081-d3142cd68251\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") "
Jul 16 21:27:19.345487 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.345350    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-tmp-dir\") pod \"934a0a57-6757-4e18-9081-d3142cd68251\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") "
Jul 16 21:27:19.345487 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.345466    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-model-cache" (OuterVolumeSpecName: "model-cache") pod "934a0a57-6757-4e18-9081-d3142cd68251" (UID: "934a0a57-6757-4e18-9081-d3142cd68251"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:27:19.345597 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.345475    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-dshm\") pod \"934a0a57-6757-4e18-9081-d3142cd68251\" (UID: \"934a0a57-6757-4e18-9081-d3142cd68251\") "
Jul 16 21:27:19.345597 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.345504    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-home" (OuterVolumeSpecName: "home") pod "934a0a57-6757-4e18-9081-d3142cd68251" (UID: "934a0a57-6757-4e18-9081-d3142cd68251"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:27:19.345685 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.345634    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "934a0a57-6757-4e18-9081-d3142cd68251" (UID: "934a0a57-6757-4e18-9081-d3142cd68251"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:27:19.345865 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.345846    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-home\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:27:19.346013 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.345992    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-tmp-dir\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:27:19.346105 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.346017    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-model-cache\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:27:19.347411 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.347383    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/934a0a57-6757-4e18-9081-d3142cd68251-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "934a0a57-6757-4e18-9081-d3142cd68251" (UID: "934a0a57-6757-4e18-9081-d3142cd68251"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:27:19.347497 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.347451    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-dshm" (OuterVolumeSpecName: "dshm") pod "934a0a57-6757-4e18-9081-d3142cd68251" (UID: "934a0a57-6757-4e18-9081-d3142cd68251"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:27:19.347687 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.347671    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/934a0a57-6757-4e18-9081-d3142cd68251-kube-api-access-c6fd6" (OuterVolumeSpecName: "kube-api-access-c6fd6") pod "934a0a57-6757-4e18-9081-d3142cd68251" (UID: "934a0a57-6757-4e18-9081-d3142cd68251"). InnerVolumeSpecName "kube-api-access-c6fd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:27:19.446599 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.446576    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6fd6\" (UniqueName: \"kubernetes.io/projected/934a0a57-6757-4e18-9081-d3142cd68251-kube-api-access-c6fd6\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:27:19.446599 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.446597    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/934a0a57-6757-4e18-9081-d3142cd68251-tls-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:27:19.446722 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.446608    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/934a0a57-6757-4e18-9081-d3142cd68251-dshm\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:27:19.558052 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.558033    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"]
Jul 16 21:27:19.561964 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:19.561946    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/sub-test-model-fb5e77-kserve-76b5958b79-2fqsc"]
Jul 16 21:27:20.689876 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:20.689844    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="934a0a57-6757-4e18-9081-d3142cd68251" path="/var/lib/kubelet/pods/934a0a57-6757-4e18-9081-d3142cd68251/volumes"
Jul 16 21:27:34.736221 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:34.736196    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 21:27:34.753275 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:34.753253    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 21:27:57.182156 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.182124    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"]
Jul 16 21:27:57.182556 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.182541    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="934a0a57-6757-4e18-9081-d3142cd68251" containerName="main"
Jul 16 21:27:57.182556 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.182554    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="934a0a57-6757-4e18-9081-d3142cd68251" containerName="main"
Jul 16 21:27:57.182631 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.182626    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="934a0a57-6757-4e18-9081-d3142cd68251" containerName="main"
Jul 16 21:27:57.185703 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.185682    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.188887 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.188852    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fb5e77\"/\"rate-test-model-fb5e77-kserve-self-signed-certs\""
Jul 16 21:27:57.189005 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.188856    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-fb5e77\"/\"default-dockercfg-m9vmz\""
Jul 16 21:27:57.189005 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.188937    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fb5e77\"/\"openshift-service-ca.crt\""
Jul 16 21:27:57.189005 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.188959    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-fb5e77\"/\"kube-root-ca.crt\""
Jul 16 21:27:57.197040 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.197016    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"]
Jul 16 21:27:57.238982 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.238962    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-dshm\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.239107 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.238995    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-tmp-dir\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.239107 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.239018    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-home\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.239228 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.239106    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-model-cache\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.239228 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.239136    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-tls-certs\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.239228 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.239190    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbcxp\" (UniqueName: \"kubernetes.io/projected/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-kube-api-access-hbcxp\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.340220 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.340107    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-dshm\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.340220 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.340166    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-tmp-dir\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.340220 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.340203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-home\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.340548 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.340239    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-model-cache\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.340548 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.340266    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-tls-certs\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.340548 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.340405    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbcxp\" (UniqueName: \"kubernetes.io/projected/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-kube-api-access-hbcxp\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.341798 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.341770    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-home\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.341967 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.341940    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-model-cache\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.342180 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.342157    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-tmp-dir\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.344626 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.344602    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-dshm\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.345098 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.345073    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-tls-certs\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.350236 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.350217    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbcxp\" (UniqueName: \"kubernetes.io/projected/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-kube-api-access-hbcxp\") pod \"rate-test-model-fb5e77-kserve-97487694c-ntq8z\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") " pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.497611 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.497545    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:27:57.625749 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:57.625718    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"]
Jul 16 21:27:57.628673 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:27:57.628644    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9363d141_9eb9_46c0_94e7_dd6e59f1f4e4.slice/crio-b1aedfb82dd54be0f9f27cb44061ed10b26ec25b1c246faddb825a5d417e44cd WatchSource:0}: Error finding container b1aedfb82dd54be0f9f27cb44061ed10b26ec25b1c246faddb825a5d417e44cd: Status 404 returned error can't find the container with id b1aedfb82dd54be0f9f27cb44061ed10b26ec25b1c246faddb825a5d417e44cd
Jul 16 21:27:58.439833 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:58.439803    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z" event={"ID":"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4","Type":"ContainerStarted","Data":"ec0baf7e2c8313d79a8a89a9e595da6cf706def170d4e6cabc1cf55ea5b6a14f"}
Jul 16 21:27:58.440191 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:58.439843    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z" event={"ID":"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4","Type":"ContainerStarted","Data":"b1aedfb82dd54be0f9f27cb44061ed10b26ec25b1c246faddb825a5d417e44cd"}
Jul 16 21:27:58.458034 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:27:58.457989    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z" podStartSLOduration=1.457971646 podStartE2EDuration="1.457971646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:27:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:27:58.456538081 +0000 UTC m=+3324.422472315" watchObservedRunningTime="2026-07-16 21:27:58.457971646 +0000 UTC m=+3324.423905879"
Jul 16 21:28:01.198960 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:01.198926    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:28:07.497722 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:07.497690    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:28:07.498312 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:07.497739    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:28:07.512057 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:07.512031    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:28:08.500166 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:08.500134    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:28:16.092018 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:16.091983    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:28:37.990470 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:37.990434    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:28:42.580805 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:42.580766    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"]
Jul 16 21:28:43.461239 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:43.461207    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"]
Jul 16 21:28:48.691813 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:48.691779    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:28:57.601582 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.601542    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr" podUID="f773a436-190a-435e-a88b-092cf87102b5" containerName="main" containerID="cri-o://54e8e0ad1ced30d362fccaf9fabd1004095dcb777b743584cd3d73a755c45701" gracePeriod=45
Jul 16 21:28:57.735236 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.735201    2576 generic.go:356] "Generic (PLEG): container finished" podID="f773a436-190a-435e-a88b-092cf87102b5" containerID="54e8e0ad1ced30d362fccaf9fabd1004095dcb777b743584cd3d73a755c45701" exitCode=0
Jul 16 21:28:57.735397 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.735270    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr" event={"ID":"f773a436-190a-435e-a88b-092cf87102b5","Type":"ContainerDied","Data":"54e8e0ad1ced30d362fccaf9fabd1004095dcb777b743584cd3d73a755c45701"}
Jul 16 21:28:57.850820 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.850798    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:28:57.946617 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.946593    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-model-cache\") pod \"f773a436-190a-435e-a88b-092cf87102b5\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") "
Jul 16 21:28:57.946750 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.946714    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f773a436-190a-435e-a88b-092cf87102b5-tls-certs\") pod \"f773a436-190a-435e-a88b-092cf87102b5\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") "
Jul 16 21:28:57.946803 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.946748    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-tmp-dir\") pod \"f773a436-190a-435e-a88b-092cf87102b5\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") "
Jul 16 21:28:57.946803 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.946782    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-dshm\") pod \"f773a436-190a-435e-a88b-092cf87102b5\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") "
Jul 16 21:28:57.946803 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.946789    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-model-cache" (OuterVolumeSpecName: "model-cache") pod "f773a436-190a-435e-a88b-092cf87102b5" (UID: "f773a436-190a-435e-a88b-092cf87102b5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:28:57.946962 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.946862    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksts6\" (UniqueName: \"kubernetes.io/projected/f773a436-190a-435e-a88b-092cf87102b5-kube-api-access-ksts6\") pod \"f773a436-190a-435e-a88b-092cf87102b5\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") "
Jul 16 21:28:57.946962 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.946883    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-home\") pod \"f773a436-190a-435e-a88b-092cf87102b5\" (UID: \"f773a436-190a-435e-a88b-092cf87102b5\") "
Jul 16 21:28:57.947073 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.947045    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f773a436-190a-435e-a88b-092cf87102b5" (UID: "f773a436-190a-435e-a88b-092cf87102b5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:28:57.947222 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.947175    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-model-cache\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:28:57.947222 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.947195    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-tmp-dir\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:28:57.947398 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.947263    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-home" (OuterVolumeSpecName: "home") pod "f773a436-190a-435e-a88b-092cf87102b5" (UID: "f773a436-190a-435e-a88b-092cf87102b5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:28:57.948943 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.948916    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f773a436-190a-435e-a88b-092cf87102b5-kube-api-access-ksts6" (OuterVolumeSpecName: "kube-api-access-ksts6") pod "f773a436-190a-435e-a88b-092cf87102b5" (UID: "f773a436-190a-435e-a88b-092cf87102b5"). InnerVolumeSpecName "kube-api-access-ksts6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:28:57.949061 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.949013    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-dshm" (OuterVolumeSpecName: "dshm") pod "f773a436-190a-435e-a88b-092cf87102b5" (UID: "f773a436-190a-435e-a88b-092cf87102b5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:28:57.949110 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:57.949078    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f773a436-190a-435e-a88b-092cf87102b5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f773a436-190a-435e-a88b-092cf87102b5" (UID: "f773a436-190a-435e-a88b-092cf87102b5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:28:58.047625 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.047601    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f773a436-190a-435e-a88b-092cf87102b5-tls-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:28:58.047625 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.047622    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-dshm\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:28:58.047758 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.047638    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksts6\" (UniqueName: \"kubernetes.io/projected/f773a436-190a-435e-a88b-092cf87102b5-kube-api-access-ksts6\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:28:58.047758 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.047649    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f773a436-190a-435e-a88b-092cf87102b5-home\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:28:58.481451 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.481388    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z" podUID="9363d141-9eb9-46c0-94e7-dd6e59f1f4e4" containerName="main" containerID="cri-o://ec0baf7e2c8313d79a8a89a9e595da6cf706def170d4e6cabc1cf55ea5b6a14f" gracePeriod=45
Jul 16 21:28:58.490114 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.490087    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z" podUID="9363d141-9eb9-46c0-94e7-dd6e59f1f4e4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": dial tcp 10.132.0.80:8000: connect: connection refused"
Jul 16 21:28:58.742065 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.742037    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"
Jul 16 21:28:58.742546 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.742079    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr" event={"ID":"f773a436-190a-435e-a88b-092cf87102b5","Type":"ContainerDied","Data":"96cd98d526f53a9774d045a0b06078cc0dc704f09437c110841f6e72dab7a1af"}
Jul 16 21:28:58.742546 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.742128    2576 scope.go:117] "RemoveContainer" containerID="54e8e0ad1ced30d362fccaf9fabd1004095dcb777b743584cd3d73a755c45701"
Jul 16 21:28:58.743792 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.743766    2576 generic.go:356] "Generic (PLEG): container finished" podID="9363d141-9eb9-46c0-94e7-dd6e59f1f4e4" containerID="ec0baf7e2c8313d79a8a89a9e595da6cf706def170d4e6cabc1cf55ea5b6a14f" exitCode=0
Jul 16 21:28:58.743904 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.743802    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z" event={"ID":"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4","Type":"ContainerDied","Data":"ec0baf7e2c8313d79a8a89a9e595da6cf706def170d4e6cabc1cf55ea5b6a14f"}
Jul 16 21:28:58.743904 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.743842    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z" event={"ID":"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4","Type":"ContainerDied","Data":"b1aedfb82dd54be0f9f27cb44061ed10b26ec25b1c246faddb825a5d417e44cd"}
Jul 16 21:28:58.743904 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.743860    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1aedfb82dd54be0f9f27cb44061ed10b26ec25b1c246faddb825a5d417e44cd"
Jul 16 21:28:58.752529 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.752500    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:28:58.763144 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.763114    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"]
Jul 16 21:28:58.766982 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.766960    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-9331e3/rate-test-model-9331e3-kserve-685fb7b87f-z7dcr"]
Jul 16 21:28:58.855191 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.855165    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-tmp-dir\") pod \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") "
Jul 16 21:28:58.855322 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.855244    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-tls-certs\") pod \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") "
Jul 16 21:28:58.855322 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.855271    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-home\") pod \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") "
Jul 16 21:28:58.855458 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.855346    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbcxp\" (UniqueName: \"kubernetes.io/projected/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-kube-api-access-hbcxp\") pod \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") "
Jul 16 21:28:58.855458 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.855435    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-dshm\") pod \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") "
Jul 16 21:28:58.855458 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.855443    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9363d141-9eb9-46c0-94e7-dd6e59f1f4e4" (UID: "9363d141-9eb9-46c0-94e7-dd6e59f1f4e4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:28:58.855617 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.855453    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-model-cache\") pod \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\" (UID: \"9363d141-9eb9-46c0-94e7-dd6e59f1f4e4\") "
Jul 16 21:28:58.855673 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.855617    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-home" (OuterVolumeSpecName: "home") pod "9363d141-9eb9-46c0-94e7-dd6e59f1f4e4" (UID: "9363d141-9eb9-46c0-94e7-dd6e59f1f4e4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:28:58.855673 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.855658    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-model-cache" (OuterVolumeSpecName: "model-cache") pod "9363d141-9eb9-46c0-94e7-dd6e59f1f4e4" (UID: "9363d141-9eb9-46c0-94e7-dd6e59f1f4e4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:28:58.855850 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.855832    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-model-cache\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:28:58.855906 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.855854    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-tmp-dir\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:28:58.855906 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.855868    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-home\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:28:58.857309 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.857289    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9363d141-9eb9-46c0-94e7-dd6e59f1f4e4" (UID: "9363d141-9eb9-46c0-94e7-dd6e59f1f4e4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 16 21:28:58.857857 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.857836    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-dshm" (OuterVolumeSpecName: "dshm") pod "9363d141-9eb9-46c0-94e7-dd6e59f1f4e4" (UID: "9363d141-9eb9-46c0-94e7-dd6e59f1f4e4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:28:58.857926 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.857853    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-kube-api-access-hbcxp" (OuterVolumeSpecName: "kube-api-access-hbcxp") pod "9363d141-9eb9-46c0-94e7-dd6e59f1f4e4" (UID: "9363d141-9eb9-46c0-94e7-dd6e59f1f4e4"). InnerVolumeSpecName "kube-api-access-hbcxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:28:58.956947 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.956921    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-tls-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:28:58.956947 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.956943    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbcxp\" (UniqueName: \"kubernetes.io/projected/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-kube-api-access-hbcxp\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:28:58.957086 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:58.956952    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4-dshm\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:28:59.750444 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:59.750417    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"
Jul 16 21:28:59.771572 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:59.771544    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"]
Jul 16 21:28:59.774998 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:28:59.774974    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-fb5e77/rate-test-model-fb5e77-kserve-97487694c-ntq8z"]
Jul 16 21:29:00.690260 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:00.690217    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9363d141-9eb9-46c0-94e7-dd6e59f1f4e4" path="/var/lib/kubelet/pods/9363d141-9eb9-46c0-94e7-dd6e59f1f4e4/volumes"
Jul 16 21:29:00.690671 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:00.690653    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f773a436-190a-435e-a88b-092cf87102b5" path="/var/lib/kubelet/pods/f773a436-190a-435e-a88b-092cf87102b5/volumes"
Jul 16 21:29:16.907855 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:16.907819    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:29:28.844918 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:28.844833    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"]
Jul 16 21:29:28.845381 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:28.845080    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp" podUID="19df913d-1cd0-480f-aeec-77c1bbecbe66" containerName="istio-proxy" containerID="cri-o://082d256dfe2f7e062c30e01e7c1df4175ae21c848ff0e3bb44719b17e53c5255" gracePeriod=30
Jul 16 21:29:29.934882 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:29.934846    2576 patch_prober.go:28] interesting pod/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 16 21:29:29.935277 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:29.934914    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp" podUID="19df913d-1cd0-480f-aeec-77c1bbecbe66" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 16 21:29:34.102248 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.102218    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:29:34.228411 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.228351    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfdx5\" (UniqueName: \"kubernetes.io/projected/19df913d-1cd0-480f-aeec-77c1bbecbe66-kube-api-access-pfdx5\") pod \"19df913d-1cd0-480f-aeec-77c1bbecbe66\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") "
Jul 16 21:29:34.228572 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.228452    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-data\") pod \"19df913d-1cd0-480f-aeec-77c1bbecbe66\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") "
Jul 16 21:29:34.228572 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.228510    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19df913d-1cd0-480f-aeec-77c1bbecbe66-istiod-ca-cert\") pod \"19df913d-1cd0-480f-aeec-77c1bbecbe66\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") "
Jul 16 21:29:34.228572 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.228544    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-podinfo\") pod \"19df913d-1cd0-480f-aeec-77c1bbecbe66\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") "
Jul 16 21:29:34.228747 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.228574    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-token\") pod \"19df913d-1cd0-480f-aeec-77c1bbecbe66\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") "
Jul 16 21:29:34.228747 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.228603    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-credential-socket\") pod \"19df913d-1cd0-480f-aeec-77c1bbecbe66\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") "
Jul 16 21:29:34.228747 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.228653    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-envoy\") pod \"19df913d-1cd0-480f-aeec-77c1bbecbe66\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") "
Jul 16 21:29:34.228747 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.228685    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-workload-certs\") pod \"19df913d-1cd0-480f-aeec-77c1bbecbe66\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") "
Jul 16 21:29:34.228747 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.228742    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-workload-socket\") pod \"19df913d-1cd0-480f-aeec-77c1bbecbe66\" (UID: \"19df913d-1cd0-480f-aeec-77c1bbecbe66\") "
Jul 16 21:29:34.229012 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.228921    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19df913d-1cd0-480f-aeec-77c1bbecbe66-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "19df913d-1cd0-480f-aeec-77c1bbecbe66" (UID: "19df913d-1cd0-480f-aeec-77c1bbecbe66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:29:34.229242 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.229189    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "19df913d-1cd0-480f-aeec-77c1bbecbe66" (UID: "19df913d-1cd0-480f-aeec-77c1bbecbe66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:29:34.229242 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.229106    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19df913d-1cd0-480f-aeec-77c1bbecbe66-istiod-ca-cert\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:29:34.229242 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.229210    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "19df913d-1cd0-480f-aeec-77c1bbecbe66" (UID: "19df913d-1cd0-480f-aeec-77c1bbecbe66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:29:34.229559 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.229400    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "19df913d-1cd0-480f-aeec-77c1bbecbe66" (UID: "19df913d-1cd0-480f-aeec-77c1bbecbe66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:29:34.230123 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.230095    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-data" (OuterVolumeSpecName: "istio-data") pod "19df913d-1cd0-480f-aeec-77c1bbecbe66" (UID: "19df913d-1cd0-480f-aeec-77c1bbecbe66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:29:34.230996 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.230969    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "19df913d-1cd0-480f-aeec-77c1bbecbe66" (UID: "19df913d-1cd0-480f-aeec-77c1bbecbe66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 21:29:34.231170 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.231147    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19df913d-1cd0-480f-aeec-77c1bbecbe66-kube-api-access-pfdx5" (OuterVolumeSpecName: "kube-api-access-pfdx5") pod "19df913d-1cd0-480f-aeec-77c1bbecbe66" (UID: "19df913d-1cd0-480f-aeec-77c1bbecbe66"). InnerVolumeSpecName "kube-api-access-pfdx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:29:34.231332 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.231305    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-token" (OuterVolumeSpecName: "istio-token") pod "19df913d-1cd0-480f-aeec-77c1bbecbe66" (UID: "19df913d-1cd0-480f-aeec-77c1bbecbe66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:29:34.231649 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.231616    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "19df913d-1cd0-480f-aeec-77c1bbecbe66" (UID: "19df913d-1cd0-480f-aeec-77c1bbecbe66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:29:34.330443 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.330415    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-envoy\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:29:34.330443 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.330442    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-workload-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:29:34.330443 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.330452    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-workload-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:29:34.330601 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.330461    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfdx5\" (UniqueName: \"kubernetes.io/projected/19df913d-1cd0-480f-aeec-77c1bbecbe66-kube-api-access-pfdx5\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:29:34.330601 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.330470    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-data\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:29:34.330601 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.330479    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-podinfo\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:29:34.330601 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.330486    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19df913d-1cd0-480f-aeec-77c1bbecbe66-istio-token\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:29:34.330601 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.330494    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19df913d-1cd0-480f-aeec-77c1bbecbe66-credential-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:29:34.935034 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.935003    2576 generic.go:356] "Generic (PLEG): container finished" podID="19df913d-1cd0-480f-aeec-77c1bbecbe66" containerID="082d256dfe2f7e062c30e01e7c1df4175ae21c848ff0e3bb44719b17e53c5255" exitCode=0
Jul 16 21:29:34.935155 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.935038    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp" event={"ID":"19df913d-1cd0-480f-aeec-77c1bbecbe66","Type":"ContainerDied","Data":"082d256dfe2f7e062c30e01e7c1df4175ae21c848ff0e3bb44719b17e53c5255"}
Jul 16 21:29:34.935155 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.935073    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"
Jul 16 21:29:34.935155 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.935093    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp" event={"ID":"19df913d-1cd0-480f-aeec-77c1bbecbe66","Type":"ContainerDied","Data":"ca33c2baa971130eccb0c0fe499ab5f9284764345fd7be9e091da55df1065ecd"}
Jul 16 21:29:34.935155 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.935120    2576 scope.go:117] "RemoveContainer" containerID="082d256dfe2f7e062c30e01e7c1df4175ae21c848ff0e3bb44719b17e53c5255"
Jul 16 21:29:34.944579 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.944561    2576 scope.go:117] "RemoveContainer" containerID="f36bf619323690be6bb3ecd469af0825d96da365b9496c27fe87a014ef34ffd8"
Jul 16 21:29:34.952779 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.952730    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"]
Jul 16 21:29:34.952983 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.952963    2576 scope.go:117] "RemoveContainer" containerID="082d256dfe2f7e062c30e01e7c1df4175ae21c848ff0e3bb44719b17e53c5255"
Jul 16 21:29:34.953237 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:29:34.953218    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"082d256dfe2f7e062c30e01e7c1df4175ae21c848ff0e3bb44719b17e53c5255\": container with ID starting with 082d256dfe2f7e062c30e01e7c1df4175ae21c848ff0e3bb44719b17e53c5255 not found: ID does not exist" containerID="082d256dfe2f7e062c30e01e7c1df4175ae21c848ff0e3bb44719b17e53c5255"
Jul 16 21:29:34.953294 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.953245    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"082d256dfe2f7e062c30e01e7c1df4175ae21c848ff0e3bb44719b17e53c5255"} err="failed to get container status \"082d256dfe2f7e062c30e01e7c1df4175ae21c848ff0e3bb44719b17e53c5255\": rpc error: code = NotFound desc = could not find container \"082d256dfe2f7e062c30e01e7c1df4175ae21c848ff0e3bb44719b17e53c5255\": container with ID starting with 082d256dfe2f7e062c30e01e7c1df4175ae21c848ff0e3bb44719b17e53c5255 not found: ID does not exist"
Jul 16 21:29:34.953294 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.953262    2576 scope.go:117] "RemoveContainer" containerID="f36bf619323690be6bb3ecd469af0825d96da365b9496c27fe87a014ef34ffd8"
Jul 16 21:29:34.953528 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:29:34.953511    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f36bf619323690be6bb3ecd469af0825d96da365b9496c27fe87a014ef34ffd8\": container with ID starting with f36bf619323690be6bb3ecd469af0825d96da365b9496c27fe87a014ef34ffd8 not found: ID does not exist" containerID="f36bf619323690be6bb3ecd469af0825d96da365b9496c27fe87a014ef34ffd8"
Jul 16 21:29:34.953634 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.953531    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f36bf619323690be6bb3ecd469af0825d96da365b9496c27fe87a014ef34ffd8"} err="failed to get container status \"f36bf619323690be6bb3ecd469af0825d96da365b9496c27fe87a014ef34ffd8\": rpc error: code = NotFound desc = could not find container \"f36bf619323690be6bb3ecd469af0825d96da365b9496c27fe87a014ef34ffd8\": container with ID starting with f36bf619323690be6bb3ecd469af0825d96da365b9496c27fe87a014ef34ffd8 not found: ID does not exist"
Jul 16 21:29:34.958506 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:34.958484    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-9331e3-openshift-default-5f9bb7b55f-xf6vp"]
Jul 16 21:29:36.690337 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:36.690300    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19df913d-1cd0-480f-aeec-77c1bbecbe66" path="/var/lib/kubelet/pods/19df913d-1cd0-480f-aeec-77c1bbecbe66/volumes"
Jul 16 21:29:42.593412 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:42.593376    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-djq6h"]
Jul 16 21:29:54.756295 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.756264    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s"]
Jul 16 21:29:54.756813 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.756617    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f773a436-190a-435e-a88b-092cf87102b5" containerName="main"
Jul 16 21:29:54.756813 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.756636    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f773a436-190a-435e-a88b-092cf87102b5" containerName="main"
Jul 16 21:29:54.756813 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.756653    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19df913d-1cd0-480f-aeec-77c1bbecbe66" containerName="istio-proxy"
Jul 16 21:29:54.756813 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.756658    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="19df913d-1cd0-480f-aeec-77c1bbecbe66" containerName="istio-proxy"
Jul 16 21:29:54.756813 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.756665    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9363d141-9eb9-46c0-94e7-dd6e59f1f4e4" containerName="main"
Jul 16 21:29:54.756813 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.756671    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9363d141-9eb9-46c0-94e7-dd6e59f1f4e4" containerName="main"
Jul 16 21:29:54.756813 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.756679    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19df913d-1cd0-480f-aeec-77c1bbecbe66" containerName="istio-proxy"
Jul 16 21:29:54.756813 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.756684    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="19df913d-1cd0-480f-aeec-77c1bbecbe66" containerName="istio-proxy"
Jul 16 21:29:54.756813 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.756737    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="19df913d-1cd0-480f-aeec-77c1bbecbe66" containerName="istio-proxy"
Jul 16 21:29:54.756813 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.756745    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f773a436-190a-435e-a88b-092cf87102b5" containerName="main"
Jul 16 21:29:54.756813 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.756752    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="19df913d-1cd0-480f-aeec-77c1bbecbe66" containerName="istio-proxy"
Jul 16 21:29:54.756813 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.756758    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9363d141-9eb9-46c0-94e7-dd6e59f1f4e4" containerName="main"
Jul 16 21:29:54.759641 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.759626    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s"
Jul 16 21:29:54.762121 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.762097    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-8hb99\""
Jul 16 21:29:54.762249 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.762102    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 16 21:29:54.766995 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.766781    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s"]
Jul 16 21:29:54.897004 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.896976    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4930d069-009f-45c8-8c97-97b76c099088-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s\" (UID: \"4930d069-009f-45c8-8c97-97b76c099088\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s"
Jul 16 21:29:54.998510 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.998478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4930d069-009f-45c8-8c97-97b76c099088-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s\" (UID: \"4930d069-009f-45c8-8c97-97b76c099088\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s"
Jul 16 21:29:54.999349 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:54.999297    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4930d069-009f-45c8-8c97-97b76c099088-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s\" (UID: \"4930d069-009f-45c8-8c97-97b76c099088\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s"
Jul 16 21:29:55.002159 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:55.002137    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"]
Jul 16 21:29:55.002341 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:55.002321    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9" podUID="4fe4d706-6abd-4aea-9663-36c2f91bf3c6" containerName="istio-proxy" containerID="cri-o://8638bf493d0298f13a8403545fa9ece937e3421754ed48a632cfc79645515a4a" gracePeriod=30
Jul 16 21:29:55.071648 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:55.071591    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s"
Jul 16 21:29:55.209938 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:55.209912    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s"]
Jul 16 21:29:55.211752 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:29:55.211725    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4930d069_009f_45c8_8c97_97b76c099088.slice/crio-268ba70c7d90c5b194603172a895b563ca1a5b93fe033967addfb48f863ea0de WatchSource:0}: Error finding container 268ba70c7d90c5b194603172a895b563ca1a5b93fe033967addfb48f863ea0de: Status 404 returned error can't find the container with id 268ba70c7d90c5b194603172a895b563ca1a5b93fe033967addfb48f863ea0de
Jul 16 21:29:56.038976 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:56.038948    2576 generic.go:356] "Generic (PLEG): container finished" podID="4930d069-009f-45c8-8c97-97b76c099088" containerID="43caa4ee570f0c0060df2586215cf244aef8ed5a79d9fcb8b0829636e646cd0e" exitCode=6
Jul 16 21:29:56.039388 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:56.039032    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s" event={"ID":"4930d069-009f-45c8-8c97-97b76c099088","Type":"ContainerDied","Data":"43caa4ee570f0c0060df2586215cf244aef8ed5a79d9fcb8b0829636e646cd0e"}
Jul 16 21:29:56.039388 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:56.039105    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s" event={"ID":"4930d069-009f-45c8-8c97-97b76c099088","Type":"ContainerStarted","Data":"268ba70c7d90c5b194603172a895b563ca1a5b93fe033967addfb48f863ea0de"}
Jul 16 21:29:56.039388 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:56.039256    2576 scope.go:117] "RemoveContainer" containerID="43caa4ee570f0c0060df2586215cf244aef8ed5a79d9fcb8b0829636e646cd0e"
Jul 16 21:29:57.046030 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:57.046000    2576 generic.go:356] "Generic (PLEG): container finished" podID="4930d069-009f-45c8-8c97-97b76c099088" containerID="3dc4f39d4b58c44ca7235a37fab047afd02c79fbddff9126ff1d04b805d753e6" exitCode=6
Jul 16 21:29:57.046436 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:57.046088    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s" event={"ID":"4930d069-009f-45c8-8c97-97b76c099088","Type":"ContainerDied","Data":"3dc4f39d4b58c44ca7235a37fab047afd02c79fbddff9126ff1d04b805d753e6"}
Jul 16 21:29:57.046436 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:57.046141    2576 scope.go:117] "RemoveContainer" containerID="43caa4ee570f0c0060df2586215cf244aef8ed5a79d9fcb8b0829636e646cd0e"
Jul 16 21:29:57.046436 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:57.046242    2576 scope.go:117] "RemoveContainer" containerID="3dc4f39d4b58c44ca7235a37fab047afd02c79fbddff9126ff1d04b805d753e6"
Jul 16 21:29:57.046592 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:29:57.046486    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s_odh-ai-gateway-infra(4930d069-009f-45c8-8c97-97b76c099088)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s" podUID="4930d069-009f-45c8-8c97-97b76c099088"
Jul 16 21:29:58.052962 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:29:58.052932    2576 scope.go:117] "RemoveContainer" containerID="3dc4f39d4b58c44ca7235a37fab047afd02c79fbddff9126ff1d04b805d753e6"
Jul 16 21:29:58.053426 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:29:58.053143    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s_odh-ai-gateway-infra(4930d069-009f-45c8-8c97-97b76c099088)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s" podUID="4930d069-009f-45c8-8c97-97b76c099088"
Jul 16 21:30:00.262451 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.262393    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:30:00.349543 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.349519    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-envoy\") pod \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") "
Jul 16 21:30:00.349677 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.349558    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-workload-socket\") pod \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") "
Jul 16 21:30:00.349677 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.349576    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-data\") pod \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") "
Jul 16 21:30:00.349677 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.349599    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-podinfo\") pod \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") "
Jul 16 21:30:00.349677 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.349625    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-workload-certs\") pod \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") "
Jul 16 21:30:00.349677 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.349648    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89h6w\" (UniqueName: \"kubernetes.io/projected/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-kube-api-access-89h6w\") pod \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") "
Jul 16 21:30:00.349940 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.349688    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-credential-socket\") pod \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") "
Jul 16 21:30:00.349940 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.349707    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istiod-ca-cert\") pod \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") "
Jul 16 21:30:00.349940 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.349741    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-token\") pod \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\" (UID: \"4fe4d706-6abd-4aea-9663-36c2f91bf3c6\") "
Jul 16 21:30:00.349940 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.349813    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4fe4d706-6abd-4aea-9663-36c2f91bf3c6" (UID: "4fe4d706-6abd-4aea-9663-36c2f91bf3c6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:30:00.350149 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.349966    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4fe4d706-6abd-4aea-9663-36c2f91bf3c6" (UID: "4fe4d706-6abd-4aea-9663-36c2f91bf3c6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:30:00.350149 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.350047    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-credential-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:30:00.350149 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.350062    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-workload-socket\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:30:00.350149 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.350125    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4fe4d706-6abd-4aea-9663-36c2f91bf3c6" (UID: "4fe4d706-6abd-4aea-9663-36c2f91bf3c6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:30:00.350394 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.350215    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4fe4d706-6abd-4aea-9663-36c2f91bf3c6" (UID: "4fe4d706-6abd-4aea-9663-36c2f91bf3c6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:30:00.350440 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.350407    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-data" (OuterVolumeSpecName: "istio-data") pod "4fe4d706-6abd-4aea-9663-36c2f91bf3c6" (UID: "4fe4d706-6abd-4aea-9663-36c2f91bf3c6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:30:00.351742 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.351714    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4fe4d706-6abd-4aea-9663-36c2f91bf3c6" (UID: "4fe4d706-6abd-4aea-9663-36c2f91bf3c6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 16 21:30:00.352035 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.352011    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-kube-api-access-89h6w" (OuterVolumeSpecName: "kube-api-access-89h6w") pod "4fe4d706-6abd-4aea-9663-36c2f91bf3c6" (UID: "4fe4d706-6abd-4aea-9663-36c2f91bf3c6"). InnerVolumeSpecName "kube-api-access-89h6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:30:00.352473 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.352452    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4fe4d706-6abd-4aea-9663-36c2f91bf3c6" (UID: "4fe4d706-6abd-4aea-9663-36c2f91bf3c6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 16 21:30:00.352555 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.352457    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-token" (OuterVolumeSpecName: "istio-token") pod "4fe4d706-6abd-4aea-9663-36c2f91bf3c6" (UID: "4fe4d706-6abd-4aea-9663-36c2f91bf3c6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 16 21:30:00.451414 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.451394    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-envoy\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:30:00.451414 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.451413    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-data\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:30:00.451546 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.451421    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-podinfo\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:30:00.451546 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.451430    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-workload-certs\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:30:00.451546 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.451438    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89h6w\" (UniqueName: \"kubernetes.io/projected/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-kube-api-access-89h6w\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:30:00.451546 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.451447    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istiod-ca-cert\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:30:00.451546 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:00.451454    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fe4d706-6abd-4aea-9663-36c2f91bf3c6-istio-token\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:30:01.069790 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:01.069727    2576 generic.go:356] "Generic (PLEG): container finished" podID="4fe4d706-6abd-4aea-9663-36c2f91bf3c6" containerID="8638bf493d0298f13a8403545fa9ece937e3421754ed48a632cfc79645515a4a" exitCode=0
Jul 16 21:30:01.069790 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:01.069784    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"
Jul 16 21:30:01.069972 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:01.069808    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9" event={"ID":"4fe4d706-6abd-4aea-9663-36c2f91bf3c6","Type":"ContainerDied","Data":"8638bf493d0298f13a8403545fa9ece937e3421754ed48a632cfc79645515a4a"}
Jul 16 21:30:01.069972 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:01.069857    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9" event={"ID":"4fe4d706-6abd-4aea-9663-36c2f91bf3c6","Type":"ContainerDied","Data":"e61b29c68ef4bf402943c72706b2807f1c1a909241cedc45046ae49633e2237c"}
Jul 16 21:30:01.069972 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:01.069883    2576 scope.go:117] "RemoveContainer" containerID="8638bf493d0298f13a8403545fa9ece937e3421754ed48a632cfc79645515a4a"
Jul 16 21:30:01.078889 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:01.078866    2576 scope.go:117] "RemoveContainer" containerID="1eef2bde8652a401edbeca8b7861ab7e2a881159feafc7fd77ae56873026698c"
Jul 16 21:30:01.087299 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:01.087283    2576 scope.go:117] "RemoveContainer" containerID="8638bf493d0298f13a8403545fa9ece937e3421754ed48a632cfc79645515a4a"
Jul 16 21:30:01.087582 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:30:01.087565    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8638bf493d0298f13a8403545fa9ece937e3421754ed48a632cfc79645515a4a\": container with ID starting with 8638bf493d0298f13a8403545fa9ece937e3421754ed48a632cfc79645515a4a not found: ID does not exist" containerID="8638bf493d0298f13a8403545fa9ece937e3421754ed48a632cfc79645515a4a"
Jul 16 21:30:01.087647 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:01.087588    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8638bf493d0298f13a8403545fa9ece937e3421754ed48a632cfc79645515a4a"} err="failed to get container status \"8638bf493d0298f13a8403545fa9ece937e3421754ed48a632cfc79645515a4a\": rpc error: code = NotFound desc = could not find container \"8638bf493d0298f13a8403545fa9ece937e3421754ed48a632cfc79645515a4a\": container with ID starting with 8638bf493d0298f13a8403545fa9ece937e3421754ed48a632cfc79645515a4a not found: ID does not exist"
Jul 16 21:30:01.087647 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:01.087602    2576 scope.go:117] "RemoveContainer" containerID="1eef2bde8652a401edbeca8b7861ab7e2a881159feafc7fd77ae56873026698c"
Jul 16 21:30:01.087824 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:30:01.087804    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eef2bde8652a401edbeca8b7861ab7e2a881159feafc7fd77ae56873026698c\": container with ID starting with 1eef2bde8652a401edbeca8b7861ab7e2a881159feafc7fd77ae56873026698c not found: ID does not exist" containerID="1eef2bde8652a401edbeca8b7861ab7e2a881159feafc7fd77ae56873026698c"
Jul 16 21:30:01.087868 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:01.087831    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eef2bde8652a401edbeca8b7861ab7e2a881159feafc7fd77ae56873026698c"} err="failed to get container status \"1eef2bde8652a401edbeca8b7861ab7e2a881159feafc7fd77ae56873026698c\": rpc error: code = NotFound desc = could not find container \"1eef2bde8652a401edbeca8b7861ab7e2a881159feafc7fd77ae56873026698c\": container with ID starting with 1eef2bde8652a401edbeca8b7861ab7e2a881159feafc7fd77ae56873026698c not found: ID does not exist"
Jul 16 21:30:01.101203 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:01.101181    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"]
Jul 16 21:30:01.106387 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:01.106351    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-fb5e77-openshift-default-f8f86487-z4qh9"]
Jul 16 21:30:01.734132 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:01.734107    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mhswl_993d7e8c-d616-4f37-b49d-1e6a7f4b44ba/server/0.log"
Jul 16 21:30:01.856797 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:01.856778    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xz7gs_3d0e5db1-b544-4e53-ae7b-9e798d79581d/manager/0.log"
Jul 16 21:30:02.100355 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:02.100293    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58fdbbf655-jpk7x_23383b4a-08bf-4e83-b4f7-13443a9e940e/copy-manifests/0.log"
Jul 16 21:30:02.107586 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:02.107553    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58fdbbf655-jpk7x_23383b4a-08bf-4e83-b4f7-13443a9e940e/manager/0.log"
Jul 16 21:30:02.690100 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:02.690068    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fe4d706-6abd-4aea-9663-36c2f91bf3c6" path="/var/lib/kubelet/pods/4fe4d706-6abd-4aea-9663-36c2f91bf3c6/volumes"
Jul 16 21:30:02.858141 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:02.858119    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737260-nrvbs_71d2cccc-202a-4177-b416-bbc58fa05427/cleanup/0.log"
Jul 16 21:30:02.974202 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:02.974143    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29737275-dqqm7_e400487d-b619-4469-b432-47a563cff330/cleanup/0.log"
Jul 16 21:30:03.205675 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:03.205646    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s_4930d069-009f-45c8-8c97-97b76c099088/revoke-keys/1.log"
Jul 16 21:30:05.845448 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:05.845412    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-xlskn_cb5301bc-a55e-45a7-a752-20fe36cad4fe/manager/0.log"
Jul 16 21:30:05.960048 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:05.960006    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-p6ldw_e5b3f53c-d73e-45dd-abfd-b17dbad05a47/manager/0.log"
Jul 16 21:30:06.199000 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:06.198959    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2dw5w_450291ef-6f9f-4952-b4d0-be1d00f94da7/registry-server/0.log"
Jul 16 21:30:06.322711 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:06.322665    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-4k4fb_a953200c-dcb6-4c55-bafd-d6ac67dfadd6/manager/0.log"
Jul 16 21:30:06.447051 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:06.447030    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-djq6h_9a11f252-e376-4afb-a7a2-0327e64e8628/limitador/0.log"
Jul 16 21:30:06.931867 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:06.931844    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mlcjj_acd59d37-8c14-4b5e-97c3-00774e15dbe2/istio-proxy/0.log"
Jul 16 21:30:07.413811 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:07.413785    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rdg46_e041dc27-7c87-4f74-902c-10c2393683bd/istio-proxy/1.log"
Jul 16 21:30:07.777300 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:07.777232    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65f496d8cd-qchkt_cbb37170-322b-4128-8731-ad11b8117cdc/router/0.log"
Jul 16 21:30:08.756976 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:08.756938    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-cnr9w_24568b9e-97df-4ce0-9108-cfc50b4f67b2/main/0.log"
Jul 16 21:30:12.684035 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:12.684006    2576 scope.go:117] "RemoveContainer" containerID="3dc4f39d4b58c44ca7235a37fab047afd02c79fbddff9126ff1d04b805d753e6"
Jul 16 21:30:13.137265 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:13.137201    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s" event={"ID":"4930d069-009f-45c8-8c97-97b76c099088","Type":"ContainerStarted","Data":"84bbfb49417c647faf72d0c4794b2d4b9855adbb4a5bc4f937c503dc4049af8c"}
Jul 16 21:30:13.153409 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:13.153346    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s" podStartSLOduration=19.153332549 podStartE2EDuration="19.153332549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:29:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:30:13.151107408 +0000 UTC m=+3459.117041642" watchObservedRunningTime="2026-07-16 21:30:13.153332549 +0000 UTC m=+3459.119266782"
Jul 16 21:30:14.145317 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.145286    2576 generic.go:356] "Generic (PLEG): container finished" podID="4930d069-009f-45c8-8c97-97b76c099088" containerID="84bbfb49417c647faf72d0c4794b2d4b9855adbb4a5bc4f937c503dc4049af8c" exitCode=6
Jul 16 21:30:14.145755 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.145382    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s" event={"ID":"4930d069-009f-45c8-8c97-97b76c099088","Type":"ContainerDied","Data":"84bbfb49417c647faf72d0c4794b2d4b9855adbb4a5bc4f937c503dc4049af8c"}
Jul 16 21:30:14.145755 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.145463    2576 scope.go:117] "RemoveContainer" containerID="3dc4f39d4b58c44ca7235a37fab047afd02c79fbddff9126ff1d04b805d753e6"
Jul 16 21:30:14.145755 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.145689    2576 scope.go:117] "RemoveContainer" containerID="84bbfb49417c647faf72d0c4794b2d4b9855adbb4a5bc4f937c503dc4049af8c"
Jul 16 21:30:14.145915 ip-10-0-143-245 kubenswrapper[2576]: E0716 21:30:14.145895    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s_odh-ai-gateway-infra(4930d069-009f-45c8-8c97-97b76c099088)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s" podUID="4930d069-009f-45c8-8c97-97b76c099088"
Jul 16 21:30:14.162125 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.162097    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s"]
Jul 16 21:30:14.638765 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.638694    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7plfz/must-gather-kkqsn"]
Jul 16 21:30:14.639062 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.639050    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fe4d706-6abd-4aea-9663-36c2f91bf3c6" containerName="istio-proxy"
Jul 16 21:30:14.639105 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.639064    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fe4d706-6abd-4aea-9663-36c2f91bf3c6" containerName="istio-proxy"
Jul 16 21:30:14.639105 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.639078    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fe4d706-6abd-4aea-9663-36c2f91bf3c6" containerName="istio-proxy"
Jul 16 21:30:14.639105 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.639083    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fe4d706-6abd-4aea-9663-36c2f91bf3c6" containerName="istio-proxy"
Jul 16 21:30:14.639799 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.639773    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fe4d706-6abd-4aea-9663-36c2f91bf3c6" containerName="istio-proxy"
Jul 16 21:30:14.639914 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.639814    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fe4d706-6abd-4aea-9663-36c2f91bf3c6" containerName="istio-proxy"
Jul 16 21:30:14.646873 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.646850    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7plfz/must-gather-kkqsn"
Jul 16 21:30:14.649180 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.649162    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7plfz\"/\"kube-root-ca.crt\""
Jul 16 21:30:14.649303 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.649266    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7plfz\"/\"openshift-service-ca.crt\""
Jul 16 21:30:14.650481 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.650463    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7plfz/must-gather-kkqsn"]
Jul 16 21:30:14.666890 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.666865    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1598cf50-ca16-4712-b894-7fd9386e84cf-must-gather-output\") pod \"must-gather-kkqsn\" (UID: \"1598cf50-ca16-4712-b894-7fd9386e84cf\") " pod="openshift-must-gather-7plfz/must-gather-kkqsn"
Jul 16 21:30:14.666978 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.666914    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmwqt\" (UniqueName: \"kubernetes.io/projected/1598cf50-ca16-4712-b894-7fd9386e84cf-kube-api-access-qmwqt\") pod \"must-gather-kkqsn\" (UID: \"1598cf50-ca16-4712-b894-7fd9386e84cf\") " pod="openshift-must-gather-7plfz/must-gather-kkqsn"
Jul 16 21:30:14.767860 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.767838    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmwqt\" (UniqueName: \"kubernetes.io/projected/1598cf50-ca16-4712-b894-7fd9386e84cf-kube-api-access-qmwqt\") pod \"must-gather-kkqsn\" (UID: \"1598cf50-ca16-4712-b894-7fd9386e84cf\") " pod="openshift-must-gather-7plfz/must-gather-kkqsn"
Jul 16 21:30:14.767968 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.767895    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1598cf50-ca16-4712-b894-7fd9386e84cf-must-gather-output\") pod \"must-gather-kkqsn\" (UID: \"1598cf50-ca16-4712-b894-7fd9386e84cf\") " pod="openshift-must-gather-7plfz/must-gather-kkqsn"
Jul 16 21:30:14.768176 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.768159    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1598cf50-ca16-4712-b894-7fd9386e84cf-must-gather-output\") pod \"must-gather-kkqsn\" (UID: \"1598cf50-ca16-4712-b894-7fd9386e84cf\") " pod="openshift-must-gather-7plfz/must-gather-kkqsn"
Jul 16 21:30:14.776755 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.776725    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmwqt\" (UniqueName: \"kubernetes.io/projected/1598cf50-ca16-4712-b894-7fd9386e84cf-kube-api-access-qmwqt\") pod \"must-gather-kkqsn\" (UID: \"1598cf50-ca16-4712-b894-7fd9386e84cf\") " pod="openshift-must-gather-7plfz/must-gather-kkqsn"
Jul 16 21:30:14.957403 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:14.957380    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7plfz/must-gather-kkqsn"
Jul 16 21:30:15.094854 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:15.094830    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7plfz/must-gather-kkqsn"]
Jul 16 21:30:15.097125 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:30:15.097085    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1598cf50_ca16_4712_b894_7fd9386e84cf.slice/crio-803628a302f486e46377fa9d2d363dfc1788bee9fe88a788d0d909778d933148 WatchSource:0}: Error finding container 803628a302f486e46377fa9d2d363dfc1788bee9fe88a788d0d909778d933148: Status 404 returned error can't find the container with id 803628a302f486e46377fa9d2d363dfc1788bee9fe88a788d0d909778d933148
Jul 16 21:30:15.098942 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:15.098921    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 16 21:30:15.151718 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:15.151685    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7plfz/must-gather-kkqsn" event={"ID":"1598cf50-ca16-4712-b894-7fd9386e84cf","Type":"ContainerStarted","Data":"803628a302f486e46377fa9d2d363dfc1788bee9fe88a788d0d909778d933148"}
Jul 16 21:30:15.283750 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:15.283729    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s"
Jul 16 21:30:15.374014 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:15.373986    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4930d069-009f-45c8-8c97-97b76c099088-maas-api-service-ca\") pod \"4930d069-009f-45c8-8c97-97b76c099088\" (UID: \"4930d069-009f-45c8-8c97-97b76c099088\") "
Jul 16 21:30:15.374352 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:15.374331    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4930d069-009f-45c8-8c97-97b76c099088-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4930d069-009f-45c8-8c97-97b76c099088" (UID: "4930d069-009f-45c8-8c97-97b76c099088"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 16 21:30:15.475582 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:15.475559    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4930d069-009f-45c8-8c97-97b76c099088-maas-api-service-ca\") on node \"ip-10-0-143-245.ec2.internal\" DevicePath \"\""
Jul 16 21:30:16.166217 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:16.162802    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s" event={"ID":"4930d069-009f-45c8-8c97-97b76c099088","Type":"ContainerDied","Data":"268ba70c7d90c5b194603172a895b563ca1a5b93fe033967addfb48f863ea0de"}
Jul 16 21:30:16.166217 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:16.162880    2576 scope.go:117] "RemoveContainer" containerID="84bbfb49417c647faf72d0c4794b2d4b9855adbb4a5bc4f937c503dc4049af8c"
Jul 16 21:30:16.166217 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:16.163016    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s"
Jul 16 21:30:16.170182 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:16.170141    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7plfz/must-gather-kkqsn" event={"ID":"1598cf50-ca16-4712-b894-7fd9386e84cf","Type":"ContainerStarted","Data":"ec442bbb99334c4f042e39a39a7593686b8d4b03bac7966ce860675a98d350b9"}
Jul 16 21:30:16.170299 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:16.170185    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7plfz/must-gather-kkqsn" event={"ID":"1598cf50-ca16-4712-b894-7fd9386e84cf","Type":"ContainerStarted","Data":"8cfd2be208001818047fb939d0503b85d4928e7f637a736dd1ba2ea2537444e0"}
Jul 16 21:30:16.192034 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:16.191985    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7plfz/must-gather-kkqsn" podStartSLOduration=1.460451396 podStartE2EDuration="2.191970963s" totalImagesPullingTime="731.519567ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 21:30:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:30:16.190581959 +0000 UTC m=+3462.156516194" watchObservedRunningTime="2026-07-16 21:30:16.191970963 +0000 UTC m=+3462.157905196"
Jul 16 21:30:16.203087 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:16.203064    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s"]
Jul 16 21:30:16.207201 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:16.207145    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-23a5a19c-ktq9s"]
Jul 16 21:30:16.692501 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:16.692460    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4930d069-009f-45c8-8c97-97b76c099088" path="/var/lib/kubelet/pods/4930d069-009f-45c8-8c97-97b76c099088/volumes"
Jul 16 21:30:17.422309 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:17.422279    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-q77jc_6ef282cd-38f3-4e0d-b862-b9d510e70226/global-pull-secret-syncer/0.log"
Jul 16 21:30:17.638721 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:17.638683    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-r5zjc_91bc89fb-502f-4c3e-8aef-1c2f07998932/konnectivity-agent/0.log"
Jul 16 21:30:17.706873 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:17.706802    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-245.ec2.internal_5861fc40cf4c0bb1ecbf0801c1a2d731/haproxy/0.log"
Jul 16 21:30:23.027233 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:23.027137    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-xlskn_cb5301bc-a55e-45a7-a752-20fe36cad4fe/manager/0.log"
Jul 16 21:30:23.057103 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:23.057071    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-p6ldw_e5b3f53c-d73e-45dd-abfd-b17dbad05a47/manager/0.log"
Jul 16 21:30:23.138805 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:23.138774    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2dw5w_450291ef-6f9f-4952-b4d0-be1d00f94da7/registry-server/0.log"
Jul 16 21:30:23.263382 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:23.263149    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-4k4fb_a953200c-dcb6-4c55-bafd-d6ac67dfadd6/manager/0.log"
Jul 16 21:30:23.289653 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:23.289574    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-djq6h_9a11f252-e376-4afb-a7a2-0327e64e8628/limitador/0.log"
Jul 16 21:30:25.327936 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:25.327682    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-mq2hs_41a9e89f-641d-4700-8c75-aefe2296b1f1/cluster-monitoring-operator/0.log"
Jul 16 21:30:25.689547 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:25.689496    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t24lg_7c62609d-708e-4e2a-b535-35a48ff7fddf/node-exporter/0.log"
Jul 16 21:30:25.712815 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:25.712784    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t24lg_7c62609d-708e-4e2a-b535-35a48ff7fddf/kube-rbac-proxy/0.log"
Jul 16 21:30:25.740014 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:25.739975    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t24lg_7c62609d-708e-4e2a-b535-35a48ff7fddf/init-textfile/0.log"
Jul 16 21:30:25.864086 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:25.864034    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ae0f683a-3a21-40ec-b0c3-832212333b3f/prometheus/0.log"
Jul 16 21:30:25.887101 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:25.887072    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ae0f683a-3a21-40ec-b0c3-832212333b3f/config-reloader/0.log"
Jul 16 21:30:25.911854 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:25.911817    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ae0f683a-3a21-40ec-b0c3-832212333b3f/thanos-sidecar/0.log"
Jul 16 21:30:25.933516 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:25.933481    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ae0f683a-3a21-40ec-b0c3-832212333b3f/kube-rbac-proxy-web/0.log"
Jul 16 21:30:25.957029 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:25.956949    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ae0f683a-3a21-40ec-b0c3-832212333b3f/kube-rbac-proxy/0.log"
Jul 16 21:30:25.978168 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:25.978139    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ae0f683a-3a21-40ec-b0c3-832212333b3f/kube-rbac-proxy-thanos/0.log"
Jul 16 21:30:26.001274 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:26.001251    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ae0f683a-3a21-40ec-b0c3-832212333b3f/init-config-reloader/0.log"
Jul 16 21:30:27.442746 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.442689    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-rz9m8_95d2f323-8204-4405-8320-05650383659f/networking-console-plugin/0.log"
Jul 16 21:30:27.606144 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.606110    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"]
Jul 16 21:30:27.606706 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.606681    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4930d069-009f-45c8-8c97-97b76c099088" containerName="revoke-keys"
Jul 16 21:30:27.606706 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.606708    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4930d069-009f-45c8-8c97-97b76c099088" containerName="revoke-keys"
Jul 16 21:30:27.606906 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.606738    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4930d069-009f-45c8-8c97-97b76c099088" containerName="revoke-keys"
Jul 16 21:30:27.606906 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.606748    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4930d069-009f-45c8-8c97-97b76c099088" containerName="revoke-keys"
Jul 16 21:30:27.606906 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.606760    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4930d069-009f-45c8-8c97-97b76c099088" containerName="revoke-keys"
Jul 16 21:30:27.606906 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.606768    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4930d069-009f-45c8-8c97-97b76c099088" containerName="revoke-keys"
Jul 16 21:30:27.606906 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.606858    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4930d069-009f-45c8-8c97-97b76c099088" containerName="revoke-keys"
Jul 16 21:30:27.606906 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.606873    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4930d069-009f-45c8-8c97-97b76c099088" containerName="revoke-keys"
Jul 16 21:30:27.606906 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.606882    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4930d069-009f-45c8-8c97-97b76c099088" containerName="revoke-keys"
Jul 16 21:30:27.612150 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.612128    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:27.614484 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.614464    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7plfz\"/\"default-dockercfg-pzbhc\""
Jul 16 21:30:27.619424 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.619401    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"]
Jul 16 21:30:27.721706 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.721272    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff0732d0-f2fc-4e6c-8622-8b6a0479dc94-proc\") pod \"perf-node-gather-daemonset-phbhp\" (UID: \"ff0732d0-f2fc-4e6c-8622-8b6a0479dc94\") " pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:27.721706 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.721339    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff0732d0-f2fc-4e6c-8622-8b6a0479dc94-lib-modules\") pod \"perf-node-gather-daemonset-phbhp\" (UID: \"ff0732d0-f2fc-4e6c-8622-8b6a0479dc94\") " pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:27.721706 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.721440    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bvq8\" (UniqueName: \"kubernetes.io/projected/ff0732d0-f2fc-4e6c-8622-8b6a0479dc94-kube-api-access-2bvq8\") pod \"perf-node-gather-daemonset-phbhp\" (UID: \"ff0732d0-f2fc-4e6c-8622-8b6a0479dc94\") " pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:27.721706 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.721489    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff0732d0-f2fc-4e6c-8622-8b6a0479dc94-sys\") pod \"perf-node-gather-daemonset-phbhp\" (UID: \"ff0732d0-f2fc-4e6c-8622-8b6a0479dc94\") " pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:27.721706 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.721521    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff0732d0-f2fc-4e6c-8622-8b6a0479dc94-podres\") pod \"perf-node-gather-daemonset-phbhp\" (UID: \"ff0732d0-f2fc-4e6c-8622-8b6a0479dc94\") " pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:27.822347 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.822304    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff0732d0-f2fc-4e6c-8622-8b6a0479dc94-proc\") pod \"perf-node-gather-daemonset-phbhp\" (UID: \"ff0732d0-f2fc-4e6c-8622-8b6a0479dc94\") " pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:27.822559 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.822418    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff0732d0-f2fc-4e6c-8622-8b6a0479dc94-lib-modules\") pod \"perf-node-gather-daemonset-phbhp\" (UID: \"ff0732d0-f2fc-4e6c-8622-8b6a0479dc94\") " pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:27.822559 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.822475    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bvq8\" (UniqueName: \"kubernetes.io/projected/ff0732d0-f2fc-4e6c-8622-8b6a0479dc94-kube-api-access-2bvq8\") pod \"perf-node-gather-daemonset-phbhp\" (UID: \"ff0732d0-f2fc-4e6c-8622-8b6a0479dc94\") " pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:27.822559 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.822523    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff0732d0-f2fc-4e6c-8622-8b6a0479dc94-sys\") pod \"perf-node-gather-daemonset-phbhp\" (UID: \"ff0732d0-f2fc-4e6c-8622-8b6a0479dc94\") " pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:27.822559 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.822557    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff0732d0-f2fc-4e6c-8622-8b6a0479dc94-podres\") pod \"perf-node-gather-daemonset-phbhp\" (UID: \"ff0732d0-f2fc-4e6c-8622-8b6a0479dc94\") " pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:27.822799 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.822772    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ff0732d0-f2fc-4e6c-8622-8b6a0479dc94-podres\") pod \"perf-node-gather-daemonset-phbhp\" (UID: \"ff0732d0-f2fc-4e6c-8622-8b6a0479dc94\") " pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:27.822859 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.822840    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ff0732d0-f2fc-4e6c-8622-8b6a0479dc94-proc\") pod \"perf-node-gather-daemonset-phbhp\" (UID: \"ff0732d0-f2fc-4e6c-8622-8b6a0479dc94\") " pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:27.822943 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.822924    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ff0732d0-f2fc-4e6c-8622-8b6a0479dc94-lib-modules\") pod \"perf-node-gather-daemonset-phbhp\" (UID: \"ff0732d0-f2fc-4e6c-8622-8b6a0479dc94\") " pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:27.823296 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.823272    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff0732d0-f2fc-4e6c-8622-8b6a0479dc94-sys\") pod \"perf-node-gather-daemonset-phbhp\" (UID: \"ff0732d0-f2fc-4e6c-8622-8b6a0479dc94\") " pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:27.831333 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.831308    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bvq8\" (UniqueName: \"kubernetes.io/projected/ff0732d0-f2fc-4e6c-8622-8b6a0479dc94-kube-api-access-2bvq8\") pod \"perf-node-gather-daemonset-phbhp\" (UID: \"ff0732d0-f2fc-4e6c-8622-8b6a0479dc94\") " pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:27.924600 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:27.924401    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:28.101213 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:28.101179    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"]
Jul 16 21:30:28.106155 ip-10-0-143-245 kubenswrapper[2576]: W0716 21:30:28.106117    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podff0732d0_f2fc_4e6c_8622_8b6a0479dc94.slice/crio-21869b73035ca61da038663e6c083a7a7f7768b9e0b4e3f279bc9ac1a038d065 WatchSource:0}: Error finding container 21869b73035ca61da038663e6c083a7a7f7768b9e0b4e3f279bc9ac1a038d065: Status 404 returned error can't find the container with id 21869b73035ca61da038663e6c083a7a7f7768b9e0b4e3f279bc9ac1a038d065
Jul 16 21:30:28.109530 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:28.109323    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/1.log"
Jul 16 21:30:28.114920 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:28.114901    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-stbb4_44befb08-b09e-471f-a0bc-59d503cd5c93/console-operator/2.log"
Jul 16 21:30:28.265726 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:28.265692    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp" event={"ID":"ff0732d0-f2fc-4e6c-8622-8b6a0479dc94","Type":"ContainerStarted","Data":"d4dedc6c967a4857892d68457af0b0be7891407c204f51f99ab43bc81b7c3f9f"}
Jul 16 21:30:28.265880 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:28.265759    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp" event={"ID":"ff0732d0-f2fc-4e6c-8622-8b6a0479dc94","Type":"ContainerStarted","Data":"21869b73035ca61da038663e6c083a7a7f7768b9e0b4e3f279bc9ac1a038d065"}
Jul 16 21:30:28.265880 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:28.265827    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:28.280837 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:28.280774    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp" podStartSLOduration=1.280756838 podStartE2EDuration="1.280756838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 21:30:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 21:30:28.280578319 +0000 UTC m=+3474.246512552" watchObservedRunningTime="2026-07-16 21:30:28.280756838 +0000 UTC m=+3474.246691072"
Jul 16 21:30:28.622243 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:28.622164    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-wd874_8e0056a5-3a26-4a65-a72b-70eaba00fc54/download-server/0.log"
Jul 16 21:30:29.195970 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:29.195940    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-v65xd_deea2dc0-618a-48aa-833f-bae435b4cdaa/volume-data-source-validator/0.log"
Jul 16 21:30:30.219334 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:30.219305    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-89j4k_c48a73e2-e6f7-468e-b923-927c7ec97110/dns/0.log"
Jul 16 21:30:30.239107 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:30.239076    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-89j4k_c48a73e2-e6f7-468e-b923-927c7ec97110/kube-rbac-proxy/0.log"
Jul 16 21:30:30.395346 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:30.395315    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9jr72_6df4aad3-be34-4b4a-bd20-4b46e9d53737/dns-node-resolver/0.log"
Jul 16 21:30:30.981209 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:30.981184    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6vjn8_3b501cab-446b-4e3d-b9f8-3e2958d42293/node-ca/0.log"
Jul 16 21:30:31.990735 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:31.990681    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mlcjj_acd59d37-8c14-4b5e-97c3-00774e15dbe2/istio-proxy/0.log"
Jul 16 21:30:32.294099 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:32.294025    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rdg46_e041dc27-7c87-4f74-902c-10c2393683bd/istio-proxy/0.log"
Jul 16 21:30:32.334500 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:32.334470    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rdg46_e041dc27-7c87-4f74-902c-10c2393683bd/istio-proxy/1.log"
Jul 16 21:30:32.441693 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:32.441643    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65f496d8cd-qchkt_cbb37170-322b-4128-8731-ad11b8117cdc/router/0.log"
Jul 16 21:30:33.202694 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:33.202668    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qd8ff_fc249a17-1357-464e-be27-57a3a9182c21/serve-healthcheck-canary/0.log"
Jul 16 21:30:33.692334 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:33.692239    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-tppbf_3f4bd827-12b2-48f5-b027-c3ba1761b83c/insights-operator/0.log"
Jul 16 21:30:33.693282 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:33.693262    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-tppbf_3f4bd827-12b2-48f5-b027-c3ba1761b83c/insights-operator/1.log"
Jul 16 21:30:33.854470 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:33.854424    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w8wzk_ff8aa4a9-dff1-4881-9148-c18582ee7864/kube-rbac-proxy/0.log"
Jul 16 21:30:33.877901 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:33.877874    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w8wzk_ff8aa4a9-dff1-4881-9148-c18582ee7864/exporter/0.log"
Jul 16 21:30:33.900267 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:33.900242    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-w8wzk_ff8aa4a9-dff1-4881-9148-c18582ee7864/extractor/0.log"
Jul 16 21:30:34.285069 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:34.285040    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7plfz/perf-node-gather-daemonset-phbhp"
Jul 16 21:30:36.730276 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:36.730248    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mhswl_993d7e8c-d616-4f37-b49d-1e6a7f4b44ba/server/0.log"
Jul 16 21:30:36.816154 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:36.816120    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xz7gs_3d0e5db1-b544-4e53-ae7b-9e798d79581d/manager/0.log"
Jul 16 21:30:36.905129 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:36.905103    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58fdbbf655-jpk7x_23383b4a-08bf-4e83-b4f7-13443a9e940e/manager/0.log"
Jul 16 21:30:36.930631 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:36.930552    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58fdbbf655-jpk7x_23383b4a-08bf-4e83-b4f7-13443a9e940e/copy-manifests/0.log"
Jul 16 21:30:37.586568 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:37.586539    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mlcjj_acd59d37-8c14-4b5e-97c3-00774e15dbe2/istio-proxy/0.log"
Jul 16 21:30:37.850777 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:37.850701    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rdg46_e041dc27-7c87-4f74-902c-10c2393683bd/istio-proxy/0.log"
Jul 16 21:30:37.886817 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:37.886787    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rdg46_e041dc27-7c87-4f74-902c-10c2393683bd/istio-proxy/1.log"
Jul 16 21:30:37.947772 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:37.947744    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65f496d8cd-qchkt_cbb37170-322b-4128-8731-ad11b8117cdc/router/0.log"
Jul 16 21:30:38.385623 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:38.385596    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-56b64d78d7-wmvqs_0903bef2-f62e-42ca-92ce-4765aca0163d/manager/0.log"
Jul 16 21:30:39.168329 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:39.168253    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mlcjj_acd59d37-8c14-4b5e-97c3-00774e15dbe2/istio-proxy/0.log"
Jul 16 21:30:39.446691 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:39.446604    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rdg46_e041dc27-7c87-4f74-902c-10c2393683bd/istio-proxy/0.log"
Jul 16 21:30:39.480315 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:39.480285    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-rdg46_e041dc27-7c87-4f74-902c-10c2393683bd/istio-proxy/1.log"
Jul 16 21:30:39.564349 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:39.564320    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65f496d8cd-qchkt_cbb37170-322b-4128-8731-ad11b8117cdc/router/0.log"
Jul 16 21:30:43.739739 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:43.739710    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-nm6rf_a3b3a919-d977-4f1e-9a34-e7c7a1368d35/kube-storage-version-migrator-operator/1.log"
Jul 16 21:30:43.740709 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:43.740687    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-nm6rf_a3b3a919-d977-4f1e-9a34-e7c7a1368d35/kube-storage-version-migrator-operator/0.log"
Jul 16 21:30:44.886958 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:44.886890    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v4qls_c77ecce4-9890-4cef-bf49-ebac541c6d6d/kube-multus-additional-cni-plugins/0.log"
Jul 16 21:30:44.908904 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:44.908882    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v4qls_c77ecce4-9890-4cef-bf49-ebac541c6d6d/egress-router-binary-copy/0.log"
Jul 16 21:30:44.931074 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:44.931050    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v4qls_c77ecce4-9890-4cef-bf49-ebac541c6d6d/cni-plugins/0.log"
Jul 16 21:30:44.954785 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:44.954756    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v4qls_c77ecce4-9890-4cef-bf49-ebac541c6d6d/bond-cni-plugin/0.log"
Jul 16 21:30:44.977262 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:44.977242    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v4qls_c77ecce4-9890-4cef-bf49-ebac541c6d6d/routeoverride-cni/0.log"
Jul 16 21:30:45.000535 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:45.000505    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v4qls_c77ecce4-9890-4cef-bf49-ebac541c6d6d/whereabouts-cni-bincopy/0.log"
Jul 16 21:30:45.023727 ip-10-0-143-245 kubenswrapper[2576]: I0716 21:30:45.023695    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-v4qls_c77ecce4-9890-4cef-bf49-ebac541c6d6d/whereabouts-cni/0.log"