Jul 25 01:09:09.592638 ip-10-0-134-238 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 25 01:09:09.592651 ip-10-0-134-238 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 25 01:09:09.592661 ip-10-0-134-238 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 25 01:09:09.592964 ip-10-0-134-238 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 25 01:09:19.596981 ip-10-0-134-238 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 25 01:09:19.597000 ip-10-0-134-238 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 27cf8a650e2f44babff60550d78f4624 --
Jul 25 01:10:48.386971 ip-10-0-134-238 systemd[1]: Starting Kubernetes Kubelet...
Jul 25 01:10:48.831996 ip-10-0-134-238 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 25 01:10:48.831996 ip-10-0-134-238 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 25 01:10:48.831996 ip-10-0-134-238 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 25 01:10:48.831996 ip-10-0-134-238 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 25 01:10:48.831996 ip-10-0-134-238 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 25 01:10:48.833097 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.833005    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 25 01:10:48.837027 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837013    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 01:10:48.837027 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837027    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837031    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837034    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837038    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837041    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837044    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837047    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837051    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837054    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837057    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837060    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837068    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837071    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837074    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837077    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837079    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837082    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837084    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837087    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837090    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 01:10:48.837088 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837092    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837095    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837098    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837101    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837104    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837107    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837109    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837113    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837115    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837118    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837121    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837123    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837126    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837128    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837131    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837133    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837136    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837138    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837141    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 01:10:48.837567 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837143    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837146    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837148    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837151    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837154    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837156    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837159    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837162    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837165    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837169    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837172    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837175    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837178    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837181    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837183    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837187    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837190    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837193    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837196    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837199    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 01:10:48.838073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837202    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837204    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837208    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837213    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837216    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837219    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837222    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837225    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837227    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837230    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837233    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837240    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837243    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837246    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837248    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837251    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837253    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837256    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837259    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 01:10:48.838546 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837261    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837264    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837267    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837269    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837272    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837274    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837277    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837279    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837282    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837285    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837288    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837292    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837296    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837299    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837301    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837304    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837307    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837309    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837312    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.837315    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 01:10:48.839018 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838024    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838031    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838033    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838036    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838039    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838042    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838045    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838047    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838050    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838053    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838055    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838058    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838061    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838063    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838066    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838069    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838072    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838074    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838077    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838080    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 01:10:48.839495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838084    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838087    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838090    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838092    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838095    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838097    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838100    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838102    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838105    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838108    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838112    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838115    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838118    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838121    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838124    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838127    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838129    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838132    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838135    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 01:10:48.839993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838137    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838140    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838142    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838145    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838147    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838150    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838153    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838156    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838158    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838161    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838163    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838166    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838168    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838172    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838174    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838177    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838180    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838182    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838185    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838187    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 01:10:48.840536 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838229    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838233    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838237    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838240    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838243    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838246    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838249    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838252    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838255    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838258    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838261    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838263    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838266    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838269    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838272    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838274    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838277    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838280    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838282    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 01:10:48.841074 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838285    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838287    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838290    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838292    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838295    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838297    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838300    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838303    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838305    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838308    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838311    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838314    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838317    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838320    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838324    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838327    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838331    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838334    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838336    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 01:10:48.841549 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838339    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838342    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838755    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838761    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838765    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838768    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838771    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838774    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838777    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838779    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838783    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838785    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838788    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838790    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838793    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838795    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838798    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838801    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838805    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 01:10:48.842022 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838809    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838812    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838816    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838818    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838822    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838825    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838827    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838830    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838833    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838836    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838839    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838841    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838844    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838847    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838849    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838852    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838855    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838857    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838860    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838862    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 01:10:48.842479 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838865    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838868    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838870    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838873    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838875    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838878    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838880    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838885    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838888    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838890    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838893    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838895    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838899    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838902    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838905    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838907    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838910    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838912    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838915    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838917    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 01:10:48.842977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838921    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838923    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838926    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838929    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838931    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838934    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838936    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838939    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838941    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838944    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838948    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838951    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838954    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838958    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838961    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838964    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838967    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838969    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838972    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 01:10:48.843458 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838974    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838977    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838980    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838982    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838985    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838987    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838990    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838992    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838995    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.838998    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839000    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839003    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839005    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839008    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839011    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839013    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839016    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839019    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839021    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 01:10:48.843919 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839024    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839027    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839030    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839032    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839096    2576 flags.go:64] FLAG: --address="0.0.0.0"
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839103    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839111    2576 flags.go:64] FLAG: --anonymous-auth="true"
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839115    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839120    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839124    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839128    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839132    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839135    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839138    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839150    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839154    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839157    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839160    2576 flags.go:64] FLAG: --cgroup-root=""
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839163    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839166    2576 flags.go:64] FLAG: --client-ca-file=""
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839169    2576 flags.go:64] FLAG: --cloud-provider="external"
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839172    2576 flags.go:64] FLAG: --cluster-dns="[]"
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839176    2576 flags.go:64] FLAG: --cluster-domain=""
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839179    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 25 01:10:48.844375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839182    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839185    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839188    2576 flags.go:64] FLAG: --container-log-max-files="5"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839199    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839202    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839205    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839209    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839212    2576 flags.go:64] FLAG: --contention-profiling="false"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839215    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839218    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839221    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839224    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839229    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839232    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839234    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839237    2576 flags.go:64] FLAG: --enable-load-reader="false"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839240    2576 flags.go:64] FLAG: --enable-server="true"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839244    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839248    2576 flags.go:64] FLAG: --event-burst="100"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839252    2576 flags.go:64] FLAG: --event-qps="50"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839255    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839258    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839261    2576 flags.go:64] FLAG: --eviction-hard=""
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839264    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 25 01:10:48.844954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839267    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839270    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839273    2576 flags.go:64] FLAG: --eviction-soft=""
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839276    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839279    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839282    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839284    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839287    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839290    2576 flags.go:64] FLAG: --fail-swap-on="true"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839293    2576 flags.go:64] FLAG: --feature-gates=""
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839297    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839301    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839304    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839307    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839310    2576 flags.go:64] FLAG: --healthz-port="10248"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839315    2576 flags.go:64] FLAG: --help="false"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839318    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-134-238.ec2.internal"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839321    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839324    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839327    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839330    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839333    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839336    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 25 01:10:48.845541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839338    2576 flags.go:64] FLAG: --image-service-endpoint=""
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839341    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839344    2576 flags.go:64] FLAG: --kube-api-burst="100"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839348    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839351    2576 flags.go:64] FLAG: --kube-api-qps="50"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839354    2576 flags.go:64] FLAG: --kube-reserved=""
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839357    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839360    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839363    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839365    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839368    2576 flags.go:64] FLAG: --lock-file=""
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839371    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839374    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839376    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839381    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839384    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839387    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839390    2576 flags.go:64] FLAG: --logging-format="text"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839393    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839396    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839400    2576 flags.go:64] FLAG: --manifest-url=""
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839403    2576 flags.go:64] FLAG: --manifest-url-header=""
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839407    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839410    2576 flags.go:64] FLAG: --max-open-files="1000000"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839415    2576 flags.go:64] FLAG: --max-pods="110"
Jul 25 01:10:48.846129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839418    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839421    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839424    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839427    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839430    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839433    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839436    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839443    2576 flags.go:64] FLAG: --node-status-max-images="50"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839446    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839449    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839451    2576 flags.go:64] FLAG: --pod-cidr=""
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839455    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839460    2576 flags.go:64] FLAG: --pod-manifest-path=""
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839463    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839466    2576 flags.go:64] FLAG: --pods-per-core="0"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839468    2576 flags.go:64] FLAG: --port="10250"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839471    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839474    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-084e3755a7e4ab683"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839477    2576 flags.go:64] FLAG: --qos-reserved=""
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839480    2576 flags.go:64] FLAG: --read-only-port="10255"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839483    2576 flags.go:64] FLAG: --register-node="true"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839486    2576 flags.go:64] FLAG: --register-with-taints=""
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839489    2576 flags.go:64] FLAG: --registry-burst="10"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839492    2576 flags.go:64] FLAG: --registry-qps="5"
Jul 25 01:10:48.846753 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839495    2576 flags.go:64] FLAG: --reserved-cpus=""
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839498    2576 flags.go:64] FLAG: --reserved-memory=""
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839502    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839506    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839509    2576 flags.go:64] FLAG: --rotate-certificates="false"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839512    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839515    2576 flags.go:64] FLAG: --runonce="false"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839518    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839521    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839524    2576 flags.go:64] FLAG: --seccomp-default="false"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839527    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839530    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839533    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839536    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839539    2576 flags.go:64] FLAG: --storage-driver-password="root"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839542    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839545    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839547    2576 flags.go:64] FLAG: --storage-driver-user="root"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839550    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839553    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839556    2576 flags.go:64] FLAG: --system-cgroups=""
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839559    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839564    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839567    2576 flags.go:64] FLAG: --tls-cert-file=""
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839569    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 25 01:10:48.847333 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839573    2576 flags.go:64] FLAG: --tls-min-version=""
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839576    2576 flags.go:64] FLAG: --tls-private-key-file=""
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839579    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839581    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839584    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839587    2576 flags.go:64] FLAG: --v="2"
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839590    2576 flags.go:64] FLAG: --version="false"
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839594    2576 flags.go:64] FLAG: --vmodule=""
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839598    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.839601    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839725    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839730    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839733    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839736    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839738    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839742    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839745    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839748    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839751    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839754    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839757    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839759    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 01:10:48.847998 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839762    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839765    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839767    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839770    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839773    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839775    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839778    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839781    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839783    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839786    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839789    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839791    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839794    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839796    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839799    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839801    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839804    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839807    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839811    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839813    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 01:10:48.848527 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839817    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839819    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839822    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839825    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839827    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839830    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839832    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839835    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839837    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839840    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839844    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839847    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839850    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839852    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839856    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839859    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839863    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839866    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839869    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 01:10:48.849024 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839873    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839875    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839878    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839880    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839883    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839885    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839888    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839891    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839894    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839896    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839898    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839901    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839903    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839910    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839913    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839915    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839917    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839920    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839923    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839926    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 01:10:48.849488 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839928    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839931    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839934    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839936    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839939    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839942    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839944    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839946    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839949    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839951    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839954    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839956    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839959    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839962    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839964    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839967    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839969    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839972    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839974    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839977    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 01:10:48.850029 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839979    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 01:10:48.850507 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839982    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 01:10:48.850507 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839984    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 01:10:48.850507 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839987    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 01:10:48.850507 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839989    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 01:10:48.850507 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839993    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 01:10:48.850507 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839995    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 01:10:48.850507 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.839998    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 01:10:48.850507 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.840913    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 25 01:10:48.850507 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.847863    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 25 01:10:48.850507 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.847877    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 25 01:10:48.850507 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.847896    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 25 01:10:48.850507 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.847902    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 25 01:10:48.850507 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.847972    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 01:10:48.850507 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.847977    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 01:10:48.850507 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.847981    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.847984    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.847987    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.847990    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.847993    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.847996    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.847999    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848003    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848006    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848009    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848013    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848016    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848018    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848021    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848024    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848026    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848029    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848032    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848034    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 01:10:48.850901 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848037    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848039    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848042    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848045    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848049    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848052    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848055    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848057    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848061    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848064    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848067    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848070    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848072    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848075    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848077    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848080    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848083    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848085    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848088    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848090    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 01:10:48.851379 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848093    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848097    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848101    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848103    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848106    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848109    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848112    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848114    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848117    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848119    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848121    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848124    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848126    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848129    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848132    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848134    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848137    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848139    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848142    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848144    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 01:10:48.851943 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848147    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848152    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848155    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848158    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848161    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848163    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848166    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848168    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848171    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848173    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848175    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848178    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848180    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848183    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848185    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848188    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848191    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848194    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848196    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 01:10:48.852412 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848199    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848201    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848204    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848206    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848209    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848211    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848214    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848216    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848219    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848221    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848224    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848226    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848229    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848231    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848234    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848236    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848239    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848241    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 01:10:48.852871 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848244    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 01:10:48.853340 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.848249    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 25 01:10:48.853340 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848373    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 01:10:48.853340 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848378    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 01:10:48.853340 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848381    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 01:10:48.853340 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848384    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 01:10:48.853340 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848386    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 01:10:48.853340 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848389    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 01:10:48.853340 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848393    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 01:10:48.853340 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848397    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 01:10:48.853340 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848401    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 01:10:48.853340 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848405    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 01:10:48.853340 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848408    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 01:10:48.853340 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848411    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 01:10:48.853340 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848414    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848417    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848420    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848423    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848426    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848429    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848431    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848434    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848437    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848440    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848442    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848445    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848448    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848450    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848453    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848456    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848458    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848461    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848463    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848466    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 01:10:48.853701 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848468    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848470    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848473    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848476    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848478    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848480    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848483    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848485    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848488    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848490    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848493    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848495    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848498    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848501    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848503    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848506    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848508    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848511    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848513    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 01:10:48.854177 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848516    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848518    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848521    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848524    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848527    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848531    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848534    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848537    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848540    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848543    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848546    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848549    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848552    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848554    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848557    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848559    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848562    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848564    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848567    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848569    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 01:10:48.854631 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848572    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848574    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848576    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848579    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848581    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848584    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848587    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848589    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848592    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848594    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848597    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848600    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848602    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848605    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848607    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848610    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848612    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848615    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848617    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848620    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 01:10:48.855116 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848622    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 01:10:48.855585 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848625    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 01:10:48.855585 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848627    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 01:10:48.855585 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848630    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 01:10:48.855585 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848632    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 01:10:48.855585 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848635    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 01:10:48.855585 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848638    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 01:10:48.855585 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:48.848640    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 01:10:48.855585 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.848645    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 25 01:10:48.855585 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.848752    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 25 01:10:48.855585 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.852011    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 25 01:10:48.855585 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.853723    2576 server.go:1019] "Starting client certificate rotation"
Jul 25 01:10:48.855585 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.853819    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 25 01:10:48.855585 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.853852    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 25 01:10:48.859081 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.859060    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 25 01:10:48.860242 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.860227    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 25 01:10:48.868748 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.868729    2576 log.go:25] "Validated CRI v1 runtime API"
Jul 25 01:10:48.871126 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.871109    2576 log.go:25] "Validated CRI v1 image API"
Jul 25 01:10:48.874967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.874923    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 25 01:10:48.877540 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.877522    2576 fs.go:135] Filesystem UUIDs: map[6eef6303-377a-49c9-aa61-ee09d8ef0276:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9160ca3e-4abd-4d58-b011-f930c56e63e8:/dev/nvme0n1p4]
Jul 25 01:10:48.877608 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.877539    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 25 01:10:48.884639 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.884462    2576 manager.go:216] Machine: {Timestamp:2026-07-25 01:10:48.882996771 +0000 UTC m=+0.370293437 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101421 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e2540a2e3c1cd3a4438f78afff6e2 SystemUUID:ec2e2540-a2e3-c1cd-3a44-38f78afff6e2 BootID:27cf8a65-0e2f-44ba-bff6-0550d78f4624 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:06:d4:1d:1d:cf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:06:d4:1d:1d:cf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:22:1f:6e:b9:d6:92 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 25 01:10:48.884639 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.884633    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 25 01:10:48.884745 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.884725    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 25 01:10:48.887439 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.887422    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 25 01:10:48.888175 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.888151    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 25 01:10:48.888316 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.888179    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-238.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 25 01:10:48.889051 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.889041    2576 topology_manager.go:138] "Creating topology manager with none policy"
Jul 25 01:10:48.889089 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.889055    2576 container_manager_linux.go:309] "Creating device plugin manager"
Jul 25 01:10:48.889089 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.889062    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 25 01:10:48.889773 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.889761    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 25 01:10:48.891082 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.891071    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 25 01:10:48.894023 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.894012    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 25 01:10:48.894141 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.894131    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 25 01:10:48.894178 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.894160    2576 kubelet.go:486] "Attempting to sync node with API server"
Jul 25 01:10:48.894178 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.894175    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 25 01:10:48.894235 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.894192    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 25 01:10:48.894235 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.894201    2576 kubelet.go:398] "Adding apiserver pod source"
Jul 25 01:10:48.894235 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.894209    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 25 01:10:48.894616 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.894605    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 25 01:10:48.894674 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.894623    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 25 01:10:48.897243 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.897207    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 25 01:10:48.901041 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.901013    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 25 01:10:48.901150 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.901111    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 25 01:10:48.903423 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.903408    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 25 01:10:48.903495 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.903430    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 25 01:10:48.903495 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.903439    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 25 01:10:48.903495 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.903448    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 25 01:10:48.903495 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.903462    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 25 01:10:48.903495 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.903471    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 25 01:10:48.903495 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.903484    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 25 01:10:48.903495 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.903493    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 25 01:10:48.903732 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.903518    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 25 01:10:48.903732 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.903535    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 25 01:10:48.903732 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.903563    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 25 01:10:48.903732 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.903584    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 25 01:10:48.904478 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.904465    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 25 01:10:48.904622 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.904611    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 25 01:10:48.907236 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.907219    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-238.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 25 01:10:48.907459 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:48.907434    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-238.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 25 01:10:48.907526 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:48.907484    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 25 01:10:48.909427 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.909410    2576 server.go:1268] "Started kubelet"
Jul 25 01:10:48.909533 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.909507    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 25 01:10:48.909579 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.909527    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 25 01:10:48.909627 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.909585    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 25 01:10:48.910198 ip-10-0-134-238 systemd[1]: Started Kubernetes Kubelet.
Jul 25 01:10:48.911767 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.911753    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 25 01:10:48.912260 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.912249    2576 server.go:310] "Adding debug handlers to kubelet server"
Jul 25 01:10:48.915453 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:48.915412    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 25 01:10:48.917851 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:48.916670    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-238.ec2.internal.18c5632790f67760  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-238.ec2.internal,UID:ip-10-0-134-238.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-238.ec2.internal,},FirstTimestamp:2026-07-25 01:10:48.909215584 +0000 UTC m=+0.396512249,LastTimestamp:2026-07-25 01:10:48.909215584 +0000 UTC m=+0.396512249,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-238.ec2.internal,}"
Jul 25 01:10:48.921529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.921514    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 25 01:10:48.921529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.921525    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 25 01:10:48.921677 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:48.921589    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-238.ec2.internal\" not found"
Jul 25 01:10:48.922267 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.922249    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 25 01:10:48.922354 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.922278    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 25 01:10:48.922354 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.922284    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 25 01:10:48.922354 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.922299    2576 factory.go:55] Registering systemd factory
Jul 25 01:10:48.922354 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.922308    2576 factory.go:223] Registration of the systemd container factory successfully
Jul 25 01:10:48.922354 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.922254    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 25 01:10:48.922558 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.922430    2576 reconstruct.go:97] "Volume reconstruction finished"
Jul 25 01:10:48.922558 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.922440    2576 reconciler.go:29] "Reconciler: start to sync state"
Jul 25 01:10:48.922641 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.922571    2576 factory.go:153] Registering CRI-O factory
Jul 25 01:10:48.922641 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.922584    2576 factory.go:223] Registration of the crio container factory successfully
Jul 25 01:10:48.922641 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.922602    2576 factory.go:103] Registering Raw factory
Jul 25 01:10:48.922641 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.922611    2576 manager.go:1196] Started watching for new ooms in manager
Jul 25 01:10:48.922983 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.922970    2576 manager.go:349] Starting recovery of all containers
Jul 25 01:10:48.923204 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:48.923177    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-238.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 25 01:10:48.923306 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:48.923273    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 25 01:10:48.931379 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.931344    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9x2rj"
Jul 25 01:10:48.935552 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.935504    2576 manager.go:354] Recovery completed
Jul 25 01:10:48.938592 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:48.938566    2576 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 25 01:10:48.939161 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.939081    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9x2rj"
Jul 25 01:10:48.942619 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.942604    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 01:10:48.944688 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.944671    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 01:10:48.944758 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.944702    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 01:10:48.944758 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.944716    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasSufficientPID"
Jul 25 01:10:48.945314 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.945297    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 25 01:10:48.945314 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.945311    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 25 01:10:48.945466 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.945327    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 25 01:10:48.945466 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.945330    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 01:10:48.946100 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.946078    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 01:10:48.946163 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.946104    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 01:10:48.946163 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.946113    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasSufficientPID"
Jul 25 01:10:48.947980 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.947967    2576 policy_none.go:49] "None policy: Start"
Jul 25 01:10:48.948047 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.947986    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 25 01:10:48.948047 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.948000    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 25 01:10:48.950032 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.950019    2576 policy_none.go:47] "Start"
Jul 25 01:10:48.976433 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.976315    2576 manager.go:337] "Starting Device Plugin manager"
Jul 25 01:10:49.001349 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:48.976473    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 25 01:10:49.001349 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.976487    2576 server.go:87] "Starting device plugin registration server"
Jul 25 01:10:49.001349 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.976715    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 25 01:10:49.001349 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.976725    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 25 01:10:49.001349 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.976819    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 25 01:10:49.001349 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.976898    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 25 01:10:49.001349 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.976906    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 25 01:10:49.001349 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:48.977386    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 25 01:10:49.001349 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:48.977425    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-238.ec2.internal\" not found"
Jul 25 01:10:49.001349 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.988633    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 25 01:10:49.001349 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.989795    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 25 01:10:49.001349 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.989823    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 25 01:10:49.001349 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.989840    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 25 01:10:49.001349 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:48.989876    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 25 01:10:49.001349 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:48.992315    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 01:10:49.077354 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.077323    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 01:10:49.080618 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.080604    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 01:10:49.080711 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.080634    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 01:10:49.080711 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.080649    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasSufficientPID"
Jul 25 01:10:49.080711 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.080699    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.090247 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.090210    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.090247 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:49.090229    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-238.ec2.internal\": node \"ip-10-0-134-238.ec2.internal\" not found"
Jul 25 01:10:49.090330 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.090298    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-238.ec2.internal"]
Jul 25 01:10:49.090370 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.090351    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 01:10:49.091098 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.091085    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 01:10:49.091173 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.091108    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 01:10:49.091173 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.091120    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasSufficientPID"
Jul 25 01:10:49.093522 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.093510    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 01:10:49.093679 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.093648    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.093731 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.093701    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 01:10:49.094191 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.094176    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 01:10:49.094191 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.094184    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 01:10:49.094310 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.094200    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 01:10:49.094310 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.094205    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 01:10:49.094310 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.094214    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasSufficientPID"
Jul 25 01:10:49.094310 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.094219    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasSufficientPID"
Jul 25 01:10:49.096453 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.096440    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.096501 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.096464    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 01:10:49.097107 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.097093    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 01:10:49.097170 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.097116    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 01:10:49.097170 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.097125    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeHasSufficientPID"
Jul 25 01:10:49.111768 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:49.111751    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-238.ec2.internal\" not found"
Jul 25 01:10:49.124112 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.124092    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a69f653c5364a5c191e807aff8316576-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal\" (UID: \"a69f653c5364a5c191e807aff8316576\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.124201 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.124136    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1b910c3459c38a928558d492ab28b0ef-config\") pod \"kube-apiserver-proxy-ip-10-0-134-238.ec2.internal\" (UID: \"1b910c3459c38a928558d492ab28b0ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.124201 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.124183    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a69f653c5364a5c191e807aff8316576-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal\" (UID: \"a69f653c5364a5c191e807aff8316576\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.135312 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:49.135287    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-238.ec2.internal\" not found" node="ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.139695 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:49.139680    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-238.ec2.internal\" not found" node="ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.211862 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:49.211827    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-238.ec2.internal\" not found"
Jul 25 01:10:49.225236 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.225218    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a69f653c5364a5c191e807aff8316576-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal\" (UID: \"a69f653c5364a5c191e807aff8316576\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.225301 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.225248    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a69f653c5364a5c191e807aff8316576-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal\" (UID: \"a69f653c5364a5c191e807aff8316576\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.225301 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.225270    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1b910c3459c38a928558d492ab28b0ef-config\") pod \"kube-apiserver-proxy-ip-10-0-134-238.ec2.internal\" (UID: \"1b910c3459c38a928558d492ab28b0ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.225369 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.225309    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1b910c3459c38a928558d492ab28b0ef-config\") pod \"kube-apiserver-proxy-ip-10-0-134-238.ec2.internal\" (UID: \"1b910c3459c38a928558d492ab28b0ef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.225369 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.225315    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a69f653c5364a5c191e807aff8316576-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal\" (UID: \"a69f653c5364a5c191e807aff8316576\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.225369 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.225325    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a69f653c5364a5c191e807aff8316576-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal\" (UID: \"a69f653c5364a5c191e807aff8316576\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.312401 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:49.312369    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-238.ec2.internal\" not found"
Jul 25 01:10:49.413142 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:49.413074    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-238.ec2.internal\" not found"
Jul 25 01:10:49.439635 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.439612    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.444426 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.444404    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.513567 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:49.513526    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-238.ec2.internal\" not found"
Jul 25 01:10:49.614028 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:49.613999    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-238.ec2.internal\" not found"
Jul 25 01:10:49.714612 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:49.714540    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-238.ec2.internal\" not found"
Jul 25 01:10:49.723168 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.723152    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 01:10:49.822222 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.822198    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.835233 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.835213    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 25 01:10:49.837397 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.837380    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-238.ec2.internal"
Jul 25 01:10:49.846521 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.846504    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 25 01:10:49.852797 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.852779    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 25 01:10:49.852959 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.852937    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 25 01:10:49.894914 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.894872    2576 apiserver.go:52] "Watching apiserver"
Jul 25 01:10:49.903506 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.903485    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 25 01:10:49.906095 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.905416    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-pmpt5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal","openshift-multus/multus-additional-cni-plugins-8srzj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm","openshift-multus/multus-swm4p","openshift-multus/network-metrics-daemon-87t4z","openshift-network-diagnostics/network-check-target-gc8gr","openshift-network-operator/iptables-alerter-rgqsm","openshift-ovn-kubernetes/ovnkube-node-kcf67","kube-system/global-pull-secret-syncer-bqzgx","kube-system/konnectivity-agent-h7kwx","kube-system/kube-apiserver-proxy-ip-10-0-134-238.ec2.internal","openshift-cluster-node-tuning-operator/tuned-mzbmt","openshift-dns/node-resolver-mttfq"]
Jul 25 01:10:49.908553 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.908533    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.910994 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.910720    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 25 01:10:49.910994 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.910787    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-sw9bt\""
Jul 25 01:10:49.910994 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.910722    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 25 01:10:49.910994 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.910940    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 25 01:10:49.910994 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.910992    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 25 01:10:49.911317 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.911093    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 25 01:10:49.911317 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.911146    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 25 01:10:49.911317 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.911189    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mttfq"
Jul 25 01:10:49.913134 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.913113    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4msrs\""
Jul 25 01:10:49.913230 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.913161    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 25 01:10:49.913344 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.913326    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 25 01:10:49.913496 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.913475    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:49.915326 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.915308    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kfkrs\""
Jul 25 01:10:49.915326 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.915325    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 25 01:10:49.915439 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.915309    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 25 01:10:49.915958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.915939    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:49.917659 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.917640    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 25 01:10:49.917779 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.917758    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 25 01:10:49.917876 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.917795    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gj7pg\""
Jul 25 01:10:49.917876 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.917804    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 25 01:10:49.918408 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.918386    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.920117 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.920098    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 25 01:10:49.920280 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.920266    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 25 01:10:49.920360 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.920306    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 25 01:10:49.920360 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.920331    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cslc9\""
Jul 25 01:10:49.920455 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.920410    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 25 01:10:49.920767 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.920751    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:10:49.920860 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:49.920826    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:10:49.921826 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.921802    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 25 01:10:49.923237 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.923216    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pmpt5"
Jul 25 01:10:49.925666 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.925642    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 25 01:10:49.925770 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.925647    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rgqsm"
Jul 25 01:10:49.925914 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.925889    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 25 01:10:49.926015 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.925683    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 25 01:10:49.926354 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.926336    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-k9mw9\""
Jul 25 01:10:49.928046 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.928007    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 25 01:10:49.928501 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.928199    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 25 01:10:49.928501 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.928206    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:10:49.928501 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:49.928289    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:10:49.928501 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.928327    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5wvsp\""
Jul 25 01:10:49.928501 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.928345    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 25 01:10:49.929142 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929120    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs8sf\" (UniqueName: \"kubernetes.io/projected/bf4b06fe-fa8a-4281-8995-d92fb33486fc-kube-api-access-fs8sf\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.929235 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929156    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-modprobe-d\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:49.929235 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929186    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-cni-netd\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.929235 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929209    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bf4b06fe-fa8a-4281-8995-d92fb33486fc-ovn-node-metrics-cert\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.929396 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929239    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c804a635-6110-4cea-b2d8-5e8660b449f8-tmp-dir\") pod \"node-resolver-mttfq\" (UID: \"c804a635-6110-4cea-b2d8-5e8660b449f8\") " pod="openshift-dns/node-resolver-mttfq"
Jul 25 01:10:49.929396 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929272    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr4mz\" (UniqueName: \"kubernetes.io/projected/c804a635-6110-4cea-b2d8-5e8660b449f8-kube-api-access-vr4mz\") pod \"node-resolver-mttfq\" (UID: \"c804a635-6110-4cea-b2d8-5e8660b449f8\") " pod="openshift-dns/node-resolver-mttfq"
Jul 25 01:10:49.929396 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929292    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-host\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:49.929396 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929316    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj92d\" (UniqueName: \"kubernetes.io/projected/6f9c02f7-779b-41bb-8260-3244475a0f20-kube-api-access-rj92d\") pod \"network-metrics-daemon-87t4z\" (UID: \"6f9c02f7-779b-41bb-8260-3244475a0f20\") " pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:10:49.929396 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929362    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-run-k8s-cni-cncf-io\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.929396 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929389    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-run-systemd\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.929633 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929412    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bf4b06fe-fa8a-4281-8995-d92fb33486fc-ovnkube-config\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.929633 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929437    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-multus-socket-dir-parent\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.929633 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929463    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-hostroot\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.929633 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929492    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c804a635-6110-4cea-b2d8-5e8660b449f8-hosts-file\") pod \"node-resolver-mttfq\" (UID: \"c804a635-6110-4cea-b2d8-5e8660b449f8\") " pod="openshift-dns/node-resolver-mttfq"
Jul 25 01:10:49.929633 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929562    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-sysconfig\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:49.929633 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929591    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-system-cni-dir\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.929633 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929615    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phlkc\" (UniqueName: \"kubernetes.io/projected/c6ebb66f-5622-481b-ae46-7b3743feec00-kube-api-access-phlkc\") pod \"node-ca-pmpt5\" (UID: \"c6ebb66f-5622-481b-ae46-7b3743feec00\") " pod="openshift-image-registry/node-ca-pmpt5"
Jul 25 01:10:49.929967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929639    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-slash\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.929967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929654    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-kubelet\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.929967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929719    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-run-netns\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.929967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929735    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-cni-bin\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.929967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929755    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-run\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:49.929967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929779    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rjqr\" (UniqueName: \"kubernetes.io/projected/d018052d-5c6b-46d5-bac9-aa0171194b90-kube-api-access-6rjqr\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:49.929967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929798    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-cnibin\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.929967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929814    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-etc-kubernetes\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.929967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929827    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6ebb66f-5622-481b-ae46-7b3743feec00-host\") pod \"node-ca-pmpt5\" (UID: \"c6ebb66f-5622-481b-ae46-7b3743feec00\") " pod="openshift-image-registry/node-ca-pmpt5"
Jul 25 01:10:49.929967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929850    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-sysctl-conf\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:49.929967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929874    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-os-release\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.929967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929893    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/58dcfb48-9812-4413-b614-dd5cca8dd63a-multus-daemon-config\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.929967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929909    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-kubernetes\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:49.929967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929944    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs\") pod \"network-metrics-daemon-87t4z\" (UID: \"6f9c02f7-779b-41bb-8260-3244475a0f20\") " pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:10:49.930744 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.929976    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-node-log\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.930744 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930000    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:49.930744 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930027    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-registration-dir\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:49.930744 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930060    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-sys-fs\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:49.930744 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930087    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-multus-cni-dir\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.930744 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930123    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-var-lib-kubelet\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.930744 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930151    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-systemd-units\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.930744 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930176    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-var-lib-openvswitch\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.930744 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930199    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-run-ovn-kubernetes\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.930744 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930225    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.930744 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930252    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bf4b06fe-fa8a-4281-8995-d92fb33486fc-env-overrides\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.930744 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930276    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-sys\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:49.930744 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930302    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-etc-selinux\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:49.930744 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930327    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-multus-conf-dir\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.930744 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930351    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-var-lib-kubelet\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:49.930744 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930377    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-device-dir\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:49.931471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930402    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/58dcfb48-9812-4413-b614-dd5cca8dd63a-cni-binary-copy\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.931471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930427    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-run-netns\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.931471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930451    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-var-lib-cni-multus\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.931471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930477    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-run-multus-certs\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.931471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930502    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwjld\" (UniqueName: \"kubernetes.io/projected/58dcfb48-9812-4413-b614-dd5cca8dd63a-kube-api-access-nwjld\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.931471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930552    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c6ebb66f-5622-481b-ae46-7b3743feec00-serviceca\") pod \"node-ca-pmpt5\" (UID: \"c6ebb66f-5622-481b-ae46-7b3743feec00\") " pod="openshift-image-registry/node-ca-pmpt5"
Jul 25 01:10:49.931471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930579    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-sysctl-d\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:49.931471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930611    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-var-lib-cni-bin\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:49.931471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930631    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bf4b06fe-fa8a-4281-8995-d92fb33486fc-ovnkube-script-lib\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.931471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930678    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-lib-modules\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:49.931471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930710    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-tuned\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:49.931471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930733    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d018052d-5c6b-46d5-bac9-aa0171194b90-tmp\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:49.931471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930766    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-etc-openvswitch\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.931471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930791    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-systemd\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:49.931471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930815    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-socket-dir\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:49.931471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930840    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7swf\" (UniqueName: \"kubernetes.io/projected/571e6c8e-9624-4be9-81a7-7a6ae27267ce-kube-api-access-z7swf\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:49.931471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930878    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-run-openvswitch\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.932110 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930911    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-run-ovn\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.932110 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.930937    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-log-socket\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:49.932110 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.931622    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:49.932110 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:49.931712    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:10:49.934071 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.934055    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h7kwx"
Jul 25 01:10:49.935857 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.935834    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9kw9r\""
Jul 25 01:10:49.935933 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.935896    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 25 01:10:49.935993 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.935951    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 25 01:10:49.937459 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.936938    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:49.938932 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.938907    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 25 01:10:49.939312 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.939293    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 25 01:10:49.939441 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.939422    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 25 01:10:49.941625 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.941605    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8k95n\""
Jul 25 01:10:49.945900 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.945851    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-24 01:05:49 +0000 UTC" deadline="2027-12-25 21:15:59.434777304 +0000 UTC"
Jul 25 01:10:49.945973 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.945902    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12452h5m9.488880447s"
Jul 25 01:10:49.958541 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.958520    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-srzv7"
Jul 25 01:10:49.971489 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.971441    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-srzv7"
Jul 25 01:10:49.994786 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:49.994768    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 01:10:50.023850 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.023831    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 25 01:10:50.031641 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.031614    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-sysctl-conf\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.031746 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.031651    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-os-release\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.031746 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.031694    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/58dcfb48-9812-4413-b614-dd5cca8dd63a-multus-daemon-config\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.031746 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.031725    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-os-release\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.031960 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.031753    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-kubernetes\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.031960 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.031779    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs\") pod \"network-metrics-daemon-87t4z\" (UID: \"6f9c02f7-779b-41bb-8260-3244475a0f20\") " pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:10:50.031960 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.031780    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-os-release\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.031960 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.031805    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-system-cni-dir\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.031960 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.031812    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-sysctl-conf\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.031960 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.031851    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/056dc3b3-b95b-4d1f-8606-333625c397a3-kubelet-config\") pod \"global-pull-secret-syncer-bqzgx\" (UID: \"056dc3b3-b95b-4d1f-8606-333625c397a3\") " pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:50.031960 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.031863    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-kubernetes\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.031960 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.031895    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-node-log\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.031960 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.031921    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:50.031960 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.031931    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-node-log\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.031960 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:50.031897    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:10:50.031960 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.031948    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-registration-dir\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:50.032529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.031975    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-sys-fs\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:50.032529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.031989    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:50.032529 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:50.032012    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs podName:6f9c02f7-779b-41bb-8260-3244475a0f20 nodeName:}" failed. No retries permitted until 2026-07-25 01:10:50.531975591 +0000 UTC m=+2.019272245 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs") pod "network-metrics-daemon-87t4z" (UID: "6f9c02f7-779b-41bb-8260-3244475a0f20") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:10:50.032529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032037    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-multus-cni-dir\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.032529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032040    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-sys-fs\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:50.032529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032053    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-var-lib-kubelet\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.032529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032049    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-registration-dir\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:50.032529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032098    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-var-lib-kubelet\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.032529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032174    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-multus-cni-dir\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.032529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032180    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.032529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.032529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032262    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-systemd-units\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.032529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032288    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-var-lib-openvswitch\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.032529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032314    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-run-ovn-kubernetes\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.032529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032339    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-var-lib-openvswitch\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.032529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032341    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.033273 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032350    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/58dcfb48-9812-4413-b614-dd5cca8dd63a-multus-daemon-config\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.033273 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032370    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-run-ovn-kubernetes\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.033273 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032372    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bf4b06fe-fa8a-4281-8995-d92fb33486fc-env-overrides\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.033273 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032354    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-systemd-units\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.033273 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032396    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-sys\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.033273 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032423    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-cni-binary-copy\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.033273 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032397    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.033273 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032439    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-sys\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.033273 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032449    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2pxf\" (UniqueName: \"kubernetes.io/projected/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-kube-api-access-b2pxf\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.033273 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032484    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-etc-selinux\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:50.033273 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032522    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-multus-conf-dir\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.033273 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032540    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cttd\" (UniqueName: \"kubernetes.io/projected/1e24f294-fa2e-4cc1-9390-adc201492447-kube-api-access-8cttd\") pod \"iptables-alerter-rgqsm\" (UID: \"1e24f294-fa2e-4cc1-9390-adc201492447\") " pod="openshift-network-operator/iptables-alerter-rgqsm"
Jul 25 01:10:50.033273 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret\") pod \"global-pull-secret-syncer-bqzgx\" (UID: \"056dc3b3-b95b-4d1f-8606-333625c397a3\") " pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:50.033273 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032573    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-var-lib-kubelet\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.033273 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032571    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-etc-selinux\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:50.033273 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032600    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-device-dir\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:50.033273 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032624    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-multus-conf-dir\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032627    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-var-lib-kubelet\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032627    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/58dcfb48-9812-4413-b614-dd5cca8dd63a-cni-binary-copy\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032650    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-device-dir\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032683    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-run-netns\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032721    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-var-lib-cni-multus\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032738    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-run-multus-certs\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032762    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwjld\" (UniqueName: \"kubernetes.io/projected/58dcfb48-9812-4413-b614-dd5cca8dd63a-kube-api-access-nwjld\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032777    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-run-netns\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032782    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-var-lib-cni-multus\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c6ebb66f-5622-481b-ae46-7b3743feec00-serviceca\") pod \"node-ca-pmpt5\" (UID: \"c6ebb66f-5622-481b-ae46-7b3743feec00\") " pod="openshift-image-registry/node-ca-pmpt5"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032808    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-run-multus-certs\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032812    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-sysctl-d\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032843    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bf4b06fe-fa8a-4281-8995-d92fb33486fc-env-overrides\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032853    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-var-lib-cni-bin\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032902    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-var-lib-cni-bin\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032909    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-sysctl-d\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032899    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.034156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.032977    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bf4b06fe-fa8a-4281-8995-d92fb33486fc-ovnkube-script-lib\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033001    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-lib-modules\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033017    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-tuned\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033032    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d018052d-5c6b-46d5-bac9-aa0171194b90-tmp\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033053    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-etc-openvswitch\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033075    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-systemd\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033097    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-socket-dir\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033122    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7swf\" (UniqueName: \"kubernetes.io/projected/571e6c8e-9624-4be9-81a7-7a6ae27267ce-kube-api-access-z7swf\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033148    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-run-openvswitch\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033154    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-lib-modules\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033171    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-run-ovn\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033194    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-log-socket\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033212    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-systemd\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033236    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs8sf\" (UniqueName: \"kubernetes.io/projected/bf4b06fe-fa8a-4281-8995-d92fb33486fc-kube-api-access-fs8sf\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033265    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-modprobe-d\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033291    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e24f294-fa2e-4cc1-9390-adc201492447-host-slash\") pod \"iptables-alerter-rgqsm\" (UID: \"1e24f294-fa2e-4cc1-9390-adc201492447\") " pod="openshift-network-operator/iptables-alerter-rgqsm"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-cni-netd\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033336    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 25 01:10:50.035132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033358    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c6ebb66f-5622-481b-ae46-7b3743feec00-serviceca\") pod \"node-ca-pmpt5\" (UID: \"c6ebb66f-5622-481b-ae46-7b3743feec00\") " pod="openshift-image-registry/node-ca-pmpt5"
Jul 25 01:10:50.035958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033365    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bf4b06fe-fa8a-4281-8995-d92fb33486fc-ovn-node-metrics-cert\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.035958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033389    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c804a635-6110-4cea-b2d8-5e8660b449f8-tmp-dir\") pod \"node-resolver-mttfq\" (UID: \"c804a635-6110-4cea-b2d8-5e8660b449f8\") " pod="openshift-dns/node-resolver-mttfq"
Jul 25 01:10:50.035958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033409    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr4mz\" (UniqueName: \"kubernetes.io/projected/c804a635-6110-4cea-b2d8-5e8660b449f8-kube-api-access-vr4mz\") pod \"node-resolver-mttfq\" (UID: \"c804a635-6110-4cea-b2d8-5e8660b449f8\") " pod="openshift-dns/node-resolver-mttfq"
Jul 25 01:10:50.035958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033414    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-log-socket\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.035958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033429    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-host\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.035958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033455    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj92d\" (UniqueName: \"kubernetes.io/projected/6f9c02f7-779b-41bb-8260-3244475a0f20-kube-api-access-rj92d\") pod \"network-metrics-daemon-87t4z\" (UID: \"6f9c02f7-779b-41bb-8260-3244475a0f20\") " pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:10:50.035958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033482    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-run-k8s-cni-cncf-io\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.035958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033507    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-run-systemd\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.035958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033509    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-etc-openvswitch\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.035958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033510    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/571e6c8e-9624-4be9-81a7-7a6ae27267ce-socket-dir\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:50.035958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033531    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bf4b06fe-fa8a-4281-8995-d92fb33486fc-ovnkube-config\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.035958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033583    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-multus-socket-dir-parent\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.035958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033619    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-hostroot\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.035958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033649    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-cnibin\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.035958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033677    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ttkw\" (UniqueName: \"kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw\") pod \"network-check-target-gc8gr\" (UID: \"772e370c-1a11-44c2-8dc9-3c9e1919df27\") " pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:10:50.035958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033728    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f8888b32-a7c5-4b96-b48d-133e3491fb81-konnectivity-ca\") pod \"konnectivity-agent-h7kwx\" (UID: \"f8888b32-a7c5-4b96-b48d-133e3491fb81\") " pod="kube-system/konnectivity-agent-h7kwx"
Jul 25 01:10:50.035958 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033757    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c804a635-6110-4cea-b2d8-5e8660b449f8-hosts-file\") pod \"node-resolver-mttfq\" (UID: \"c804a635-6110-4cea-b2d8-5e8660b449f8\") " pod="openshift-dns/node-resolver-mttfq"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033760    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-run-openvswitch\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033782    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-sysconfig\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033807    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-run-ovn\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033810    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f8888b32-a7c5-4b96-b48d-133e3491fb81-agent-certs\") pod \"konnectivity-agent-h7kwx\" (UID: \"f8888b32-a7c5-4b96-b48d-133e3491fb81\") " pod="kube-system/konnectivity-agent-h7kwx"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033841    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/58dcfb48-9812-4413-b614-dd5cca8dd63a-cni-binary-copy\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033867    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-hostroot\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033869    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c804a635-6110-4cea-b2d8-5e8660b449f8-hosts-file\") pod \"node-resolver-mttfq\" (UID: \"c804a635-6110-4cea-b2d8-5e8660b449f8\") " pod="openshift-dns/node-resolver-mttfq"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033919    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-sysconfig\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.033924    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-system-cni-dir\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034013    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bf4b06fe-fa8a-4281-8995-d92fb33486fc-ovnkube-config\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034102    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phlkc\" (UniqueName: \"kubernetes.io/projected/c6ebb66f-5622-481b-ae46-7b3743feec00-kube-api-access-phlkc\") pod \"node-ca-pmpt5\" (UID: \"c6ebb66f-5622-481b-ae46-7b3743feec00\") " pod="openshift-image-registry/node-ca-pmpt5"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-cni-netd\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034150    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1e24f294-fa2e-4cc1-9390-adc201492447-iptables-alerter-script\") pod \"iptables-alerter-rgqsm\" (UID: \"1e24f294-fa2e-4cc1-9390-adc201492447\") " pod="openshift-network-operator/iptables-alerter-rgqsm"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034183    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-system-cni-dir\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034184    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-slash\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034225    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/056dc3b3-b95b-4d1f-8606-333625c397a3-dbus\") pod \"global-pull-secret-syncer-bqzgx\" (UID: \"056dc3b3-b95b-4d1f-8606-333625c397a3\") " pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034256    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-kubelet\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.036881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034261    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-slash\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034286    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-run-netns\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034316    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-cni-bin\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034329    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-multus-socket-dir-parent\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034370    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-run\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034376    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-run-netns\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034392    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rjqr\" (UniqueName: \"kubernetes.io/projected/d018052d-5c6b-46d5-bac9-aa0171194b90-kube-api-access-6rjqr\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034418    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-cnibin\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034432    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-cni-bin\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034472    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-etc-kubernetes\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034488    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-run\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034494    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6ebb66f-5622-481b-ae46-7b3743feec00-host\") pod \"node-ca-pmpt5\" (UID: \"c6ebb66f-5622-481b-ae46-7b3743feec00\") " pod="openshift-image-registry/node-ca-pmpt5"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034503    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-host-kubelet\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034470    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-modprobe-d\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034557    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d018052d-5c6b-46d5-bac9-aa0171194b90-host\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034594    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bf4b06fe-fa8a-4281-8995-d92fb33486fc-run-systemd\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034669    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6ebb66f-5622-481b-ae46-7b3743feec00-host\") pod \"node-ca-pmpt5\" (UID: \"c6ebb66f-5622-481b-ae46-7b3743feec00\") " pod="openshift-image-registry/node-ca-pmpt5"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034705    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-cnibin\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.037709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034760    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-host-run-k8s-cni-cncf-io\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.038269 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.034846    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58dcfb48-9812-4413-b614-dd5cca8dd63a-etc-kubernetes\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.038269 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.035082    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bf4b06fe-fa8a-4281-8995-d92fb33486fc-ovnkube-script-lib\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.038269 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.035130    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c804a635-6110-4cea-b2d8-5e8660b449f8-tmp-dir\") pod \"node-resolver-mttfq\" (UID: \"c804a635-6110-4cea-b2d8-5e8660b449f8\") " pod="openshift-dns/node-resolver-mttfq"
Jul 25 01:10:50.038269 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.037506    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d018052d-5c6b-46d5-bac9-aa0171194b90-tmp\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.038269 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.037518    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bf4b06fe-fa8a-4281-8995-d92fb33486fc-ovn-node-metrics-cert\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.038269 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.037541    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d018052d-5c6b-46d5-bac9-aa0171194b90-etc-tuned\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.041515 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.041489    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwjld\" (UniqueName: \"kubernetes.io/projected/58dcfb48-9812-4413-b614-dd5cca8dd63a-kube-api-access-nwjld\") pod \"multus-swm4p\" (UID: \"58dcfb48-9812-4413-b614-dd5cca8dd63a\") " pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.042292 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.042269    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs8sf\" (UniqueName: \"kubernetes.io/projected/bf4b06fe-fa8a-4281-8995-d92fb33486fc-kube-api-access-fs8sf\") pod \"ovnkube-node-kcf67\" (UID: \"bf4b06fe-fa8a-4281-8995-d92fb33486fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.042424 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.042402    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7swf\" (UniqueName: \"kubernetes.io/projected/571e6c8e-9624-4be9-81a7-7a6ae27267ce-kube-api-access-z7swf\") pod \"aws-ebs-csi-driver-node-b8gdm\" (UID: \"571e6c8e-9624-4be9-81a7-7a6ae27267ce\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:50.046621 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.043896    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rjqr\" (UniqueName: \"kubernetes.io/projected/d018052d-5c6b-46d5-bac9-aa0171194b90-kube-api-access-6rjqr\") pod \"tuned-mzbmt\" (UID: \"d018052d-5c6b-46d5-bac9-aa0171194b90\") " pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.047019 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.046996    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr4mz\" (UniqueName: \"kubernetes.io/projected/c804a635-6110-4cea-b2d8-5e8660b449f8-kube-api-access-vr4mz\") pod \"node-resolver-mttfq\" (UID: \"c804a635-6110-4cea-b2d8-5e8660b449f8\") " pod="openshift-dns/node-resolver-mttfq"
Jul 25 01:10:50.047206 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.047174    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj92d\" (UniqueName: \"kubernetes.io/projected/6f9c02f7-779b-41bb-8260-3244475a0f20-kube-api-access-rj92d\") pod \"network-metrics-daemon-87t4z\" (UID: \"6f9c02f7-779b-41bb-8260-3244475a0f20\") " pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:10:50.047294 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:50.047228    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b910c3459c38a928558d492ab28b0ef.slice/crio-4932e5b11d212587a5259b9eee1e6c8517cedf0fddfcc9842f99ea8821f83148 WatchSource:0}: Error finding container 4932e5b11d212587a5259b9eee1e6c8517cedf0fddfcc9842f99ea8821f83148: Status 404 returned error can't find the container with id 4932e5b11d212587a5259b9eee1e6c8517cedf0fddfcc9842f99ea8821f83148
Jul 25 01:10:50.048004 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.047981    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phlkc\" (UniqueName: \"kubernetes.io/projected/c6ebb66f-5622-481b-ae46-7b3743feec00-kube-api-access-phlkc\") pod \"node-ca-pmpt5\" (UID: \"c6ebb66f-5622-481b-ae46-7b3743feec00\") " pod="openshift-image-registry/node-ca-pmpt5"
Jul 25 01:10:50.048359 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:50.048341    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda69f653c5364a5c191e807aff8316576.slice/crio-776a274557d953a06e031f0c2d4fbfa6f6d50118ed95085ed53f71d7784cccc9 WatchSource:0}: Error finding container 776a274557d953a06e031f0c2d4fbfa6f6d50118ed95085ed53f71d7784cccc9: Status 404 returned error can't find the container with id 776a274557d953a06e031f0c2d4fbfa6f6d50118ed95085ed53f71d7784cccc9
Jul 25 01:10:50.051568 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.051551    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 01:10:50.135477 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135439    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.135477 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135481    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e24f294-fa2e-4cc1-9390-adc201492447-host-slash\") pod \"iptables-alerter-rgqsm\" (UID: \"1e24f294-fa2e-4cc1-9390-adc201492447\") " pod="openshift-network-operator/iptables-alerter-rgqsm"
Jul 25 01:10:50.135652 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135510    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-cnibin\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.135652 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135566    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-cnibin\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.135652 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135569    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1e24f294-fa2e-4cc1-9390-adc201492447-host-slash\") pod \"iptables-alerter-rgqsm\" (UID: \"1e24f294-fa2e-4cc1-9390-adc201492447\") " pod="openshift-network-operator/iptables-alerter-rgqsm"
Jul 25 01:10:50.135652 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135604    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttkw\" (UniqueName: \"kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw\") pod \"network-check-target-gc8gr\" (UID: \"772e370c-1a11-44c2-8dc9-3c9e1919df27\") " pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:10:50.135652 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135630    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f8888b32-a7c5-4b96-b48d-133e3491fb81-konnectivity-ca\") pod \"konnectivity-agent-h7kwx\" (UID: \"f8888b32-a7c5-4b96-b48d-133e3491fb81\") " pod="kube-system/konnectivity-agent-h7kwx"
Jul 25 01:10:50.135872 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135659    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f8888b32-a7c5-4b96-b48d-133e3491fb81-agent-certs\") pod \"konnectivity-agent-h7kwx\" (UID: \"f8888b32-a7c5-4b96-b48d-133e3491fb81\") " pod="kube-system/konnectivity-agent-h7kwx"
Jul 25 01:10:50.135872 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135695    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.135872 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135704    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1e24f294-fa2e-4cc1-9390-adc201492447-iptables-alerter-script\") pod \"iptables-alerter-rgqsm\" (UID: \"1e24f294-fa2e-4cc1-9390-adc201492447\") " pod="openshift-network-operator/iptables-alerter-rgqsm"
Jul 25 01:10:50.135872 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135731    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/056dc3b3-b95b-4d1f-8606-333625c397a3-dbus\") pod \"global-pull-secret-syncer-bqzgx\" (UID: \"056dc3b3-b95b-4d1f-8606-333625c397a3\") " pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:50.135872 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135754    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-os-release\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.135872 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135784    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-system-cni-dir\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.135872 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/056dc3b3-b95b-4d1f-8606-333625c397a3-kubelet-config\") pod \"global-pull-secret-syncer-bqzgx\" (UID: \"056dc3b3-b95b-4d1f-8606-333625c397a3\") " pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:50.135872 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135840    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.135872 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135849    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/056dc3b3-b95b-4d1f-8606-333625c397a3-dbus\") pod \"global-pull-secret-syncer-bqzgx\" (UID: \"056dc3b3-b95b-4d1f-8606-333625c397a3\") " pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:50.136289 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135879    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.136289 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135897    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-system-cni-dir\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.136289 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135906    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-os-release\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.136289 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135917    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-cni-binary-copy\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.136289 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135949    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/056dc3b3-b95b-4d1f-8606-333625c397a3-kubelet-config\") pod \"global-pull-secret-syncer-bqzgx\" (UID: \"056dc3b3-b95b-4d1f-8606-333625c397a3\") " pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:50.136289 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.135987    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2pxf\" (UniqueName: \"kubernetes.io/projected/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-kube-api-access-b2pxf\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.136289 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.136035    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cttd\" (UniqueName: \"kubernetes.io/projected/1e24f294-fa2e-4cc1-9390-adc201492447-kube-api-access-8cttd\") pod \"iptables-alerter-rgqsm\" (UID: \"1e24f294-fa2e-4cc1-9390-adc201492447\") " pod="openshift-network-operator/iptables-alerter-rgqsm"
Jul 25 01:10:50.136289 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.136064    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret\") pod \"global-pull-secret-syncer-bqzgx\" (UID: \"056dc3b3-b95b-4d1f-8606-333625c397a3\") " pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:50.136289 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:50.136170    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 01:10:50.136289 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:50.136237    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret podName:056dc3b3-b95b-4d1f-8606-333625c397a3 nodeName:}" failed. No retries permitted until 2026-07-25 01:10:50.636218823 +0000 UTC m=+2.123515488 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret") pod "global-pull-secret-syncer-bqzgx" (UID: "056dc3b3-b95b-4d1f-8606-333625c397a3") : object "kube-system"/"original-pull-secret" not registered
Jul 25 01:10:50.136844 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.136325    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f8888b32-a7c5-4b96-b48d-133e3491fb81-konnectivity-ca\") pod \"konnectivity-agent-h7kwx\" (UID: \"f8888b32-a7c5-4b96-b48d-133e3491fb81\") " pod="kube-system/konnectivity-agent-h7kwx"
Jul 25 01:10:50.136844 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.136359    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1e24f294-fa2e-4cc1-9390-adc201492447-iptables-alerter-script\") pod \"iptables-alerter-rgqsm\" (UID: \"1e24f294-fa2e-4cc1-9390-adc201492447\") " pod="openshift-network-operator/iptables-alerter-rgqsm"
Jul 25 01:10:50.136844 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.136490    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.136844 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.136651    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.137365 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.137347    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-cni-binary-copy\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.138178 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.138158    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f8888b32-a7c5-4b96-b48d-133e3491fb81-agent-certs\") pod \"konnectivity-agent-h7kwx\" (UID: \"f8888b32-a7c5-4b96-b48d-133e3491fb81\") " pod="kube-system/konnectivity-agent-h7kwx"
Jul 25 01:10:50.141775 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:50.141758    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 01:10:50.141775 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:50.141776    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 01:10:50.141899 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:50.141786    2576 projected.go:196] Error preparing data for projected volume kube-api-access-8ttkw for pod openshift-network-diagnostics/network-check-target-gc8gr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:10:50.141899 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:50.141834    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw podName:772e370c-1a11-44c2-8dc9-3c9e1919df27 nodeName:}" failed. No retries permitted until 2026-07-25 01:10:50.641821614 +0000 UTC m=+2.129118283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8ttkw" (UniqueName: "kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw") pod "network-check-target-gc8gr" (UID: "772e370c-1a11-44c2-8dc9-3c9e1919df27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:10:50.144872 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.144853    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cttd\" (UniqueName: \"kubernetes.io/projected/1e24f294-fa2e-4cc1-9390-adc201492447-kube-api-access-8cttd\") pod \"iptables-alerter-rgqsm\" (UID: \"1e24f294-fa2e-4cc1-9390-adc201492447\") " pod="openshift-network-operator/iptables-alerter-rgqsm"
Jul 25 01:10:50.144937 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.144876    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2pxf\" (UniqueName: \"kubernetes.io/projected/026f8bf6-bdcf-47a9-930e-59323c2cb9f3-kube-api-access-b2pxf\") pod \"multus-additional-cni-plugins-8srzj\" (UID: \"026f8bf6-bdcf-47a9-930e-59323c2cb9f3\") " pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.197229 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.197208    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 01:10:50.233394 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.233341    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:10:50.243691 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:50.243670    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf4b06fe_fa8a_4281_8995_d92fb33486fc.slice/crio-e8c3fdc01c178aba3b5d32e3c8a9c4edac2ce509b2e440d883c6e6a0cf400731 WatchSource:0}: Error finding container e8c3fdc01c178aba3b5d32e3c8a9c4edac2ce509b2e440d883c6e6a0cf400731: Status 404 returned error can't find the container with id e8c3fdc01c178aba3b5d32e3c8a9c4edac2ce509b2e440d883c6e6a0cf400731
Jul 25 01:10:50.253708 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.253690    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mttfq"
Jul 25 01:10:50.262351 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.262334    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mzbmt"
Jul 25 01:10:50.262532 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:50.262515    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc804a635_6110_4cea_b2d8_5e8660b449f8.slice/crio-5d3ce028e22a7ee4aefe1558383a6192f23d3f455cae5f21eea2a7bfe975e8d2 WatchSource:0}: Error finding container 5d3ce028e22a7ee4aefe1558383a6192f23d3f455cae5f21eea2a7bfe975e8d2: Status 404 returned error can't find the container with id 5d3ce028e22a7ee4aefe1558383a6192f23d3f455cae5f21eea2a7bfe975e8d2
Jul 25 01:10:50.270515 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.270503    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm"
Jul 25 01:10:50.270689 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:50.270652    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd018052d_5c6b_46d5_bac9_aa0171194b90.slice/crio-b224e847086d2b0afb4d4659521d3cf1b96dbe13d31e4e1d5f8077b12649b22f WatchSource:0}: Error finding container b224e847086d2b0afb4d4659521d3cf1b96dbe13d31e4e1d5f8077b12649b22f: Status 404 returned error can't find the container with id b224e847086d2b0afb4d4659521d3cf1b96dbe13d31e4e1d5f8077b12649b22f
Jul 25 01:10:50.278256 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:50.278235    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod571e6c8e_9624_4be9_81a7_7a6ae27267ce.slice/crio-0ded7c1b5fb7bee978bb9caef5aaa54e5a006cbe7c017bbd6284f4a1e9c9e777 WatchSource:0}: Error finding container 0ded7c1b5fb7bee978bb9caef5aaa54e5a006cbe7c017bbd6284f4a1e9c9e777: Status 404 returned error can't find the container with id 0ded7c1b5fb7bee978bb9caef5aaa54e5a006cbe7c017bbd6284f4a1e9c9e777
Jul 25 01:10:50.285100 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.285085    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-swm4p"
Jul 25 01:10:50.294133 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:50.294108    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58dcfb48_9812_4413_b614_dd5cca8dd63a.slice/crio-83ceaaa53f02ee30045780803356dc2bb97e596db51524a006c1abb8f033db2a WatchSource:0}: Error finding container 83ceaaa53f02ee30045780803356dc2bb97e596db51524a006c1abb8f033db2a: Status 404 returned error can't find the container with id 83ceaaa53f02ee30045780803356dc2bb97e596db51524a006c1abb8f033db2a
Jul 25 01:10:50.301871 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.301853    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pmpt5"
Jul 25 01:10:50.309898 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:50.309878    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6ebb66f_5622_481b_ae46_7b3743feec00.slice/crio-c49e37e09245b86b83f98c08642611b1f522e6491836cb5725660bc7f8d1eeb8 WatchSource:0}: Error finding container c49e37e09245b86b83f98c08642611b1f522e6491836cb5725660bc7f8d1eeb8: Status 404 returned error can't find the container with id c49e37e09245b86b83f98c08642611b1f522e6491836cb5725660bc7f8d1eeb8
Jul 25 01:10:50.313858 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.313844    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rgqsm"
Jul 25 01:10:50.322001 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:50.321979    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e24f294_fa2e_4cc1_9390_adc201492447.slice/crio-8308d5317fe593838a31a02a636255e6915dc40087d8ef3b7798fd1a0ea8bb47 WatchSource:0}: Error finding container 8308d5317fe593838a31a02a636255e6915dc40087d8ef3b7798fd1a0ea8bb47: Status 404 returned error can't find the container with id 8308d5317fe593838a31a02a636255e6915dc40087d8ef3b7798fd1a0ea8bb47
Jul 25 01:10:50.335332 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.335311    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h7kwx"
Jul 25 01:10:50.344968 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:50.344948    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8888b32_a7c5_4b96_b48d_133e3491fb81.slice/crio-8aebaf61b690163092117c18dab5146545c5c185d89f91d9357fdc85d1af2dea WatchSource:0}: Error finding container 8aebaf61b690163092117c18dab5146545c5c185d89f91d9357fdc85d1af2dea: Status 404 returned error can't find the container with id 8aebaf61b690163092117c18dab5146545c5c185d89f91d9357fdc85d1af2dea
Jul 25 01:10:50.361355 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.361336    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8srzj"
Jul 25 01:10:50.370658 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:10:50.370634    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026f8bf6_bdcf_47a9_930e_59323c2cb9f3.slice/crio-4e4c7e80d53a7751d4ffb44bd2d0b9f8ff9f4c7b9bac6c9a25e96438b5122474 WatchSource:0}: Error finding container 4e4c7e80d53a7751d4ffb44bd2d0b9f8ff9f4c7b9bac6c9a25e96438b5122474: Status 404 returned error can't find the container with id 4e4c7e80d53a7751d4ffb44bd2d0b9f8ff9f4c7b9bac6c9a25e96438b5122474
Jul 25 01:10:50.538881 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.538798    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs\") pod \"network-metrics-daemon-87t4z\" (UID: \"6f9c02f7-779b-41bb-8260-3244475a0f20\") " pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:10:50.539028 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:50.538995    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:10:50.539092 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:50.539079    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs podName:6f9c02f7-779b-41bb-8260-3244475a0f20 nodeName:}" failed. No retries permitted until 2026-07-25 01:10:51.539056241 +0000 UTC m=+3.026352911 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs") pod "network-metrics-daemon-87t4z" (UID: "6f9c02f7-779b-41bb-8260-3244475a0f20") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:10:50.639321 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.639283    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret\") pod \"global-pull-secret-syncer-bqzgx\" (UID: \"056dc3b3-b95b-4d1f-8606-333625c397a3\") " pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:50.639492 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:50.639414    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 01:10:50.639492 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:50.639479    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret podName:056dc3b3-b95b-4d1f-8606-333625c397a3 nodeName:}" failed. No retries permitted until 2026-07-25 01:10:51.639458622 +0000 UTC m=+3.126755278 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret") pod "global-pull-secret-syncer-bqzgx" (UID: "056dc3b3-b95b-4d1f-8606-333625c397a3") : object "kube-system"/"original-pull-secret" not registered
Jul 25 01:10:50.740233 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.740196    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttkw\" (UniqueName: \"kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw\") pod \"network-check-target-gc8gr\" (UID: \"772e370c-1a11-44c2-8dc9-3c9e1919df27\") " pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:10:50.740413 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:50.740363    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 01:10:50.740413 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:50.740387    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 01:10:50.740413 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:50.740400    2576 projected.go:196] Error preparing data for projected volume kube-api-access-8ttkw for pod openshift-network-diagnostics/network-check-target-gc8gr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:10:50.740600 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:50.740458    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw podName:772e370c-1a11-44c2-8dc9-3c9e1919df27 nodeName:}" failed. No retries permitted until 2026-07-25 01:10:51.740439526 +0000 UTC m=+3.227736186 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ttkw" (UniqueName: "kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw") pod "network-check-target-gc8gr" (UID: "772e370c-1a11-44c2-8dc9-3c9e1919df27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:10:50.974801 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.974681    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-24 01:05:50 +0000 UTC" deadline="2028-03-09 00:38:52.796514198 +0000 UTC"
Jul 25 01:10:50.974801 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:50.974744    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14231h28m1.821774696s"
Jul 25 01:10:51.019984 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.019923    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8srzj" event={"ID":"026f8bf6-bdcf-47a9-930e-59323c2cb9f3","Type":"ContainerStarted","Data":"4e4c7e80d53a7751d4ffb44bd2d0b9f8ff9f4c7b9bac6c9a25e96438b5122474"}
Jul 25 01:10:51.024221 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.024140    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h7kwx" event={"ID":"f8888b32-a7c5-4b96-b48d-133e3491fb81","Type":"ContainerStarted","Data":"8aebaf61b690163092117c18dab5146545c5c185d89f91d9357fdc85d1af2dea"}
Jul 25 01:10:51.052709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.052627    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pmpt5" event={"ID":"c6ebb66f-5622-481b-ae46-7b3743feec00","Type":"ContainerStarted","Data":"c49e37e09245b86b83f98c08642611b1f522e6491836cb5725660bc7f8d1eeb8"}
Jul 25 01:10:51.059844 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.059814    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-swm4p" event={"ID":"58dcfb48-9812-4413-b614-dd5cca8dd63a","Type":"ContainerStarted","Data":"83ceaaa53f02ee30045780803356dc2bb97e596db51524a006c1abb8f033db2a"}
Jul 25 01:10:51.072986 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.072918    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm" event={"ID":"571e6c8e-9624-4be9-81a7-7a6ae27267ce","Type":"ContainerStarted","Data":"0ded7c1b5fb7bee978bb9caef5aaa54e5a006cbe7c017bbd6284f4a1e9c9e777"}
Jul 25 01:10:51.083811 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.083779    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mzbmt" event={"ID":"d018052d-5c6b-46d5-bac9-aa0171194b90","Type":"ContainerStarted","Data":"b224e847086d2b0afb4d4659521d3cf1b96dbe13d31e4e1d5f8077b12649b22f"}
Jul 25 01:10:51.088889 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.088861    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mttfq" event={"ID":"c804a635-6110-4cea-b2d8-5e8660b449f8","Type":"ContainerStarted","Data":"5d3ce028e22a7ee4aefe1558383a6192f23d3f455cae5f21eea2a7bfe975e8d2"}
Jul 25 01:10:51.097643 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.097611    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal" event={"ID":"a69f653c5364a5c191e807aff8316576","Type":"ContainerStarted","Data":"776a274557d953a06e031f0c2d4fbfa6f6d50118ed95085ed53f71d7784cccc9"}
Jul 25 01:10:51.117368 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.117341    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rgqsm" event={"ID":"1e24f294-fa2e-4cc1-9390-adc201492447","Type":"ContainerStarted","Data":"8308d5317fe593838a31a02a636255e6915dc40087d8ef3b7798fd1a0ea8bb47"}
Jul 25 01:10:51.129750 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.129670    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67" event={"ID":"bf4b06fe-fa8a-4281-8995-d92fb33486fc","Type":"ContainerStarted","Data":"e8c3fdc01c178aba3b5d32e3c8a9c4edac2ce509b2e440d883c6e6a0cf400731"}
Jul 25 01:10:51.139183 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.138118    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-238.ec2.internal" event={"ID":"1b910c3459c38a928558d492ab28b0ef","Type":"ContainerStarted","Data":"4932e5b11d212587a5259b9eee1e6c8517cedf0fddfcc9842f99ea8821f83148"}
Jul 25 01:10:51.334895 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.334806    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 01:10:51.547384 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.547346    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs\") pod \"network-metrics-daemon-87t4z\" (UID: \"6f9c02f7-779b-41bb-8260-3244475a0f20\") " pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:10:51.547592 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:51.547561    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:10:51.547660 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:51.547626    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs podName:6f9c02f7-779b-41bb-8260-3244475a0f20 nodeName:}" failed. No retries permitted until 2026-07-25 01:10:53.547606659 +0000 UTC m=+5.034903319 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs") pod "network-metrics-daemon-87t4z" (UID: "6f9c02f7-779b-41bb-8260-3244475a0f20") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:10:51.648330 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.648246    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret\") pod \"global-pull-secret-syncer-bqzgx\" (UID: \"056dc3b3-b95b-4d1f-8606-333625c397a3\") " pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:51.648510 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:51.648403    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 01:10:51.648510 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:51.648470    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret podName:056dc3b3-b95b-4d1f-8606-333625c397a3 nodeName:}" failed. No retries permitted until 2026-07-25 01:10:53.648450375 +0000 UTC m=+5.135747038 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret") pod "global-pull-secret-syncer-bqzgx" (UID: "056dc3b3-b95b-4d1f-8606-333625c397a3") : object "kube-system"/"original-pull-secret" not registered
Jul 25 01:10:51.749007 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.748970    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttkw\" (UniqueName: \"kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw\") pod \"network-check-target-gc8gr\" (UID: \"772e370c-1a11-44c2-8dc9-3c9e1919df27\") " pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:10:51.749212 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:51.749194    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 01:10:51.749293 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:51.749220    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 01:10:51.749293 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:51.749236    2576 projected.go:196] Error preparing data for projected volume kube-api-access-8ttkw for pod openshift-network-diagnostics/network-check-target-gc8gr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:10:51.749403 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:51.749296    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw podName:772e370c-1a11-44c2-8dc9-3c9e1919df27 nodeName:}" failed. No retries permitted until 2026-07-25 01:10:53.749276977 +0000 UTC m=+5.236573654 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ttkw" (UniqueName: "kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw") pod "network-check-target-gc8gr" (UID: "772e370c-1a11-44c2-8dc9-3c9e1919df27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:10:51.975841 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.975748    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-24 01:05:50 +0000 UTC" deadline="2028-01-22 21:44:57.784954144 +0000 UTC"
Jul 25 01:10:51.975841 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.975790    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13124h34m5.809168447s"
Jul 25 01:10:51.990453 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.990402    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:10:51.990595 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:51.990534    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:10:51.991009 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.990971    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:51.991121 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:51.991084    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:10:51.991180 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:51.991154    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:10:51.991269 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:51.991245    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:10:53.565630 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:53.565486    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs\") pod \"network-metrics-daemon-87t4z\" (UID: \"6f9c02f7-779b-41bb-8260-3244475a0f20\") " pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:10:53.566125 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:53.565689    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:10:53.566125 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:53.565759    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs podName:6f9c02f7-779b-41bb-8260-3244475a0f20 nodeName:}" failed. No retries permitted until 2026-07-25 01:10:57.565737205 +0000 UTC m=+9.053033875 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs") pod "network-metrics-daemon-87t4z" (UID: "6f9c02f7-779b-41bb-8260-3244475a0f20") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:10:53.666183 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:53.666148    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret\") pod \"global-pull-secret-syncer-bqzgx\" (UID: \"056dc3b3-b95b-4d1f-8606-333625c397a3\") " pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:53.666366 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:53.666325    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 01:10:53.666434 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:53.666384    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret podName:056dc3b3-b95b-4d1f-8606-333625c397a3 nodeName:}" failed. No retries permitted until 2026-07-25 01:10:57.666366109 +0000 UTC m=+9.153662779 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret") pod "global-pull-secret-syncer-bqzgx" (UID: "056dc3b3-b95b-4d1f-8606-333625c397a3") : object "kube-system"/"original-pull-secret" not registered
Jul 25 01:10:53.766902 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:53.766868    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttkw\" (UniqueName: \"kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw\") pod \"network-check-target-gc8gr\" (UID: \"772e370c-1a11-44c2-8dc9-3c9e1919df27\") " pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:10:53.767121 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:53.767105    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 01:10:53.767200 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:53.767131    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 01:10:53.767200 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:53.767144    2576 projected.go:196] Error preparing data for projected volume kube-api-access-8ttkw for pod openshift-network-diagnostics/network-check-target-gc8gr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:10:53.767304 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:53.767202    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw podName:772e370c-1a11-44c2-8dc9-3c9e1919df27 nodeName:}" failed. No retries permitted until 2026-07-25 01:10:57.767183899 +0000 UTC m=+9.254480568 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ttkw" (UniqueName: "kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw") pod "network-check-target-gc8gr" (UID: "772e370c-1a11-44c2-8dc9-3c9e1919df27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:10:53.990967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:53.990229    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:53.990967 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:53.990356    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:10:53.990967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:53.990420    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:10:53.990967 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:53.990507    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:10:53.990967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:53.990546    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:10:53.990967 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:53.990608    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:10:55.990982 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:55.990938    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:10:55.990982 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:55.990969    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:55.991471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:55.990938    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:10:55.991471 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:55.991077    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:10:55.991471 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:55.991176    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:10:55.991471 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:55.991261    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:10:57.602420 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:57.602385    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs\") pod \"network-metrics-daemon-87t4z\" (UID: \"6f9c02f7-779b-41bb-8260-3244475a0f20\") " pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:10:57.602887 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:57.602600    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:10:57.602887 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:57.602691    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs podName:6f9c02f7-779b-41bb-8260-3244475a0f20 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:05.602671379 +0000 UTC m=+17.089968042 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs") pod "network-metrics-daemon-87t4z" (UID: "6f9c02f7-779b-41bb-8260-3244475a0f20") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:10:57.703669 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:57.703615    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret\") pod \"global-pull-secret-syncer-bqzgx\" (UID: \"056dc3b3-b95b-4d1f-8606-333625c397a3\") " pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:57.703846 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:57.703761    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 01:10:57.703846 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:57.703835    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret podName:056dc3b3-b95b-4d1f-8606-333625c397a3 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:05.703814368 +0000 UTC m=+17.191111022 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret") pod "global-pull-secret-syncer-bqzgx" (UID: "056dc3b3-b95b-4d1f-8606-333625c397a3") : object "kube-system"/"original-pull-secret" not registered
Jul 25 01:10:57.805083 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:57.804952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttkw\" (UniqueName: \"kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw\") pod \"network-check-target-gc8gr\" (UID: \"772e370c-1a11-44c2-8dc9-3c9e1919df27\") " pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:10:57.805248 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:57.805115    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 01:10:57.805248 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:57.805141    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 01:10:57.805248 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:57.805155    2576 projected.go:196] Error preparing data for projected volume kube-api-access-8ttkw for pod openshift-network-diagnostics/network-check-target-gc8gr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:10:57.805248 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:57.805225    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw podName:772e370c-1a11-44c2-8dc9-3c9e1919df27 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:05.805206348 +0000 UTC m=+17.292503017 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ttkw" (UniqueName: "kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw") pod "network-check-target-gc8gr" (UID: "772e370c-1a11-44c2-8dc9-3c9e1919df27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:10:57.990218 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:57.990133    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:57.990218 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:57.990158    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:10:57.990447 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:57.990265    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:10:57.990680 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:57.990616    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:10:57.990807 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:57.990736    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:10:57.990878 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:57.990830    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:10:59.990695 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:59.990606    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:10:59.991125 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:59.990613    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:10:59.991125 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:59.990742    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:10:59.991125 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:10:59.990754    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:10:59.991125 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:59.990852    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:10:59.991125 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:10:59.990941    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:11:01.991087 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:01.991036    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:01.991531 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:01.991152    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:01.991531 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:01.991159    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:11:01.991531 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:01.991186    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:01.991531 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:01.991285    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:11:01.991531 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:01.991380    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:11:03.990594 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:03.990557    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:03.991039 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:03.990608    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:03.991039 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:03.990608    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:03.991039 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:03.990694    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:11:03.991039 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:03.990796    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:11:03.991039 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:03.990896    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:11:05.662969 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:05.662934    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs\") pod \"network-metrics-daemon-87t4z\" (UID: \"6f9c02f7-779b-41bb-8260-3244475a0f20\") " pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:05.663397 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:05.663093    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:11:05.663397 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:05.663157    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs podName:6f9c02f7-779b-41bb-8260-3244475a0f20 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:21.663142755 +0000 UTC m=+33.150439411 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs") pod "network-metrics-daemon-87t4z" (UID: "6f9c02f7-779b-41bb-8260-3244475a0f20") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:11:05.763445 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:05.763404    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret\") pod \"global-pull-secret-syncer-bqzgx\" (UID: \"056dc3b3-b95b-4d1f-8606-333625c397a3\") " pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:05.763622 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:05.763565    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 01:11:05.763704 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:05.763641    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret podName:056dc3b3-b95b-4d1f-8606-333625c397a3 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:21.763620844 +0000 UTC m=+33.250917500 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret") pod "global-pull-secret-syncer-bqzgx" (UID: "056dc3b3-b95b-4d1f-8606-333625c397a3") : object "kube-system"/"original-pull-secret" not registered
Jul 25 01:11:05.864765 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:05.864725    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttkw\" (UniqueName: \"kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw\") pod \"network-check-target-gc8gr\" (UID: \"772e370c-1a11-44c2-8dc9-3c9e1919df27\") " pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:05.864933 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:05.864916    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 01:11:05.865008 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:05.864943    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 01:11:05.865008 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:05.864958    2576 projected.go:196] Error preparing data for projected volume kube-api-access-8ttkw for pod openshift-network-diagnostics/network-check-target-gc8gr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:11:05.865100 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:05.865019    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw podName:772e370c-1a11-44c2-8dc9-3c9e1919df27 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:21.865002952 +0000 UTC m=+33.352299609 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ttkw" (UniqueName: "kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw") pod "network-check-target-gc8gr" (UID: "772e370c-1a11-44c2-8dc9-3c9e1919df27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:11:05.990615 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:05.990521    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:05.990615 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:05.990558    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:05.990615 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:05.990558    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:05.990891 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:05.990689    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:11:05.990891 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:05.990825    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:11:05.990995 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:05.990916    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:11:07.990665 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:07.990628    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:07.991098 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:07.990689    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:07.991098 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:07.990628    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:07.991098 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:07.990774    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:11:07.991098 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:07.990864    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:11:07.991098 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:07.990916    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:11:09.990484 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:09.990400    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:09.990484 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:09.990433    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:09.990992 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:09.990538    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:11:09.990992 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:09.990766    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:11:09.990992 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:09.990801    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:09.990992 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:09.990887    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:11:11.181411 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:11.181149    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-swm4p" event={"ID":"58dcfb48-9812-4413-b614-dd5cca8dd63a","Type":"ContainerStarted","Data":"6002a5a653cb7f7aa4a723de320b3454d99d5939bb1c1849758a0d485688f31b"}
Jul 25 01:11:11.182622 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:11.182592    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mzbmt" event={"ID":"d018052d-5c6b-46d5-bac9-aa0171194b90","Type":"ContainerStarted","Data":"68d3caed328a0e5d909e0c9d71bfed80a2f22b615887b2343d051bab61bcd04e"}
Jul 25 01:11:11.184374 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:11.184353    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67" event={"ID":"bf4b06fe-fa8a-4281-8995-d92fb33486fc","Type":"ContainerStarted","Data":"f49b4dc7134937e0a92d04e44854d722db05120cdb730e5613534f1b85651e19"}
Jul 25 01:11:11.184455 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:11.184408    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67" event={"ID":"bf4b06fe-fa8a-4281-8995-d92fb33486fc","Type":"ContainerStarted","Data":"ac4c39062f7af643947178f4e53dc6a263d040ce01ddde62927e0338e552a4b4"}
Jul 25 01:11:11.185555 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:11.185536    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-238.ec2.internal" event={"ID":"1b910c3459c38a928558d492ab28b0ef","Type":"ContainerStarted","Data":"2d35ccb1d2144efa3cc380c6f393063bc7e899d92fe98093d7d11a6f36b76c86"}
Jul 25 01:11:11.199025 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:11.198984    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-swm4p" podStartSLOduration=1.9949520889999999 podStartE2EDuration="22.198972954s" totalImagesPullingTime="20.204020865s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:11.19883061 +0000 UTC m=+22.686127286" watchObservedRunningTime="2026-07-25 01:11:11.198972954 +0000 UTC m=+22.686269629"
Jul 25 01:11:11.210492 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:11.210454    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-238.ec2.internal" podStartSLOduration=21.210443321 podStartE2EDuration="21.210443321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:10:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:11.210208741 +0000 UTC m=+22.697505416" watchObservedRunningTime="2026-07-25 01:11:11.210443321 +0000 UTC m=+22.697739995"
Jul 25 01:11:11.224923 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:11.224641    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mzbmt" podStartSLOduration=2.01055324 podStartE2EDuration="22.224629682s" totalImagesPullingTime="20.214076442s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:11.224521633 +0000 UTC m=+22.711818290" watchObservedRunningTime="2026-07-25 01:11:11.224629682 +0000 UTC m=+22.711926357"
Jul 25 01:11:11.990807 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:11.990609    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:11.990987 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:11.990647    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:11.990987 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:11.990905    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:11:11.990987 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:11.990667    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:11.990987 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:11.990980    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:11:11.991164 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:11.991072    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:11:12.189743 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.189712    2576 generic.go:356] "Generic (PLEG): container finished" podID="026f8bf6-bdcf-47a9-930e-59323c2cb9f3" containerID="4c7ada2a09ea5be23174268759aa9edac0f8814bd0317ccf64abfec453d8bf84" exitCode=0
Jul 25 01:11:12.190559 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.189801    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8srzj" event={"ID":"026f8bf6-bdcf-47a9-930e-59323c2cb9f3","Type":"ContainerDied","Data":"4c7ada2a09ea5be23174268759aa9edac0f8814bd0317ccf64abfec453d8bf84"}
Jul 25 01:11:12.191332 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.191262    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h7kwx" event={"ID":"f8888b32-a7c5-4b96-b48d-133e3491fb81","Type":"ContainerStarted","Data":"39e68692954ed99a0de05bc1050c45c67c62b5a97f8eada936b91da259aeb723"}
Jul 25 01:11:12.192911 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.192878    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pmpt5" event={"ID":"c6ebb66f-5622-481b-ae46-7b3743feec00","Type":"ContainerStarted","Data":"d2618c7e81bdab6c54a9d75b78a2a1e1b6b973c781a83f9a82fe71d3e7843010"}
Jul 25 01:11:12.194416 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.194393    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm" event={"ID":"571e6c8e-9624-4be9-81a7-7a6ae27267ce","Type":"ContainerStarted","Data":"8c5bf63ee8b8c2ca3253f91034a8dd61a9939bec5162068e3e09e23024324091"}
Jul 25 01:11:12.195832 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.195811    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mttfq" event={"ID":"c804a635-6110-4cea-b2d8-5e8660b449f8","Type":"ContainerStarted","Data":"570b621b6ceb5bc4ed3ac149154baf760c54119aab62ea9a0c2a0f9a50b198ab"}
Jul 25 01:11:12.197272 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.197252    2576 generic.go:356] "Generic (PLEG): container finished" podID="a69f653c5364a5c191e807aff8316576" containerID="094fbc9c6fd20d0c4a584c811bf20d9d730e16659515fbea8c49c3ab12ad42f4" exitCode=0
Jul 25 01:11:12.197364 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.197315    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal" event={"ID":"a69f653c5364a5c191e807aff8316576","Type":"ContainerDied","Data":"094fbc9c6fd20d0c4a584c811bf20d9d730e16659515fbea8c49c3ab12ad42f4"}
Jul 25 01:11:12.198808 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.198784    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rgqsm" event={"ID":"1e24f294-fa2e-4cc1-9390-adc201492447","Type":"ContainerStarted","Data":"8931ce8dd7e8ae30640b3acf410b764b3f6ae846856ebacd04840db88a031403"}
Jul 25 01:11:12.201628 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.201601    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:11:12.201994 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.201975    2576 generic.go:356] "Generic (PLEG): container finished" podID="bf4b06fe-fa8a-4281-8995-d92fb33486fc" containerID="f49b4dc7134937e0a92d04e44854d722db05120cdb730e5613534f1b85651e19" exitCode=1
Jul 25 01:11:12.202091 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.202062    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67" event={"ID":"bf4b06fe-fa8a-4281-8995-d92fb33486fc","Type":"ContainerDied","Data":"f49b4dc7134937e0a92d04e44854d722db05120cdb730e5613534f1b85651e19"}
Jul 25 01:11:12.202155 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.202135    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67" event={"ID":"bf4b06fe-fa8a-4281-8995-d92fb33486fc","Type":"ContainerStarted","Data":"ec16a591f7d1587100b2355a6b21e440370d16802132575104724fb144ebdcf5"}
Jul 25 01:11:12.202212 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.202202    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67" event={"ID":"bf4b06fe-fa8a-4281-8995-d92fb33486fc","Type":"ContainerStarted","Data":"665ffb5d8e8b2e0b9321c3c7a76562a3ea60d7b7739f5429bb9c18ae8b508a07"}
Jul 25 01:11:12.202259 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.202239    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67" event={"ID":"bf4b06fe-fa8a-4281-8995-d92fb33486fc","Type":"ContainerStarted","Data":"b759ea3495b80d9f81cfe4ff5a3c54c83efca9ce3f379655e5e4f050719327cc"}
Jul 25 01:11:12.202310 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.202291    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67" event={"ID":"bf4b06fe-fa8a-4281-8995-d92fb33486fc","Type":"ContainerStarted","Data":"2a8160e66dd439eb5770d0d36045c8945176887f965cead027d71f244e57872a"}
Jul 25 01:11:12.232454 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.232415    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rgqsm" podStartSLOduration=3.106768692 podStartE2EDuration="23.232402106s" totalImagesPullingTime="20.125633414s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:12.231929266 +0000 UTC m=+23.719225976" watchObservedRunningTime="2026-07-25 01:11:12.232402106 +0000 UTC m=+23.719698781"
Jul 25 01:11:12.245047 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.245012    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-h7kwx" podStartSLOduration=2.12875231 podStartE2EDuration="22.245000168s" totalImagesPullingTime="20.116247858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:12.244697246 +0000 UTC m=+23.731993922" watchObservedRunningTime="2026-07-25 01:11:12.245000168 +0000 UTC m=+23.732296877"
Jul 25 01:11:12.269413 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.269377    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mttfq" podStartSLOduration=3.070781655 podStartE2EDuration="23.269361587s" totalImagesPullingTime="20.198579932s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:12.257526396 +0000 UTC m=+23.744823072" watchObservedRunningTime="2026-07-25 01:11:12.269361587 +0000 UTC m=+23.756658295"
Jul 25 01:11:12.269529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.269439    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pmpt5" podStartSLOduration=3.097210631 podStartE2EDuration="23.26943549s" totalImagesPullingTime="20.172224859s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:12.2692754 +0000 UTC m=+23.756572074" watchObservedRunningTime="2026-07-25 01:11:12.26943549 +0000 UTC m=+23.756732165"
Jul 25 01:11:12.631733 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.631702    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-h7kwx"
Jul 25 01:11:12.632260 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.632243    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-h7kwx"
Jul 25 01:11:12.749037 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.749018    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 25 01:11:12.991706 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.991554    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-25T01:11:12.749033406Z","UUID":"ec954d8f-5208-4b86-a2d1-69327a29e750","Handler":null,"Name":"","Endpoint":""}
Jul 25 01:11:12.993108 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.993074    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 25 01:11:12.993206 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:12.993114    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 25 01:11:13.207028 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:13.206993    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm" event={"ID":"571e6c8e-9624-4be9-81a7-7a6ae27267ce","Type":"ContainerStarted","Data":"9b6f49203b12cfec5660b6aa25d4486bad6db775297a5cfebfa3ad8b561a3894"}
Jul 25 01:11:13.208990 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:13.208951    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal" event={"ID":"a69f653c5364a5c191e807aff8316576","Type":"ContainerStarted","Data":"5e4d303084a92e709750487adee2770fb472472a024258c7bf27ce92262c0259"}
Jul 25 01:11:13.224586 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:13.224539    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-238.ec2.internal" podStartSLOduration=23.22452437 podStartE2EDuration="23.22452437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:10:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:13.224125263 +0000 UTC m=+24.711421939" watchObservedRunningTime="2026-07-25 01:11:13.22452437 +0000 UTC m=+24.711821045"
Jul 25 01:11:13.990318 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:13.990244    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:13.990318 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:13.990292    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:13.990561 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:13.990343    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:13.990561 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:13.990356    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:11:13.990561 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:13.990449    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:11:13.990561 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:13.990529    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:11:14.213109 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:14.212884    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm" event={"ID":"571e6c8e-9624-4be9-81a7-7a6ae27267ce","Type":"ContainerStarted","Data":"6f72250f7c537bd5226a443c8de61fcf94588a268bdd664acb9a58900030a23d"}
Jul 25 01:11:14.215846 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:14.215827    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:11:14.216161 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:14.216143    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67" event={"ID":"bf4b06fe-fa8a-4281-8995-d92fb33486fc","Type":"ContainerStarted","Data":"05306cbbacee0eba1d3dc0707ccf7f81184716c0f902e45de865c96bb75454d0"}
Jul 25 01:11:14.216367 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:14.216356    2576 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 25 01:11:14.229224 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:14.229184    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8gdm" podStartSLOduration=2.543607841 podStartE2EDuration="25.2291719s" totalImagesPullingTime="22.685564059s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:10:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:14.228899075 +0000 UTC m=+25.716195749" watchObservedRunningTime="2026-07-25 01:11:14.2291719 +0000 UTC m=+25.716468574"
Jul 25 01:11:15.835835 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:15.835801    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-h7kwx"
Jul 25 01:11:15.836314 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:15.835975    2576 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 25 01:11:15.836466 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:15.836453    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-h7kwx"
Jul 25 01:11:15.991040 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:15.991010    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:15.991040 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:15.991032    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:15.991248 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:15.991024    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:15.991248 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:15.991142    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:11:15.991352 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:15.991262    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:11:15.991393 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:15.991358    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:11:17.227017 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:17.226845    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:11:17.227634 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:17.227382    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67" event={"ID":"bf4b06fe-fa8a-4281-8995-d92fb33486fc","Type":"ContainerStarted","Data":"a0f274f472ca94126bb1598a769f8646c3de7ca9dd742dfad4e90dc7951763d6"}
Jul 25 01:11:17.227998 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:17.227822    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:11:17.227998 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:17.227898    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:11:17.227998 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:17.227948    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:11:17.227998 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:17.227981    2576 scope.go:117] "RemoveContainer" containerID="f49b4dc7134937e0a92d04e44854d722db05120cdb730e5613534f1b85651e19"
Jul 25 01:11:17.247307 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:17.247283    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:11:17.247726 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:17.247704    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:11:17.990647 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:17.990616    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:17.990647 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:17.990616    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:17.990893 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:17.990639    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:17.990893 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:17.990733    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:11:17.990893 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:17.990806    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:11:17.990893 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:17.990856    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:11:18.243592 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:18.243560    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:11:18.244064 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:18.244022    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67" event={"ID":"bf4b06fe-fa8a-4281-8995-d92fb33486fc","Type":"ContainerStarted","Data":"83c7496dbb47ed15d2a19ccf55672dde119156aeeec00363f5730dab3bd7e54f"}
Jul 25 01:11:18.247079 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:18.247051    2576 generic.go:356] "Generic (PLEG): container finished" podID="026f8bf6-bdcf-47a9-930e-59323c2cb9f3" containerID="8d2d1e3e604b5c03c877d8a4145f3cbeb88b34d21be44f0555a0b9d781cad780" exitCode=0
Jul 25 01:11:18.247191 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:18.247105    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8srzj" event={"ID":"026f8bf6-bdcf-47a9-930e-59323c2cb9f3","Type":"ContainerDied","Data":"8d2d1e3e604b5c03c877d8a4145f3cbeb88b34d21be44f0555a0b9d781cad780"}
Jul 25 01:11:18.293136 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:18.293088    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67" podStartSLOduration=8.591350773 podStartE2EDuration="29.293071434s" totalImagesPullingTime="20.701720661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:18.273473917 +0000 UTC m=+29.760770616" watchObservedRunningTime="2026-07-25 01:11:18.293071434 +0000 UTC m=+29.780368108"
Jul 25 01:11:19.056110 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:19.056078    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gc8gr"]
Jul 25 01:11:19.056297 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:19.056188    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:19.056363 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:19.056290    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:11:19.058507 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:19.058484    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bqzgx"]
Jul 25 01:11:19.058613 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:19.058552    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:19.058703 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:19.058626    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:11:19.067495 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:19.067469    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-87t4z"]
Jul 25 01:11:19.067581 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:19.067563    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:19.067693 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:19.067672    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:11:20.253490 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:20.253407    2576 generic.go:356] "Generic (PLEG): container finished" podID="026f8bf6-bdcf-47a9-930e-59323c2cb9f3" containerID="752baf4d7e4e069053e02fc38c35efd1386fb32b96133526853587605f8afb4f" exitCode=0
Jul 25 01:11:20.253490 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:20.253473    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8srzj" event={"ID":"026f8bf6-bdcf-47a9-930e-59323c2cb9f3","Type":"ContainerDied","Data":"752baf4d7e4e069053e02fc38c35efd1386fb32b96133526853587605f8afb4f"}
Jul 25 01:11:20.990887 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:20.990676    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:20.991042 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:20.990676    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:20.991042 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:20.990933    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:11:20.991042 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:20.990716    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:20.991189 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:20.990983    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:11:20.991189 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:20.991111    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:11:21.257320 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:21.257262    2576 generic.go:356] "Generic (PLEG): container finished" podID="026f8bf6-bdcf-47a9-930e-59323c2cb9f3" containerID="4ec6310210765454805b420b79bebe1942ee5ee5eae7d1d5290edc0773c4f8b2" exitCode=0
Jul 25 01:11:21.257584 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:21.257331    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8srzj" event={"ID":"026f8bf6-bdcf-47a9-930e-59323c2cb9f3","Type":"ContainerDied","Data":"4ec6310210765454805b420b79bebe1942ee5ee5eae7d1d5290edc0773c4f8b2"}
Jul 25 01:11:21.690424 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:21.690399    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs\") pod \"network-metrics-daemon-87t4z\" (UID: \"6f9c02f7-779b-41bb-8260-3244475a0f20\") " pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:21.690584 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:21.690549    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:11:21.690633 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:21.690604    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs podName:6f9c02f7-779b-41bb-8260-3244475a0f20 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:53.69058985 +0000 UTC m=+65.177886503 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs") pod "network-metrics-daemon-87t4z" (UID: "6f9c02f7-779b-41bb-8260-3244475a0f20") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:11:21.790949 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:21.790920    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret\") pod \"global-pull-secret-syncer-bqzgx\" (UID: \"056dc3b3-b95b-4d1f-8606-333625c397a3\") " pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:21.791066 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:21.791018    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 01:11:21.791066 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:21.791065    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret podName:056dc3b3-b95b-4d1f-8606-333625c397a3 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:53.791052187 +0000 UTC m=+65.278348841 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret") pod "global-pull-secret-syncer-bqzgx" (UID: "056dc3b3-b95b-4d1f-8606-333625c397a3") : object "kube-system"/"original-pull-secret" not registered
Jul 25 01:11:21.892269 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:21.892232    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttkw\" (UniqueName: \"kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw\") pod \"network-check-target-gc8gr\" (UID: \"772e370c-1a11-44c2-8dc9-3c9e1919df27\") " pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:21.892503 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:21.892387    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 01:11:21.892503 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:21.892407    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 01:11:21.892503 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:21.892418    2576 projected.go:196] Error preparing data for projected volume kube-api-access-8ttkw for pod openshift-network-diagnostics/network-check-target-gc8gr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:11:21.892503 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:21.892466    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw podName:772e370c-1a11-44c2-8dc9-3c9e1919df27 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:53.892452075 +0000 UTC m=+65.379748728 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8ttkw" (UniqueName: "kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw") pod "network-check-target-gc8gr" (UID: "772e370c-1a11-44c2-8dc9-3c9e1919df27") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:11:22.990142 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:22.990109    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:22.990593 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:22.990109    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:22.990593 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:22.990236    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:22.990593 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:22.990233    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bqzgx" podUID="056dc3b3-b95b-4d1f-8606-333625c397a3"
Jul 25 01:11:22.990593 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:22.990324    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87t4z" podUID="6f9c02f7-779b-41bb-8260-3244475a0f20"
Jul 25 01:11:22.990593 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:22.990391    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gc8gr" podUID="772e370c-1a11-44c2-8dc9-3c9e1919df27"
Jul 25 01:11:24.345570 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.345537    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-238.ec2.internal" event="NodeReady"
Jul 25 01:11:24.346006 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.345703    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 25 01:11:24.389117 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.389093    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-95f54d94c-n7p57"]
Jul 25 01:11:24.392611 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.392587    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn"]
Jul 25 01:11:24.392734 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.392720    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.394834 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.394811    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 25 01:11:24.394834 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.394820    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 25 01:11:24.395003 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.394840    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j7vlc\""
Jul 25 01:11:24.395003 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.394976    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-59l68"]
Jul 25 01:11:24.395128 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.395109    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn"
Jul 25 01:11:24.395359 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.395114    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 25 01:11:24.397203 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.397184    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-28cvc"]
Jul 25 01:11:24.397913 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.397796    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-59l68"
Jul 25 01:11:24.397913 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.397906    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 25 01:11:24.398342 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.398322    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-f6zpp\""
Jul 25 01:11:24.398786 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.398764    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 25 01:11:24.399368 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.399350    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 25 01:11:24.401164 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.401070    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-64pv8\""
Jul 25 01:11:24.401245 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.401197    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 25 01:11:24.401450 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.401423    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb"]
Jul 25 01:11:24.401556 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.401532    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 25 01:11:24.401824 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.401809    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:24.404244 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.404220    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 25 01:11:24.404351 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.404260    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 25 01:11:24.404410 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.404371    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-7r7lb\""
Jul 25 01:11:24.404504 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.404472    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 25 01:11:24.405060 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.404691    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 25 01:11:24.405156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.404720    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 25 01:11:24.405745 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.405722    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x"]
Jul 25 01:11:24.405913 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.405898    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb"
Jul 25 01:11:24.408826 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.408809    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x"
Jul 25 01:11:24.413360 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.413266    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 25 01:11:24.413519 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.413500    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-58gqm\""
Jul 25 01:11:24.413901 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.413880    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 25 01:11:24.414516 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.414495    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 25 01:11:24.414594 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.414561    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 25 01:11:24.414757 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.414740    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn"]
Jul 25 01:11:24.414840 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.414781    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 25 01:11:24.414961 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.414930    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-tx6s5\""
Jul 25 01:11:24.415164 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.415061    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 25 01:11:24.415233 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.415178    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 25 01:11:24.415294 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.415261    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 25 01:11:24.415538 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.415487    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-95f54d94c-n7p57"]
Jul 25 01:11:24.416354 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.416308    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-28cvc"]
Jul 25 01:11:24.417268 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.417215    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb"]
Jul 25 01:11:24.417905 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.417887    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-bv82n"]
Jul 25 01:11:24.419218 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.419184    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 25 01:11:24.420683 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.420588    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tqwn9"]
Jul 25 01:11:24.420778 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.420691    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-bv82n"
Jul 25 01:11:24.423010 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.422989    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 25 01:11:24.423101 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.423043    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 25 01:11:24.423101 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.423070    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-wrcvc\""
Jul 25 01:11:24.423492 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.423470    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:24.426558 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.426539    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4svzl\""
Jul 25 01:11:24.426699 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.426600    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 25 01:11:24.426876 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.426859    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 25 01:11:24.428268 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.428249    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-59l68"]
Jul 25 01:11:24.429933 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.429917    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x"]
Jul 25 01:11:24.431622 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.431607    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jhxl9"]
Jul 25 01:11:24.436191 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.435925    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tqwn9"]
Jul 25 01:11:24.436191 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.435968    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-bv82n"]
Jul 25 01:11:24.436191 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.436051    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jhxl9"
Jul 25 01:11:24.438239 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.438217    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 25 01:11:24.438631 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.438614    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 25 01:11:24.438889 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.438871    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 25 01:11:24.439161 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.439138    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-srmhs\""
Jul 25 01:11:24.458499 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.458476    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jhxl9"]
Jul 25 01:11:24.515459 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.515371    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/62b7cd6a-9bb4-4182-9c2b-db95bf27faca-config\") pod \"service-ca-operator-5d6c75f84f-tr4jb\" (UID: \"62b7cd6a-9bb4-4182-9c2b-db95bf27faca\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb"
Jul 25 01:11:24.515459 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.515411    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aa437a19-20d8-4209-9886-7c07c2b5bf75-config-volume\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:24.515684 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.515461    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b66rx\" (UniqueName: \"kubernetes.io/projected/f85eed90-9183-4673-8d7f-cf7279d53a71-kube-api-access-b66rx\") pod \"network-check-source-7d88cb55d8-bv82n\" (UID: \"f85eed90-9183-4673-8d7f-cf7279d53a71\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-bv82n"
Jul 25 01:11:24.515684 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.515491    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-bound-sa-token\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.515684 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.515511    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6acccfec-8c59-4af6-af57-fd9445830798-trusted-ca\") pod \"console-operator-584dcb6b54-28cvc\" (UID: \"6acccfec-8c59-4af6-af57-fd9445830798\") " pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:24.515684 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.515588    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kww8\" (UniqueName: \"kubernetes.io/projected/6acccfec-8c59-4af6-af57-fd9445830798-kube-api-access-8kww8\") pod \"console-operator-584dcb6b54-28cvc\" (UID: \"6acccfec-8c59-4af6-af57-fd9445830798\") " pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:24.515684 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.515624    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-ca-trust-extracted\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.515684 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.515673    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-certificates\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.515989 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.515696    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-trusted-ca\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.515989 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.515736    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6acccfec-8c59-4af6-af57-fd9445830798-config\") pod \"console-operator-584dcb6b54-28cvc\" (UID: \"6acccfec-8c59-4af6-af57-fd9445830798\") " pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:24.515989 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.515757    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdtjt\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-kube-api-access-jdtjt\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.515989 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.515779    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmhcq\" (UniqueName: \"kubernetes.io/projected/405e24e0-1385-4056-82c3-bf7eca2bd7e4-kube-api-access-wmhcq\") pod \"volume-data-source-validator-549c6d54cf-59l68\" (UID: \"405e24e0-1385-4056-82c3-bf7eca2bd7e4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-59l68"
Jul 25 01:11:24.515989 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.515807    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kth9j\" (UniqueName: \"kubernetes.io/projected/aa437a19-20d8-4209-9886-7c07c2b5bf75-kube-api-access-kth9j\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:24.515989 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.515860    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6acccfec-8c59-4af6-af57-fd9445830798-serving-cert\") pod \"console-operator-584dcb6b54-28cvc\" (UID: \"6acccfec-8c59-4af6-af57-fd9445830798\") " pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:24.515989 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.515884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hvpgn\" (UID: \"e6b09e76-4845-4198-8dcd-b6de8642a958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn"
Jul 25 01:11:24.515989 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.515908    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqzww\" (UniqueName: \"kubernetes.io/projected/62b7cd6a-9bb4-4182-9c2b-db95bf27faca-kube-api-access-tqzww\") pod \"service-ca-operator-5d6c75f84f-tr4jb\" (UID: \"62b7cd6a-9bb4-4182-9c2b-db95bf27faca\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb"
Jul 25 01:11:24.516343 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.515997    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq587\" (UniqueName: \"kubernetes.io/projected/860680d1-a426-4fea-9c8f-892d44983420-kube-api-access-dq587\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x\" (UID: \"860680d1-a426-4fea-9c8f-892d44983420\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x"
Jul 25 01:11:24.516343 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.516037    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/62b7cd6a-9bb4-4182-9c2b-db95bf27faca-serving-cert\") pod \"service-ca-operator-5d6c75f84f-tr4jb\" (UID: \"62b7cd6a-9bb4-4182-9c2b-db95bf27faca\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb"
Jul 25 01:11:24.516343 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.516065    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:24.516343 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.516095    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa437a19-20d8-4209-9886-7c07c2b5bf75-tmp-dir\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:24.516343 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.516159    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/860680d1-a426-4fea-9c8f-892d44983420-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x\" (UID: \"860680d1-a426-4fea-9c8f-892d44983420\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x"
Jul 25 01:11:24.516343 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.516197    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.516343 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.516230    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-installation-pull-secrets\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.516343 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.516254    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8hwm\" (UniqueName: \"kubernetes.io/projected/e6b09e76-4845-4198-8dcd-b6de8642a958-kube-api-access-x8hwm\") pod \"cluster-samples-operator-6748489656-hvpgn\" (UID: \"e6b09e76-4845-4198-8dcd-b6de8642a958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn"
Jul 25 01:11:24.516343 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.516318    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/860680d1-a426-4fea-9c8f-892d44983420-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x\" (UID: \"860680d1-a426-4fea-9c8f-892d44983420\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x"
Jul 25 01:11:24.516715 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.516355    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-image-registry-private-configuration\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.617014 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.616983    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aa437a19-20d8-4209-9886-7c07c2b5bf75-config-volume\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:24.617205 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.617026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:24.617205 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.617055    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/62b7cd6a-9bb4-4182-9c2b-db95bf27faca-serving-cert\") pod \"service-ca-operator-5d6c75f84f-tr4jb\" (UID: \"62b7cd6a-9bb4-4182-9c2b-db95bf27faca\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb"
Jul 25 01:11:24.617205 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.617075    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/860680d1-a426-4fea-9c8f-892d44983420-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x\" (UID: \"860680d1-a426-4fea-9c8f-892d44983420\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x"
Jul 25 01:11:24.617205 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.617149    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8hwm\" (UniqueName: \"kubernetes.io/projected/e6b09e76-4845-4198-8dcd-b6de8642a958-kube-api-access-x8hwm\") pod \"cluster-samples-operator-6748489656-hvpgn\" (UID: \"e6b09e76-4845-4198-8dcd-b6de8642a958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn"
Jul 25 01:11:24.617205 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:24.617169    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 01:11:24.617205 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.617188    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0b30ace-19f7-4c3c-949e-7526c7a69d90-cert\") pod \"ingress-canary-jhxl9\" (UID: \"a0b30ace-19f7-4c3c-949e-7526c7a69d90\") " pod="openshift-ingress-canary/ingress-canary-jhxl9"
Jul 25 01:11:24.617486 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:24.617251    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls podName:aa437a19-20d8-4209-9886-7c07c2b5bf75 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:25.117227767 +0000 UTC m=+36.604524425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls") pod "dns-default-tqwn9" (UID: "aa437a19-20d8-4209-9886-7c07c2b5bf75") : secret "dns-default-metrics-tls" not found
Jul 25 01:11:24.617486 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.617411    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-bound-sa-token\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.617486 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.617441    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6acccfec-8c59-4af6-af57-fd9445830798-trusted-ca\") pod \"console-operator-584dcb6b54-28cvc\" (UID: \"6acccfec-8c59-4af6-af57-fd9445830798\") " pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:24.617486 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.617473    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kww8\" (UniqueName: \"kubernetes.io/projected/6acccfec-8c59-4af6-af57-fd9445830798-kube-api-access-8kww8\") pod \"console-operator-584dcb6b54-28cvc\" (UID: \"6acccfec-8c59-4af6-af57-fd9445830798\") " pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:24.617728 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.617490    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-ca-trust-extracted\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.617728 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.617508    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-certificates\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.617728 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.617524    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-trusted-ca\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.617728 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.617639    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aa437a19-20d8-4209-9886-7c07c2b5bf75-config-volume\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:24.618012 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.617994    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-ca-trust-extracted\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.618071 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618043    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdtjt\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-kube-api-access-jdtjt\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.618127 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618074    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmhcq\" (UniqueName: \"kubernetes.io/projected/405e24e0-1385-4056-82c3-bf7eca2bd7e4-kube-api-access-wmhcq\") pod \"volume-data-source-validator-549c6d54cf-59l68\" (UID: \"405e24e0-1385-4056-82c3-bf7eca2bd7e4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-59l68"
Jul 25 01:11:24.618127 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-image-registry-private-configuration\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.618223 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618153    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6acccfec-8c59-4af6-af57-fd9445830798-serving-cert\") pod \"console-operator-584dcb6b54-28cvc\" (UID: \"6acccfec-8c59-4af6-af57-fd9445830798\") " pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:24.618223 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618180    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hvpgn\" (UID: \"e6b09e76-4845-4198-8dcd-b6de8642a958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn"
Jul 25 01:11:24.618223 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618204    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqzww\" (UniqueName: \"kubernetes.io/projected/62b7cd6a-9bb4-4182-9c2b-db95bf27faca-kube-api-access-tqzww\") pod \"service-ca-operator-5d6c75f84f-tr4jb\" (UID: \"62b7cd6a-9bb4-4182-9c2b-db95bf27faca\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb"
Jul 25 01:11:24.618223 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618212    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-certificates\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.618408 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618232    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa437a19-20d8-4209-9886-7c07c2b5bf75-tmp-dir\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:24.618408 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618263    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52fhq\" (UniqueName: \"kubernetes.io/projected/a0b30ace-19f7-4c3c-949e-7526c7a69d90-kube-api-access-52fhq\") pod \"ingress-canary-jhxl9\" (UID: \"a0b30ace-19f7-4c3c-949e-7526c7a69d90\") " pod="openshift-ingress-canary/ingress-canary-jhxl9"
Jul 25 01:11:24.618408 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618291    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq587\" (UniqueName: \"kubernetes.io/projected/860680d1-a426-4fea-9c8f-892d44983420-kube-api-access-dq587\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x\" (UID: \"860680d1-a426-4fea-9c8f-892d44983420\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x"
Jul 25 01:11:24.618408 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618319    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.618408 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618345    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-installation-pull-secrets\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.618408 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618388    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b66rx\" (UniqueName: \"kubernetes.io/projected/f85eed90-9183-4673-8d7f-cf7279d53a71-kube-api-access-b66rx\") pod \"network-check-source-7d88cb55d8-bv82n\" (UID: \"f85eed90-9183-4673-8d7f-cf7279d53a71\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-bv82n"
Jul 25 01:11:24.618408 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618393    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6acccfec-8c59-4af6-af57-fd9445830798-trusted-ca\") pod \"console-operator-584dcb6b54-28cvc\" (UID: \"6acccfec-8c59-4af6-af57-fd9445830798\") " pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:24.618741 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618446    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6acccfec-8c59-4af6-af57-fd9445830798-config\") pod \"console-operator-584dcb6b54-28cvc\" (UID: \"6acccfec-8c59-4af6-af57-fd9445830798\") " pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:24.618741 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618471    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kth9j\" (UniqueName: \"kubernetes.io/projected/aa437a19-20d8-4209-9886-7c07c2b5bf75-kube-api-access-kth9j\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:24.618741 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618500    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/860680d1-a426-4fea-9c8f-892d44983420-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x\" (UID: \"860680d1-a426-4fea-9c8f-892d44983420\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x"
Jul 25 01:11:24.618741 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:24.618504    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 25 01:11:24.618741 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618524    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/62b7cd6a-9bb4-4182-9c2b-db95bf27faca-config\") pod \"service-ca-operator-5d6c75f84f-tr4jb\" (UID: \"62b7cd6a-9bb4-4182-9c2b-db95bf27faca\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb"
Jul 25 01:11:24.618741 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:24.618602    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls podName:e6b09e76-4845-4198-8dcd-b6de8642a958 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:25.118586076 +0000 UTC m=+36.605882745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls") pod "cluster-samples-operator-6748489656-hvpgn" (UID: "e6b09e76-4845-4198-8dcd-b6de8642a958") : secret "samples-operator-tls" not found
Jul 25 01:11:24.619021 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618781    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-trusted-ca\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.619021 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.618780    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa437a19-20d8-4209-9886-7c07c2b5bf75-tmp-dir\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:24.619021 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:24.618988    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 25 01:11:24.619021 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:24.619001    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-95f54d94c-n7p57: secret "image-registry-tls" not found
Jul 25 01:11:24.619202 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:24.619048    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls podName:5f4be3b2-eda3-4d6b-be6d-9806c73138ae nodeName:}" failed. No retries permitted until 2026-07-25 01:11:25.119035385 +0000 UTC m=+36.606332055 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls") pod "image-registry-95f54d94c-n7p57" (UID: "5f4be3b2-eda3-4d6b-be6d-9806c73138ae") : secret "image-registry-tls" not found
Jul 25 01:11:24.619202 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.619086    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/62b7cd6a-9bb4-4182-9c2b-db95bf27faca-config\") pod \"service-ca-operator-5d6c75f84f-tr4jb\" (UID: \"62b7cd6a-9bb4-4182-9c2b-db95bf27faca\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb"
Jul 25 01:11:24.619576 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.619535    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/860680d1-a426-4fea-9c8f-892d44983420-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x\" (UID: \"860680d1-a426-4fea-9c8f-892d44983420\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x"
Jul 25 01:11:24.619707 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.619683    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6acccfec-8c59-4af6-af57-fd9445830798-config\") pod \"console-operator-584dcb6b54-28cvc\" (UID: \"6acccfec-8c59-4af6-af57-fd9445830798\") " pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:24.622550 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.622508    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/860680d1-a426-4fea-9c8f-892d44983420-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x\" (UID: \"860680d1-a426-4fea-9c8f-892d44983420\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x"
Jul 25 01:11:24.622673 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.622581    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/62b7cd6a-9bb4-4182-9c2b-db95bf27faca-serving-cert\") pod \"service-ca-operator-5d6c75f84f-tr4jb\" (UID: \"62b7cd6a-9bb4-4182-9c2b-db95bf27faca\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb"
Jul 25 01:11:24.622730 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.622710    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-installation-pull-secrets\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.622764 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.622730    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-image-registry-private-configuration\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.622835 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.622805    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6acccfec-8c59-4af6-af57-fd9445830798-serving-cert\") pod \"console-operator-584dcb6b54-28cvc\" (UID: \"6acccfec-8c59-4af6-af57-fd9445830798\") " pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:24.627345 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.627319    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-bound-sa-token\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.627532 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.627478    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kww8\" (UniqueName: \"kubernetes.io/projected/6acccfec-8c59-4af6-af57-fd9445830798-kube-api-access-8kww8\") pod \"console-operator-584dcb6b54-28cvc\" (UID: \"6acccfec-8c59-4af6-af57-fd9445830798\") " pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:24.627740 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.627716    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq587\" (UniqueName: \"kubernetes.io/projected/860680d1-a426-4fea-9c8f-892d44983420-kube-api-access-dq587\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x\" (UID: \"860680d1-a426-4fea-9c8f-892d44983420\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x"
Jul 25 01:11:24.627880 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.627863    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8hwm\" (UniqueName: \"kubernetes.io/projected/e6b09e76-4845-4198-8dcd-b6de8642a958-kube-api-access-x8hwm\") pod \"cluster-samples-operator-6748489656-hvpgn\" (UID: \"e6b09e76-4845-4198-8dcd-b6de8642a958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn"
Jul 25 01:11:24.627940 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.627865    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqzww\" (UniqueName: \"kubernetes.io/projected/62b7cd6a-9bb4-4182-9c2b-db95bf27faca-kube-api-access-tqzww\") pod \"service-ca-operator-5d6c75f84f-tr4jb\" (UID: \"62b7cd6a-9bb4-4182-9c2b-db95bf27faca\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb"
Jul 25 01:11:24.628588 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.628540    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmhcq\" (UniqueName: \"kubernetes.io/projected/405e24e0-1385-4056-82c3-bf7eca2bd7e4-kube-api-access-wmhcq\") pod \"volume-data-source-validator-549c6d54cf-59l68\" (UID: \"405e24e0-1385-4056-82c3-bf7eca2bd7e4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-59l68"
Jul 25 01:11:24.629221 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.629200    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdtjt\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-kube-api-access-jdtjt\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:24.629766 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.629748    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b66rx\" (UniqueName: \"kubernetes.io/projected/f85eed90-9183-4673-8d7f-cf7279d53a71-kube-api-access-b66rx\") pod \"network-check-source-7d88cb55d8-bv82n\" (UID: \"f85eed90-9183-4673-8d7f-cf7279d53a71\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-bv82n"
Jul 25 01:11:24.630551 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.630531    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kth9j\" (UniqueName: \"kubernetes.io/projected/aa437a19-20d8-4209-9886-7c07c2b5bf75-kube-api-access-kth9j\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:24.719021 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.718988    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0b30ace-19f7-4c3c-949e-7526c7a69d90-cert\") pod \"ingress-canary-jhxl9\" (UID: \"a0b30ace-19f7-4c3c-949e-7526c7a69d90\") " pod="openshift-ingress-canary/ingress-canary-jhxl9"
Jul 25 01:11:24.719182 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.719078    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52fhq\" (UniqueName: \"kubernetes.io/projected/a0b30ace-19f7-4c3c-949e-7526c7a69d90-kube-api-access-52fhq\") pod \"ingress-canary-jhxl9\" (UID: \"a0b30ace-19f7-4c3c-949e-7526c7a69d90\") " pod="openshift-ingress-canary/ingress-canary-jhxl9"
Jul 25 01:11:24.721558 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.721528    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0b30ace-19f7-4c3c-949e-7526c7a69d90-cert\") pod \"ingress-canary-jhxl9\" (UID: \"a0b30ace-19f7-4c3c-949e-7526c7a69d90\") " pod="openshift-ingress-canary/ingress-canary-jhxl9"
Jul 25 01:11:24.725328 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.725305    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-59l68"
Jul 25 01:11:24.730792 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.730774    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52fhq\" (UniqueName: \"kubernetes.io/projected/a0b30ace-19f7-4c3c-949e-7526c7a69d90-kube-api-access-52fhq\") pod \"ingress-canary-jhxl9\" (UID: \"a0b30ace-19f7-4c3c-949e-7526c7a69d90\") " pod="openshift-ingress-canary/ingress-canary-jhxl9"
Jul 25 01:11:24.734562 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.734545    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:24.743009 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.742989    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb"
Jul 25 01:11:24.750134 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.750118    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x"
Jul 25 01:11:24.758216 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.758199    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-bv82n"
Jul 25 01:11:24.771425 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.771381    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jhxl9"
Jul 25 01:11:24.990983 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.990948    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:24.990983 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.990983    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:24.991190 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.990948    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:24.993120 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.993095    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 25 01:11:24.993248 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.993118    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f4ttp\""
Jul 25 01:11:24.993454 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.993337    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 25 01:11:24.993454 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:24.993414    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-djkst\""
Jul 25 01:11:25.123092 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.123011    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hvpgn\" (UID: \"e6b09e76-4845-4198-8dcd-b6de8642a958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn"
Jul 25 01:11:25.123092 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.123051    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:25.123342 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:25.123178    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 25 01:11:25.123342 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.123212    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:25.123342 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:25.123291    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls podName:e6b09e76-4845-4198-8dcd-b6de8642a958 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:26.123264443 +0000 UTC m=+37.610561102 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls") pod "cluster-samples-operator-6748489656-hvpgn" (UID: "e6b09e76-4845-4198-8dcd-b6de8642a958") : secret "samples-operator-tls" not found
Jul 25 01:11:25.123342 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:25.123180    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 25 01:11:25.123342 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:25.123311    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-95f54d94c-n7p57: secret "image-registry-tls" not found
Jul 25 01:11:25.123342 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:25.123345    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls podName:5f4be3b2-eda3-4d6b-be6d-9806c73138ae nodeName:}" failed. No retries permitted until 2026-07-25 01:11:26.123334451 +0000 UTC m=+37.610631110 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls") pod "image-registry-95f54d94c-n7p57" (UID: "5f4be3b2-eda3-4d6b-be6d-9806c73138ae") : secret "image-registry-tls" not found
Jul 25 01:11:25.123613 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:25.123351    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 01:11:25.123613 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:25.123423    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls podName:aa437a19-20d8-4209-9886-7c07c2b5bf75 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:26.123402295 +0000 UTC m=+37.610698953 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls") pod "dns-default-tqwn9" (UID: "aa437a19-20d8-4209-9886-7c07c2b5bf75") : secret "dns-default-metrics-tls" not found
Jul 25 01:11:25.634041 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.634011    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-52w8s"]
Jul 25 01:11:25.636954 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.636936    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-52w8s"
Jul 25 01:11:25.638834 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.638807    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 25 01:11:25.638968 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.638947    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-nlp45\""
Jul 25 01:11:25.639169 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.639146    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 25 01:11:25.644639 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.644587    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-52w8s"]
Jul 25 01:11:25.830986 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.830937    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0b0f3243-f94a-47e2-a884-3ae235a03f07-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-52w8s\" (UID: \"0b0f3243-f94a-47e2-a884-3ae235a03f07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-52w8s"
Jul 25 01:11:25.831157 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.830988    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-52w8s\" (UID: \"0b0f3243-f94a-47e2-a884-3ae235a03f07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-52w8s"
Jul 25 01:11:25.913914 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.913892    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-59l68"]
Jul 25 01:11:25.920323 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.920262    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb"]
Jul 25 01:11:25.920323 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:11:25.920316    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod405e24e0_1385_4056_82c3_bf7eca2bd7e4.slice/crio-85dd65043cf94ac6c2d5ca1250d1adfbeab7f94da579a18c2faf6671639d7192 WatchSource:0}: Error finding container 85dd65043cf94ac6c2d5ca1250d1adfbeab7f94da579a18c2faf6671639d7192: Status 404 returned error can't find the container with id 85dd65043cf94ac6c2d5ca1250d1adfbeab7f94da579a18c2faf6671639d7192
Jul 25 01:11:25.924190 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:11:25.924167    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b7cd6a_9bb4_4182_9c2b_db95bf27faca.slice/crio-58c6f6951cd22a7f676845c35ad315c609e158963a5daf346cd388b7a2d6a51d WatchSource:0}: Error finding container 58c6f6951cd22a7f676845c35ad315c609e158963a5daf346cd388b7a2d6a51d: Status 404 returned error can't find the container with id 58c6f6951cd22a7f676845c35ad315c609e158963a5daf346cd388b7a2d6a51d
Jul 25 01:11:25.928199 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.928178    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-28cvc"]
Jul 25 01:11:25.930703 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:11:25.930643    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6acccfec_8c59_4af6_af57_fd9445830798.slice/crio-9a83dfda0a8cc772fdb1d04d4f39cc0fda4a0614b6fd9bc80d5adf8eeaac9438 WatchSource:0}: Error finding container 9a83dfda0a8cc772fdb1d04d4f39cc0fda4a0614b6fd9bc80d5adf8eeaac9438: Status 404 returned error can't find the container with id 9a83dfda0a8cc772fdb1d04d4f39cc0fda4a0614b6fd9bc80d5adf8eeaac9438
Jul 25 01:11:25.931750 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.931646    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0b0f3243-f94a-47e2-a884-3ae235a03f07-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-52w8s\" (UID: \"0b0f3243-f94a-47e2-a884-3ae235a03f07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-52w8s"
Jul 25 01:11:25.931833 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.931775    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-52w8s\" (UID: \"0b0f3243-f94a-47e2-a884-3ae235a03f07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-52w8s"
Jul 25 01:11:25.931973 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:25.931955    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 25 01:11:25.932106 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:25.932061    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert podName:0b0f3243-f94a-47e2-a884-3ae235a03f07 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:26.432037292 +0000 UTC m=+37.919333948 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-52w8s" (UID: "0b0f3243-f94a-47e2-a884-3ae235a03f07") : secret "networking-console-plugin-cert" not found
Jul 25 01:11:25.933341 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.933316    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0b0f3243-f94a-47e2-a884-3ae235a03f07-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-52w8s\" (UID: \"0b0f3243-f94a-47e2-a884-3ae235a03f07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-52w8s"
Jul 25 01:11:25.934228 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.934183    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jhxl9"]
Jul 25 01:11:25.935317 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.935296    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x"]
Jul 25 01:11:25.936947 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:11:25.936924    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0b30ace_19f7_4c3c_949e_7526c7a69d90.slice/crio-bea30942ef011ded91c998aacc8eaf5fcff1ba7755efe3c16c7b8761afa37e3e WatchSource:0}: Error finding container bea30942ef011ded91c998aacc8eaf5fcff1ba7755efe3c16c7b8761afa37e3e: Status 404 returned error can't find the container with id bea30942ef011ded91c998aacc8eaf5fcff1ba7755efe3c16c7b8761afa37e3e
Jul 25 01:11:25.937973 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:11:25.937950    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod860680d1_a426_4fea_9c8f_892d44983420.slice/crio-0635b872bc9fc26b817d86f9341eefaa0e836a8c83fd90d3ee873494d8a7096f WatchSource:0}: Error finding container 0635b872bc9fc26b817d86f9341eefaa0e836a8c83fd90d3ee873494d8a7096f: Status 404 returned error can't find the container with id 0635b872bc9fc26b817d86f9341eefaa0e836a8c83fd90d3ee873494d8a7096f
Jul 25 01:11:25.942598 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:25.942569    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-bv82n"]
Jul 25 01:11:25.946667 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:11:25.946628    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf85eed90_9183_4673_8d7f_cf7279d53a71.slice/crio-278d3e701ca4f5c1d8c4f5057bba26d392d95009ff59ce72a417a1132cddb5b0 WatchSource:0}: Error finding container 278d3e701ca4f5c1d8c4f5057bba26d392d95009ff59ce72a417a1132cddb5b0: Status 404 returned error can't find the container with id 278d3e701ca4f5c1d8c4f5057bba26d392d95009ff59ce72a417a1132cddb5b0
Jul 25 01:11:26.133953 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:26.133923    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hvpgn\" (UID: \"e6b09e76-4845-4198-8dcd-b6de8642a958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn"
Jul 25 01:11:26.134079 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:26.133961    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:26.134113 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:26.134087    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 25 01:11:26.134143 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:26.134111    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:26.134234 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:26.134147    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls podName:e6b09e76-4845-4198-8dcd-b6de8642a958 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:28.134130764 +0000 UTC m=+39.621427417 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls") pod "cluster-samples-operator-6748489656-hvpgn" (UID: "e6b09e76-4845-4198-8dcd-b6de8642a958") : secret "samples-operator-tls" not found
Jul 25 01:11:26.134234 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:26.134189    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 01:11:26.134234 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:26.134195    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 25 01:11:26.134234 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:26.134208    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-95f54d94c-n7p57: secret "image-registry-tls" not found
Jul 25 01:11:26.134234 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:26.134232    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls podName:aa437a19-20d8-4209-9886-7c07c2b5bf75 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:28.134221537 +0000 UTC m=+39.621518190 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls") pod "dns-default-tqwn9" (UID: "aa437a19-20d8-4209-9886-7c07c2b5bf75") : secret "dns-default-metrics-tls" not found
Jul 25 01:11:26.134384 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:26.134245    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls podName:5f4be3b2-eda3-4d6b-be6d-9806c73138ae nodeName:}" failed. No retries permitted until 2026-07-25 01:11:28.134239379 +0000 UTC m=+39.621536031 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls") pod "image-registry-95f54d94c-n7p57" (UID: "5f4be3b2-eda3-4d6b-be6d-9806c73138ae") : secret "image-registry-tls" not found
Jul 25 01:11:26.270759 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:26.270725    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc" event={"ID":"6acccfec-8c59-4af6-af57-fd9445830798","Type":"ContainerStarted","Data":"9a83dfda0a8cc772fdb1d04d4f39cc0fda4a0614b6fd9bc80d5adf8eeaac9438"}
Jul 25 01:11:26.273828 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:26.273794    2576 generic.go:356] "Generic (PLEG): container finished" podID="026f8bf6-bdcf-47a9-930e-59323c2cb9f3" containerID="ab64ac534b65b4e744ec02186a6e83e3d85e2636b8294ed82341a87a370bdf1e" exitCode=0
Jul 25 01:11:26.273955 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:26.273892    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8srzj" event={"ID":"026f8bf6-bdcf-47a9-930e-59323c2cb9f3","Type":"ContainerDied","Data":"ab64ac534b65b4e744ec02186a6e83e3d85e2636b8294ed82341a87a370bdf1e"}
Jul 25 01:11:26.275255 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:26.275226    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x" event={"ID":"860680d1-a426-4fea-9c8f-892d44983420","Type":"ContainerStarted","Data":"0635b872bc9fc26b817d86f9341eefaa0e836a8c83fd90d3ee873494d8a7096f"}
Jul 25 01:11:26.277604 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:26.277581    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-59l68" event={"ID":"405e24e0-1385-4056-82c3-bf7eca2bd7e4","Type":"ContainerStarted","Data":"85dd65043cf94ac6c2d5ca1250d1adfbeab7f94da579a18c2faf6671639d7192"}
Jul 25 01:11:26.279208 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:26.279183    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jhxl9" event={"ID":"a0b30ace-19f7-4c3c-949e-7526c7a69d90","Type":"ContainerStarted","Data":"bea30942ef011ded91c998aacc8eaf5fcff1ba7755efe3c16c7b8761afa37e3e"}
Jul 25 01:11:26.280423 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:26.280377    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb" event={"ID":"62b7cd6a-9bb4-4182-9c2b-db95bf27faca","Type":"ContainerStarted","Data":"58c6f6951cd22a7f676845c35ad315c609e158963a5daf346cd388b7a2d6a51d"}
Jul 25 01:11:26.281736 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:26.281711    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-bv82n" event={"ID":"f85eed90-9183-4673-8d7f-cf7279d53a71","Type":"ContainerStarted","Data":"278d3e701ca4f5c1d8c4f5057bba26d392d95009ff59ce72a417a1132cddb5b0"}
Jul 25 01:11:26.436992 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:26.436957    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-52w8s\" (UID: \"0b0f3243-f94a-47e2-a884-3ae235a03f07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-52w8s"
Jul 25 01:11:26.437982 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:26.437953    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 25 01:11:26.438106 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:26.438024    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert podName:0b0f3243-f94a-47e2-a884-3ae235a03f07 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:27.438004628 +0000 UTC m=+38.925301295 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-52w8s" (UID: "0b0f3243-f94a-47e2-a884-3ae235a03f07") : secret "networking-console-plugin-cert" not found
Jul 25 01:11:27.291683 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:27.291210    2576 generic.go:356] "Generic (PLEG): container finished" podID="026f8bf6-bdcf-47a9-930e-59323c2cb9f3" containerID="1406cb4e56f4c3c8ffcf3c358856e4c5554e75d5cefa6bf328b7eb8430b81f15" exitCode=0
Jul 25 01:11:27.291683 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:27.291257    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8srzj" event={"ID":"026f8bf6-bdcf-47a9-930e-59323c2cb9f3","Type":"ContainerDied","Data":"1406cb4e56f4c3c8ffcf3c358856e4c5554e75d5cefa6bf328b7eb8430b81f15"}
Jul 25 01:11:27.459885 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:27.459854    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-52w8s\" (UID: \"0b0f3243-f94a-47e2-a884-3ae235a03f07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-52w8s"
Jul 25 01:11:27.460010 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:27.459998    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 25 01:11:27.460418 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:27.460063    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert podName:0b0f3243-f94a-47e2-a884-3ae235a03f07 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:29.460043071 +0000 UTC m=+40.947339746 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-52w8s" (UID: "0b0f3243-f94a-47e2-a884-3ae235a03f07") : secret "networking-console-plugin-cert" not found
Jul 25 01:11:28.166489 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:28.166147    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hvpgn\" (UID: \"e6b09e76-4845-4198-8dcd-b6de8642a958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn"
Jul 25 01:11:28.166489 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:28.166422    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:28.166801 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:28.166506    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:28.166801 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:28.166276    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 25 01:11:28.166801 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:28.166643    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 01:11:28.166801 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:28.166701    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls podName:aa437a19-20d8-4209-9886-7c07c2b5bf75 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:32.16668287 +0000 UTC m=+43.653979542 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls") pod "dns-default-tqwn9" (UID: "aa437a19-20d8-4209-9886-7c07c2b5bf75") : secret "dns-default-metrics-tls" not found
Jul 25 01:11:28.166801 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:28.166718    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls podName:e6b09e76-4845-4198-8dcd-b6de8642a958 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:32.166709179 +0000 UTC m=+43.654005832 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls") pod "cluster-samples-operator-6748489656-hvpgn" (UID: "e6b09e76-4845-4198-8dcd-b6de8642a958") : secret "samples-operator-tls" not found
Jul 25 01:11:28.166801 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:28.166779    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 25 01:11:28.166801 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:28.166790    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-95f54d94c-n7p57: secret "image-registry-tls" not found
Jul 25 01:11:28.167133 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:28.166820    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls podName:5f4be3b2-eda3-4d6b-be6d-9806c73138ae nodeName:}" failed. No retries permitted until 2026-07-25 01:11:32.16681023 +0000 UTC m=+43.654106894 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls") pod "image-registry-95f54d94c-n7p57" (UID: "5f4be3b2-eda3-4d6b-be6d-9806c73138ae") : secret "image-registry-tls" not found
Jul 25 01:11:28.299816 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:28.299705    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8srzj" event={"ID":"026f8bf6-bdcf-47a9-930e-59323c2cb9f3","Type":"ContainerStarted","Data":"91cfd6173bd004339365c602735ea17f777f207a088373667d3df7c8ac586b28"}
Jul 25 01:11:28.302443 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:28.302413    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-59l68" event={"ID":"405e24e0-1385-4056-82c3-bf7eca2bd7e4","Type":"ContainerStarted","Data":"6f64d9c9f9f115eabccf19b6d429c9e48bb0d5b886008993050b85e3ebe65509"}
Jul 25 01:11:28.323835 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:28.323787    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8srzj" podStartSLOduration=6.934568144 podStartE2EDuration="38.323767426s" totalImagesPullingTime="31.389199282s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:10:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:28.322208376 +0000 UTC m=+39.809505054" watchObservedRunningTime="2026-07-25 01:11:28.323767426 +0000 UTC m=+39.811064104"
Jul 25 01:11:28.339149 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:28.339060    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-59l68" podStartSLOduration=36.768879079 podStartE2EDuration="38.33904619s" totalImagesPullingTime="1.570167111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:28.338957394 +0000 UTC m=+39.826254069" watchObservedRunningTime="2026-07-25 01:11:28.33904619 +0000 UTC m=+39.826342859"
Jul 25 01:11:29.480105 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:29.480057    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-52w8s\" (UID: \"0b0f3243-f94a-47e2-a884-3ae235a03f07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-52w8s"
Jul 25 01:11:29.480647 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:29.480234    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 25 01:11:29.480647 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:29.480319    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert podName:0b0f3243-f94a-47e2-a884-3ae235a03f07 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:33.480297436 +0000 UTC m=+44.967594104 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-52w8s" (UID: "0b0f3243-f94a-47e2-a884-3ae235a03f07") : secret "networking-console-plugin-cert" not found
Jul 25 01:11:32.206911 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:32.206873    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:32.207330 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:32.207022    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 01:11:32.207330 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:32.207049    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hvpgn\" (UID: \"e6b09e76-4845-4198-8dcd-b6de8642a958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn"
Jul 25 01:11:32.207330 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:32.207083    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls podName:aa437a19-20d8-4209-9886-7c07c2b5bf75 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:40.207065742 +0000 UTC m=+51.694362394 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls") pod "dns-default-tqwn9" (UID: "aa437a19-20d8-4209-9886-7c07c2b5bf75") : secret "dns-default-metrics-tls" not found
Jul 25 01:11:32.207330 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:32.207117    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:32.207330 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:32.207128    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 25 01:11:32.207330 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:32.207172    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls podName:e6b09e76-4845-4198-8dcd-b6de8642a958 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:40.207160524 +0000 UTC m=+51.694457177 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls") pod "cluster-samples-operator-6748489656-hvpgn" (UID: "e6b09e76-4845-4198-8dcd-b6de8642a958") : secret "samples-operator-tls" not found
Jul 25 01:11:32.207330 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:32.207199    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 25 01:11:32.207330 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:32.207208    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-95f54d94c-n7p57: secret "image-registry-tls" not found
Jul 25 01:11:32.207330 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:32.207241    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls podName:5f4be3b2-eda3-4d6b-be6d-9806c73138ae nodeName:}" failed. No retries permitted until 2026-07-25 01:11:40.207233147 +0000 UTC m=+51.694529800 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls") pod "image-registry-95f54d94c-n7p57" (UID: "5f4be3b2-eda3-4d6b-be6d-9806c73138ae") : secret "image-registry-tls" not found
Jul 25 01:11:33.316533 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:33.316492    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x" event={"ID":"860680d1-a426-4fea-9c8f-892d44983420","Type":"ContainerStarted","Data":"a79f105a260f34062d13b1e02000bd8b110e50fb09df4ee8241523f67d8e965e"}
Jul 25 01:11:33.318135 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:33.318108    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jhxl9" event={"ID":"a0b30ace-19f7-4c3c-949e-7526c7a69d90","Type":"ContainerStarted","Data":"2a8631c62569a1352e7973b3caf16010f1ff31102f0b163e99848aba6b971734"}
Jul 25 01:11:33.319792 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:33.319671    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb" event={"ID":"62b7cd6a-9bb4-4182-9c2b-db95bf27faca","Type":"ContainerStarted","Data":"246f6552a4844586397f63590cf24bf4b7a6f8d75aeceb0f0480d0b576d756ab"}
Jul 25 01:11:33.321257 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:33.321235    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-bv82n" event={"ID":"f85eed90-9183-4673-8d7f-cf7279d53a71","Type":"ContainerStarted","Data":"82a98eb6de02b8f0e7ac99790a7e99d5d32c2297da1731a8ea698e92f3252e44"}
Jul 25 01:11:33.322983 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:33.322964    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/0.log"
Jul 25 01:11:33.323074 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:33.323006    2576 generic.go:356] "Generic (PLEG): container finished" podID="6acccfec-8c59-4af6-af57-fd9445830798" containerID="964c7e5c48c04de4e1f839314126702a882bde9bcbed457021e9e7ae64449267" exitCode=255
Jul 25 01:11:33.323074 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:33.323054    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc" event={"ID":"6acccfec-8c59-4af6-af57-fd9445830798","Type":"ContainerDied","Data":"964c7e5c48c04de4e1f839314126702a882bde9bcbed457021e9e7ae64449267"}
Jul 25 01:11:33.323265 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:33.323252    2576 scope.go:117] "RemoveContainer" containerID="964c7e5c48c04de4e1f839314126702a882bde9bcbed457021e9e7ae64449267"
Jul 25 01:11:33.332445 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:33.332405    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x" podStartSLOduration=30.715013633 podStartE2EDuration="37.33239265s" totalImagesPullingTime="6.617379017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:33.331779786 +0000 UTC m=+44.819076462" watchObservedRunningTime="2026-07-25 01:11:33.33239265 +0000 UTC m=+44.819689369"
Jul 25 01:11:33.361832 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:33.361793    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-bv82n" podStartSLOduration=28.740292762 podStartE2EDuration="35.361780117s" totalImagesPullingTime="6.621487355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:10:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:33.36124971 +0000 UTC m=+44.848546386" watchObservedRunningTime="2026-07-25 01:11:33.361780117 +0000 UTC m=+44.849076791"
Jul 25 01:11:33.379138 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:33.379087    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb" podStartSLOduration=30.74491111 podStartE2EDuration="37.379065604s" totalImagesPullingTime="6.634154494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:33.377934272 +0000 UTC m=+44.865230947" watchObservedRunningTime="2026-07-25 01:11:33.379065604 +0000 UTC m=+44.866362280"
Jul 25 01:11:33.392440 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:33.392394    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jhxl9" podStartSLOduration=2.770518369 podStartE2EDuration="9.392377812s" totalImagesPullingTime="6.621859443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:11:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:33.391135843 +0000 UTC m=+44.878432515" watchObservedRunningTime="2026-07-25 01:11:33.392377812 +0000 UTC m=+44.879674488"
Jul 25 01:11:33.518404 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:33.518372    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-52w8s\" (UID: \"0b0f3243-f94a-47e2-a884-3ae235a03f07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-52w8s"
Jul 25 01:11:33.518562 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:33.518541    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 25 01:11:33.518632 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:33.518620    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert podName:0b0f3243-f94a-47e2-a884-3ae235a03f07 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:41.518597933 +0000 UTC m=+53.005894587 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-52w8s" (UID: "0b0f3243-f94a-47e2-a884-3ae235a03f07") : secret "networking-console-plugin-cert" not found
Jul 25 01:11:34.328259 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:34.328229    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/1.log"
Jul 25 01:11:34.328731 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:34.328681    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/0.log"
Jul 25 01:11:34.328731 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:34.328717    2576 generic.go:356] "Generic (PLEG): container finished" podID="6acccfec-8c59-4af6-af57-fd9445830798" containerID="2eb8f7a7303a683089bc111fb5f09a3e7d0c4f5d2a0aa5faf1ce923d4c576410" exitCode=255
Jul 25 01:11:34.328841 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:34.328819    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc" event={"ID":"6acccfec-8c59-4af6-af57-fd9445830798","Type":"ContainerDied","Data":"2eb8f7a7303a683089bc111fb5f09a3e7d0c4f5d2a0aa5faf1ce923d4c576410"}
Jul 25 01:11:34.328919 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:34.328903    2576 scope.go:117] "RemoveContainer" containerID="964c7e5c48c04de4e1f839314126702a882bde9bcbed457021e9e7ae64449267"
Jul 25 01:11:34.329105 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:34.329086    2576 scope.go:117] "RemoveContainer" containerID="2eb8f7a7303a683089bc111fb5f09a3e7d0c4f5d2a0aa5faf1ce923d4c576410"
Jul 25 01:11:34.329273 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:34.329255    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-28cvc_openshift-console-operator(6acccfec-8c59-4af6-af57-fd9445830798)\"" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc" podUID="6acccfec-8c59-4af6-af57-fd9445830798"
Jul 25 01:11:34.735363 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:34.735328    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:34.735523 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:34.735376    2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:35.336263 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:35.336238    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/1.log"
Jul 25 01:11:35.336632 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:35.336538    2576 scope.go:117] "RemoveContainer" containerID="2eb8f7a7303a683089bc111fb5f09a3e7d0c4f5d2a0aa5faf1ce923d4c576410"
Jul 25 01:11:35.336702 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:35.336683    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-28cvc_openshift-console-operator(6acccfec-8c59-4af6-af57-fd9445830798)\"" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc" podUID="6acccfec-8c59-4af6-af57-fd9445830798"
Jul 25 01:11:36.339429 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:36.339406    2576 scope.go:117] "RemoveContainer" containerID="2eb8f7a7303a683089bc111fb5f09a3e7d0c4f5d2a0aa5faf1ce923d4c576410"
Jul 25 01:11:36.339796 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:36.339550    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-28cvc_openshift-console-operator(6acccfec-8c59-4af6-af57-fd9445830798)\"" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc" podUID="6acccfec-8c59-4af6-af57-fd9445830798"
Jul 25 01:11:40.277935 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:40.277902    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:40.278309 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:40.277966    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hvpgn\" (UID: \"e6b09e76-4845-4198-8dcd-b6de8642a958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn"
Jul 25 01:11:40.278309 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:40.277987    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:40.278309 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:40.278056    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 01:11:40.278309 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:40.278083    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 25 01:11:40.278309 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:40.278094    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-95f54d94c-n7p57: secret "image-registry-tls" not found
Jul 25 01:11:40.278309 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:40.278114    2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 25 01:11:40.278309 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:40.278127    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls podName:aa437a19-20d8-4209-9886-7c07c2b5bf75 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:56.278111168 +0000 UTC m=+67.765407820 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls") pod "dns-default-tqwn9" (UID: "aa437a19-20d8-4209-9886-7c07c2b5bf75") : secret "dns-default-metrics-tls" not found
Jul 25 01:11:40.278309 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:40.278140    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls podName:5f4be3b2-eda3-4d6b-be6d-9806c73138ae nodeName:}" failed. No retries permitted until 2026-07-25 01:11:56.278134604 +0000 UTC m=+67.765431256 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls") pod "image-registry-95f54d94c-n7p57" (UID: "5f4be3b2-eda3-4d6b-be6d-9806c73138ae") : secret "image-registry-tls" not found
Jul 25 01:11:40.278309 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:40.278168    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls podName:e6b09e76-4845-4198-8dcd-b6de8642a958 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:56.278155494 +0000 UTC m=+67.765452147 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls") pod "cluster-samples-operator-6748489656-hvpgn" (UID: "e6b09e76-4845-4198-8dcd-b6de8642a958") : secret "samples-operator-tls" not found
Jul 25 01:11:41.589813 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:41.589775    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-52w8s\" (UID: \"0b0f3243-f94a-47e2-a884-3ae235a03f07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-52w8s"
Jul 25 01:11:41.590203 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:41.589931    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 25 01:11:41.590203 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:41.589997    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert podName:0b0f3243-f94a-47e2-a884-3ae235a03f07 nodeName:}" failed. No retries permitted until 2026-07-25 01:11:57.589981507 +0000 UTC m=+69.077278160 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-52w8s" (UID: "0b0f3243-f94a-47e2-a884-3ae235a03f07") : secret "networking-console-plugin-cert" not found
Jul 25 01:11:47.990842 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:47.990812    2576 scope.go:117] "RemoveContainer" containerID="2eb8f7a7303a683089bc111fb5f09a3e7d0c4f5d2a0aa5faf1ce923d4c576410"
Jul 25 01:11:48.376321 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:48.376239    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:11:48.376706 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:48.376688    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/1.log"
Jul 25 01:11:48.376782 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:48.376723    2576 generic.go:356] "Generic (PLEG): container finished" podID="6acccfec-8c59-4af6-af57-fd9445830798" containerID="01c8272895cc65c353f1132e6e13ba677de9fb3c8091c85f9a76513fa122f7e4" exitCode=255
Jul 25 01:11:48.376823 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:48.376788    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc" event={"ID":"6acccfec-8c59-4af6-af57-fd9445830798","Type":"ContainerDied","Data":"01c8272895cc65c353f1132e6e13ba677de9fb3c8091c85f9a76513fa122f7e4"}
Jul 25 01:11:48.376863 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:48.376855    2576 scope.go:117] "RemoveContainer" containerID="2eb8f7a7303a683089bc111fb5f09a3e7d0c4f5d2a0aa5faf1ce923d4c576410"
Jul 25 01:11:48.377180 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:48.377159    2576 scope.go:117] "RemoveContainer" containerID="01c8272895cc65c353f1132e6e13ba677de9fb3c8091c85f9a76513fa122f7e4"
Jul 25 01:11:48.377370 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:48.377347    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-28cvc_openshift-console-operator(6acccfec-8c59-4af6-af57-fd9445830798)\"" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc" podUID="6acccfec-8c59-4af6-af57-fd9445830798"
Jul 25 01:11:49.266913 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:49.266883    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kcf67"
Jul 25 01:11:49.380811 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:49.380784    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:11:53.789941 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:53.789912    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs\") pod \"network-metrics-daemon-87t4z\" (UID: \"6f9c02f7-779b-41bb-8260-3244475a0f20\") " pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:53.792144 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:53.792122    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 25 01:11:53.804455 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:53.804421    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6f9c02f7-779b-41bb-8260-3244475a0f20-metrics-certs\") pod \"network-metrics-daemon-87t4z\" (UID: \"6f9c02f7-779b-41bb-8260-3244475a0f20\") " pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:53.816712 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:53.816693    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-djkst\""
Jul 25 01:11:53.825412 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:53.825399    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87t4z"
Jul 25 01:11:53.890847 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:53.890820    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret\") pod \"global-pull-secret-syncer-bqzgx\" (UID: \"056dc3b3-b95b-4d1f-8606-333625c397a3\") " pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:53.892967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:53.892945    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 25 01:11:53.903943 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:53.903918    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/056dc3b3-b95b-4d1f-8606-333625c397a3-original-pull-secret\") pod \"global-pull-secret-syncer-bqzgx\" (UID: \"056dc3b3-b95b-4d1f-8606-333625c397a3\") " pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:53.942899 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:53.942862    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-87t4z"]
Jul 25 01:11:53.947633 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:11:53.947605    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9c02f7_779b_41bb_8260_3244475a0f20.slice/crio-cbee077f5cd9cc39776bb15f1334a2adfe6b0868734ea3a1c26b418ffce6f1ef WatchSource:0}: Error finding container cbee077f5cd9cc39776bb15f1334a2adfe6b0868734ea3a1c26b418ffce6f1ef: Status 404 returned error can't find the container with id cbee077f5cd9cc39776bb15f1334a2adfe6b0868734ea3a1c26b418ffce6f1ef
Jul 25 01:11:53.991956 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:53.991935    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ttkw\" (UniqueName: \"kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw\") pod \"network-check-target-gc8gr\" (UID: \"772e370c-1a11-44c2-8dc9-3c9e1919df27\") " pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:53.994270 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:53.994252    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ttkw\" (UniqueName: \"kubernetes.io/projected/772e370c-1a11-44c2-8dc9-3c9e1919df27-kube-api-access-8ttkw\") pod \"network-check-target-gc8gr\" (UID: \"772e370c-1a11-44c2-8dc9-3c9e1919df27\") " pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:54.102283 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:54.102221    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bqzgx"
Jul 25 01:11:54.110909 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:54.110887    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f4ttp\""
Jul 25 01:11:54.119526 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:54.119506    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:54.229567 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:54.229533    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bqzgx"]
Jul 25 01:11:54.232495 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:11:54.232453    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod056dc3b3_b95b_4d1f_8606_333625c397a3.slice/crio-910304094abc90880446dbe67fd8afcd4c65ff99eb72c6b247550816e19332d8 WatchSource:0}: Error finding container 910304094abc90880446dbe67fd8afcd4c65ff99eb72c6b247550816e19332d8: Status 404 returned error can't find the container with id 910304094abc90880446dbe67fd8afcd4c65ff99eb72c6b247550816e19332d8
Jul 25 01:11:54.245938 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:54.245917    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gc8gr"]
Jul 25 01:11:54.248762 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:11:54.248737    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod772e370c_1a11_44c2_8dc9_3c9e1919df27.slice/crio-3ed6157cf5a473733a801cb6abd928d082a135a00b79a7563ae696bc5bd8d0d2 WatchSource:0}: Error finding container 3ed6157cf5a473733a801cb6abd928d082a135a00b79a7563ae696bc5bd8d0d2: Status 404 returned error can't find the container with id 3ed6157cf5a473733a801cb6abd928d082a135a00b79a7563ae696bc5bd8d0d2
Jul 25 01:11:54.398998 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:54.398907    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gc8gr" event={"ID":"772e370c-1a11-44c2-8dc9-3c9e1919df27","Type":"ContainerStarted","Data":"27e312d1b55989a1bf76de51d0f83a9ecc133e4f0655ea3a32d4221efad83d49"}
Jul 25 01:11:54.398998 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:54.398966    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gc8gr" event={"ID":"772e370c-1a11-44c2-8dc9-3c9e1919df27","Type":"ContainerStarted","Data":"3ed6157cf5a473733a801cb6abd928d082a135a00b79a7563ae696bc5bd8d0d2"}
Jul 25 01:11:54.399205 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:54.399040    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:11:54.400306 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:54.400280    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bqzgx" event={"ID":"056dc3b3-b95b-4d1f-8606-333625c397a3","Type":"ContainerStarted","Data":"910304094abc90880446dbe67fd8afcd4c65ff99eb72c6b247550816e19332d8"}
Jul 25 01:11:54.401555 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:54.401529    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-87t4z" event={"ID":"6f9c02f7-779b-41bb-8260-3244475a0f20","Type":"ContainerStarted","Data":"cbee077f5cd9cc39776bb15f1334a2adfe6b0868734ea3a1c26b418ffce6f1ef"}
Jul 25 01:11:54.413744 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:54.413701    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gc8gr" podStartSLOduration=64.413685156 podStartE2EDuration="1m4.413685156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:10:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:54.41258582 +0000 UTC m=+65.899882496" watchObservedRunningTime="2026-07-25 01:11:54.413685156 +0000 UTC m=+65.900981830"
Jul 25 01:11:54.735528 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:54.735490    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:54.735722 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:54.735565    2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:11:54.735995 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:54.735964    2576 scope.go:117] "RemoveContainer" containerID="01c8272895cc65c353f1132e6e13ba677de9fb3c8091c85f9a76513fa122f7e4"
Jul 25 01:11:54.736161 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:11:54.736139    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-28cvc_openshift-console-operator(6acccfec-8c59-4af6-af57-fd9445830798)\"" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc" podUID="6acccfec-8c59-4af6-af57-fd9445830798"
Jul 25 01:11:55.406694 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:55.406642    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-87t4z" event={"ID":"6f9c02f7-779b-41bb-8260-3244475a0f20","Type":"ContainerStarted","Data":"df3036666b779e9e3ca3a9e4f4ba5ac1a651d17385a6f0f16db7374fcee98f6e"}
Jul 25 01:11:55.407117 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:55.406714    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-87t4z" event={"ID":"6f9c02f7-779b-41bb-8260-3244475a0f20","Type":"ContainerStarted","Data":"86b0691067c8c5a03cc3a59bb20156c3ee535d805e1ffd58aa06354cd443f910"}
Jul 25 01:11:55.420604 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:55.420553    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-87t4z" podStartSLOduration=65.491908431 podStartE2EDuration="1m6.420533518s" totalImagesPullingTime="928.625087ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:55.419471949 +0000 UTC m=+66.906768624" watchObservedRunningTime="2026-07-25 01:11:55.420533518 +0000 UTC m=+66.907830194"
Jul 25 01:11:56.312938 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:56.312903    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:56.313102 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:56.312966    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hvpgn\" (UID: \"e6b09e76-4845-4198-8dcd-b6de8642a958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn"
Jul 25 01:11:56.313102 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:56.313000    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:56.315694 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:56.315649    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aa437a19-20d8-4209-9886-7c07c2b5bf75-metrics-tls\") pod \"dns-default-tqwn9\" (UID: \"aa437a19-20d8-4209-9886-7c07c2b5bf75\") " pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:56.315810 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:56.315757    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e6b09e76-4845-4198-8dcd-b6de8642a958-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hvpgn\" (UID: \"e6b09e76-4845-4198-8dcd-b6de8642a958\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn"
Jul 25 01:11:56.315854 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:56.315834    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls\") pod \"image-registry-95f54d94c-n7p57\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") " pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:56.511156 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:56.511121    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j7vlc\""
Jul 25 01:11:56.518447 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:56.518420    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-f6zpp\""
Jul 25 01:11:56.519423 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:56.519405    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:56.527058 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:56.527036    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn"
Jul 25 01:11:56.567527 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:56.567507    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4svzl\""
Jul 25 01:11:56.576408 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:56.576385    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:11:56.713537 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:56.713334    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-95f54d94c-n7p57"]
Jul 25 01:11:56.718993 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:11:56.718917    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f4be3b2_eda3_4d6b_be6d_9806c73138ae.slice/crio-bd053520c7308fa51d6bbc7ed99d955ffafc5baffb47210af41d81c91e7596a4 WatchSource:0}: Error finding container bd053520c7308fa51d6bbc7ed99d955ffafc5baffb47210af41d81c91e7596a4: Status 404 returned error can't find the container with id bd053520c7308fa51d6bbc7ed99d955ffafc5baffb47210af41d81c91e7596a4
Jul 25 01:11:56.742856 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:56.742829    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn"]
Jul 25 01:11:56.787568 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:56.787534    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tqwn9"]
Jul 25 01:11:56.791157 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:11:56.791137    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa437a19_20d8_4209_9886_7c07c2b5bf75.slice/crio-7ff2eea9efd480aa28000f7d9c6e645f14954e6a71a8b2e6c15c2b593089917d WatchSource:0}: Error finding container 7ff2eea9efd480aa28000f7d9c6e645f14954e6a71a8b2e6c15c2b593089917d: Status 404 returned error can't find the container with id 7ff2eea9efd480aa28000f7d9c6e645f14954e6a71a8b2e6c15c2b593089917d
Jul 25 01:11:57.271170 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.271137    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"]
Jul 25 01:11:57.308081 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.308045    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"]
Jul 25 01:11:57.308243 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.308150    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.311368 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.311200    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 25 01:11:57.311368 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.311243    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 25 01:11:57.311368 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.311353    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 25 01:11:57.311800 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.311397    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 25 01:11:57.311800 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.311400    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 25 01:11:57.311800 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.311479    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 25 01:11:57.311800 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.311758    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 25 01:11:57.415359 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.415314    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tqwn9" event={"ID":"aa437a19-20d8-4209-9886-7c07c2b5bf75","Type":"ContainerStarted","Data":"7ff2eea9efd480aa28000f7d9c6e645f14954e6a71a8b2e6c15c2b593089917d"}
Jul 25 01:11:57.417229 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.417198    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-95f54d94c-n7p57" event={"ID":"5f4be3b2-eda3-4d6b-be6d-9806c73138ae","Type":"ContainerStarted","Data":"97f0e2f5a454553b29bbe8068aa95c2170eb67c87c26f7cdb9f940edbd3109a6"}
Jul 25 01:11:57.417354 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.417242    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-95f54d94c-n7p57" event={"ID":"5f4be3b2-eda3-4d6b-be6d-9806c73138ae","Type":"ContainerStarted","Data":"bd053520c7308fa51d6bbc7ed99d955ffafc5baffb47210af41d81c91e7596a4"}
Jul 25 01:11:57.417354 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.417317    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:11:57.419031 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.418979    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn" event={"ID":"e6b09e76-4845-4198-8dcd-b6de8642a958","Type":"ContainerStarted","Data":"024290289f0d05a4bf589423b12a953e42d0e8519d7e834e86c0892e51a4c49e"}
Jul 25 01:11:57.422898 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.422880    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.423021 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.422920    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzg97\" (UniqueName: \"kubernetes.io/projected/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-kube-api-access-rzg97\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.423021 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.422945    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.423114 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.423048    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-hub\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.423114 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.423081    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-ca\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.423215 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.423125    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.438343 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.438285    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-95f54d94c-n7p57" podStartSLOduration=67.438272604 podStartE2EDuration="1m7.438272604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:10:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:57.437510068 +0000 UTC m=+68.924806743" watchObservedRunningTime="2026-07-25 01:11:57.438272604 +0000 UTC m=+68.925569278"
Jul 25 01:11:57.523816 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.523738    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzg97\" (UniqueName: \"kubernetes.io/projected/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-kube-api-access-rzg97\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.523816 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.523783    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.525863 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.524288    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-hub\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.525863 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.524338    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-ca\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.525863 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.524637    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.525863 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.524719    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.525863 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.525379    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.526948 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.526925    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.527187 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.527169    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-hub\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.527719 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.527684    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.527818 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.527732    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-ca\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.534930 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.534892    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzg97\" (UniqueName: \"kubernetes.io/projected/1e9aace7-1fb9-4a1b-989e-5bc39884e2a6-kube-api-access-rzg97\") pod \"cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf\" (UID: \"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.625316 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.625275    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-52w8s\" (UID: \"0b0f3243-f94a-47e2-a884-3ae235a03f07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-52w8s"
Jul 25 01:11:57.627906 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.627885    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b0f3243-f94a-47e2-a884-3ae235a03f07-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-52w8s\" (UID: \"0b0f3243-f94a-47e2-a884-3ae235a03f07\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-52w8s"
Jul 25 01:11:57.635928 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.635904    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"
Jul 25 01:11:57.749899 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.749865    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-nlp45\""
Jul 25 01:11:57.758589 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:57.758565    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-52w8s"
Jul 25 01:11:58.474942 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:58.474916    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf"]
Jul 25 01:11:58.477922 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:11:58.477893    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e9aace7_1fb9_4a1b_989e_5bc39884e2a6.slice/crio-68ac25ce61f2628fcbe25a5da9d33bbf457980d7c16173768b71beed52a4b4fb WatchSource:0}: Error finding container 68ac25ce61f2628fcbe25a5da9d33bbf457980d7c16173768b71beed52a4b4fb: Status 404 returned error can't find the container with id 68ac25ce61f2628fcbe25a5da9d33bbf457980d7c16173768b71beed52a4b4fb
Jul 25 01:11:58.486013 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:58.485957    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-52w8s"]
Jul 25 01:11:58.488960 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:11:58.488938    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b0f3243_f94a_47e2_a884_3ae235a03f07.slice/crio-95ffba394c23a5219a049fdf6f3968267b057549d2cb9f8c110be2889649f26b WatchSource:0}: Error finding container 95ffba394c23a5219a049fdf6f3968267b057549d2cb9f8c110be2889649f26b: Status 404 returned error can't find the container with id 95ffba394c23a5219a049fdf6f3968267b057549d2cb9f8c110be2889649f26b
Jul 25 01:11:59.427871 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:59.427827    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf" event={"ID":"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6","Type":"ContainerStarted","Data":"68ac25ce61f2628fcbe25a5da9d33bbf457980d7c16173768b71beed52a4b4fb"}
Jul 25 01:11:59.429484 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:59.429428    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-52w8s" event={"ID":"0b0f3243-f94a-47e2-a884-3ae235a03f07","Type":"ContainerStarted","Data":"95ffba394c23a5219a049fdf6f3968267b057549d2cb9f8c110be2889649f26b"}
Jul 25 01:11:59.431445 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:59.431417    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bqzgx" event={"ID":"056dc3b3-b95b-4d1f-8606-333625c397a3","Type":"ContainerStarted","Data":"bdc637d69c894e0a4d911ba3ccdc23088828f6d4ac99f748fe49aee557596b35"}
Jul 25 01:11:59.446830 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:11:59.446781    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bqzgx" podStartSLOduration=65.03600302 podStartE2EDuration="1m9.446762425s" totalImagesPullingTime="4.410759405s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:11:59.445416234 +0000 UTC m=+70.932712910" watchObservedRunningTime="2026-07-25 01:11:59.446762425 +0000 UTC m=+70.934059101"
Jul 25 01:12:02.905307 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:02.905272    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-f4npq"]
Jul 25 01:12:02.920554 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:02.920433    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f4npq"]
Jul 25 01:12:02.920748 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:02.920613    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:02.923018 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:02.922995    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 25 01:12:02.923829 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:02.923795    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 25 01:12:02.923829 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:02.923806    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 25 01:12:02.924046 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:02.923878    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-z9n5z\""
Jul 25 01:12:02.924046 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:02.923794    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 25 01:12:02.972013 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:02.971958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn2jm\" (UniqueName: \"kubernetes.io/projected/8dab96cc-78b2-4915-b417-b04a84616d13-kube-api-access-gn2jm\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:02.972013 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:02.971989    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8dab96cc-78b2-4915-b417-b04a84616d13-crio-socket\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:02.972164 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:02.972020    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8dab96cc-78b2-4915-b417-b04a84616d13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:02.972164 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:02.972088    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8dab96cc-78b2-4915-b417-b04a84616d13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:02.972164 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:02.972137    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8dab96cc-78b2-4915-b417-b04a84616d13-data-volume\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:03.072892 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.072856    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8dab96cc-78b2-4915-b417-b04a84616d13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:03.073066 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.072922    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8dab96cc-78b2-4915-b417-b04a84616d13-data-volume\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:03.073066 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.072959    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn2jm\" (UniqueName: \"kubernetes.io/projected/8dab96cc-78b2-4915-b417-b04a84616d13-kube-api-access-gn2jm\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:03.073066 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:03.072972    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 01:12:03.073066 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.073003    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8dab96cc-78b2-4915-b417-b04a84616d13-crio-socket\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:03.073066 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:03.073045    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8dab96cc-78b2-4915-b417-b04a84616d13-kube-rbac-proxy-cm podName:8dab96cc-78b2-4915-b417-b04a84616d13 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:03.573027113 +0000 UTC m=+75.060323789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8dab96cc-78b2-4915-b417-b04a84616d13-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f4npq" (UID: "8dab96cc-78b2-4915-b417-b04a84616d13") : configmap "kube-rbac-proxy" not found
Jul 25 01:12:03.073308 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.073080    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8dab96cc-78b2-4915-b417-b04a84616d13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:03.073308 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.073083    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8dab96cc-78b2-4915-b417-b04a84616d13-crio-socket\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:03.073308 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:03.073203    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:03.073308 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:03.073250    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dab96cc-78b2-4915-b417-b04a84616d13-insights-runtime-extractor-tls podName:8dab96cc-78b2-4915-b417-b04a84616d13 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:03.573235705 +0000 UTC m=+75.060532372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8dab96cc-78b2-4915-b417-b04a84616d13-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f4npq" (UID: "8dab96cc-78b2-4915-b417-b04a84616d13") : secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:03.073308 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.073286    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8dab96cc-78b2-4915-b417-b04a84616d13-data-volume\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:03.083739 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.083705    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn2jm\" (UniqueName: \"kubernetes.io/projected/8dab96cc-78b2-4915-b417-b04a84616d13-kube-api-access-gn2jm\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:03.449010 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.448967    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tqwn9" event={"ID":"aa437a19-20d8-4209-9886-7c07c2b5bf75","Type":"ContainerStarted","Data":"7012e09c1c460d7dd5fbfc9547649f4502c195a4b382daa21267839feb1097a8"}
Jul 25 01:12:03.449199 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.449030    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tqwn9" event={"ID":"aa437a19-20d8-4209-9886-7c07c2b5bf75","Type":"ContainerStarted","Data":"1e888bf6508b928cf740a58b1018d513ab5a9454efeb5e05b0d672d6732625b5"}
Jul 25 01:12:03.449199 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.449072    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:12:03.450816 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.450788    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-52w8s" event={"ID":"0b0f3243-f94a-47e2-a884-3ae235a03f07","Type":"ContainerStarted","Data":"9f8fd6b35c845582c13ffff7ecb1b09c330b72daf5cdc2217dafe98e4a9d947c"}
Jul 25 01:12:03.452856 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.452829    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn" event={"ID":"e6b09e76-4845-4198-8dcd-b6de8642a958","Type":"ContainerStarted","Data":"10e271ff7cc5db22b5d34514e51d2bd6e0a4938aa23972f2f40ca180025ee097"}
Jul 25 01:12:03.452977 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.452879    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn" event={"ID":"e6b09e76-4845-4198-8dcd-b6de8642a958","Type":"ContainerStarted","Data":"39fdeec37f910b86f155677117348003ab330bb5925f89283fa63e3a49ff0c8a"}
Jul 25 01:12:03.466478 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.466437    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tqwn9" podStartSLOduration=33.779337885 podStartE2EDuration="39.466422691s" totalImagesPullingTime="5.687084806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:11:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:03.464552721 +0000 UTC m=+74.951849399" watchObservedRunningTime="2026-07-25 01:12:03.466422691 +0000 UTC m=+74.953719366"
Jul 25 01:12:03.478909 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.478870    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-52w8s" podStartSLOduration=34.493588805 podStartE2EDuration="38.478853437s" totalImagesPullingTime="3.985264632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:11:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:03.477601374 +0000 UTC m=+74.964898053" watchObservedRunningTime="2026-07-25 01:12:03.478853437 +0000 UTC m=+74.966150112"
Jul 25 01:12:03.494037 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.493994    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hvpgn" podStartSLOduration=67.826054637 podStartE2EDuration="1m13.493976019s" totalImagesPullingTime="5.667921382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:03.492432187 +0000 UTC m=+74.979728863" watchObservedRunningTime="2026-07-25 01:12:03.493976019 +0000 UTC m=+74.981272697"
Jul 25 01:12:03.576053 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.576018    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8dab96cc-78b2-4915-b417-b04a84616d13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:03.576228 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:03.576070    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8dab96cc-78b2-4915-b417-b04a84616d13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:03.576228 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:03.576155    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 01:12:03.576228 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:03.576200    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:03.576228 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:03.576222    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8dab96cc-78b2-4915-b417-b04a84616d13-kube-rbac-proxy-cm podName:8dab96cc-78b2-4915-b417-b04a84616d13 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:04.576208085 +0000 UTC m=+76.063504738 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8dab96cc-78b2-4915-b417-b04a84616d13-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f4npq" (UID: "8dab96cc-78b2-4915-b417-b04a84616d13") : configmap "kube-rbac-proxy" not found
Jul 25 01:12:03.576424 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:03.576276    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dab96cc-78b2-4915-b417-b04a84616d13-insights-runtime-extractor-tls podName:8dab96cc-78b2-4915-b417-b04a84616d13 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:04.576259024 +0000 UTC m=+76.063555680 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8dab96cc-78b2-4915-b417-b04a84616d13-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f4npq" (UID: "8dab96cc-78b2-4915-b417-b04a84616d13") : secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:04.342487 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:04.342433    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tqwn9_aa437a19-20d8-4209-9886-7c07c2b5bf75/dns/0.log"
Jul 25 01:12:04.508537 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:04.508498    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tqwn9_aa437a19-20d8-4209-9886-7c07c2b5bf75/kube-rbac-proxy/0.log"
Jul 25 01:12:04.584164 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:04.584132    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8dab96cc-78b2-4915-b417-b04a84616d13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:04.584283 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:04.584201    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8dab96cc-78b2-4915-b417-b04a84616d13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:04.584345 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:04.584310    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:04.584345 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:04.584317    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 01:12:04.584437 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:04.584377    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dab96cc-78b2-4915-b417-b04a84616d13-insights-runtime-extractor-tls podName:8dab96cc-78b2-4915-b417-b04a84616d13 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:06.584357344 +0000 UTC m=+78.071654025 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8dab96cc-78b2-4915-b417-b04a84616d13-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f4npq" (UID: "8dab96cc-78b2-4915-b417-b04a84616d13") : secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:04.584437 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:04.584399    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8dab96cc-78b2-4915-b417-b04a84616d13-kube-rbac-proxy-cm podName:8dab96cc-78b2-4915-b417-b04a84616d13 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:06.584388926 +0000 UTC m=+78.071685606 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8dab96cc-78b2-4915-b417-b04a84616d13-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f4npq" (UID: "8dab96cc-78b2-4915-b417-b04a84616d13") : configmap "kube-rbac-proxy" not found
Jul 25 01:12:05.109471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:05.109445    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mttfq_c804a635-6110-4cea-b2d8-5e8660b449f8/dns-node-resolver/0.log"
Jul 25 01:12:05.461695 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:05.461645    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf" event={"ID":"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6","Type":"ContainerStarted","Data":"0217caf21f3a5226e1aebedb465bec1c7ad398e31936104afdb3e12ca6fe133b"}
Jul 25 01:12:05.462060 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:05.461707    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf" event={"ID":"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6","Type":"ContainerStarted","Data":"6f66fb6c9f26bb5b541154a34cbe820c20696ad68da9d363c74d6452e977e541"}
Jul 25 01:12:05.462060 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:05.461727    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf" event={"ID":"1e9aace7-1fb9-4a1b-989e-5bc39884e2a6","Type":"ContainerStarted","Data":"6d08cea6f0694bb6913c53c647cc0b360a0cd789e353b2a1635664b4b959761c"}
Jul 25 01:12:05.478416 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:05.478374    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-67d8d5d5b9-f2gnf" podStartSLOduration=2.439573325 podStartE2EDuration="8.478359274s" totalImagesPullingTime="6.038785949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:11:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:05.477034416 +0000 UTC m=+76.964331091" watchObservedRunningTime="2026-07-25 01:12:05.478359274 +0000 UTC m=+76.965655949"
Jul 25 01:12:05.709518 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:05.709487    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-95f54d94c-n7p57_5f4be3b2-eda3-4d6b-be6d-9806c73138ae/registry/0.log"
Jul 25 01:12:06.108339 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:06.108311    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pmpt5_c6ebb66f-5622-481b-ae46-7b3743feec00/node-ca/0.log"
Jul 25 01:12:06.599556 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:06.599506    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8dab96cc-78b2-4915-b417-b04a84616d13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:06.599556 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:06.599567    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8dab96cc-78b2-4915-b417-b04a84616d13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:06.600001 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:06.599632    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 01:12:06.600001 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:06.599644    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:06.600001 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:06.599730    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8dab96cc-78b2-4915-b417-b04a84616d13-kube-rbac-proxy-cm podName:8dab96cc-78b2-4915-b417-b04a84616d13 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:10.599711711 +0000 UTC m=+82.087008367 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8dab96cc-78b2-4915-b417-b04a84616d13-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f4npq" (UID: "8dab96cc-78b2-4915-b417-b04a84616d13") : configmap "kube-rbac-proxy" not found
Jul 25 01:12:06.600001 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:06.599745    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dab96cc-78b2-4915-b417-b04a84616d13-insights-runtime-extractor-tls podName:8dab96cc-78b2-4915-b417-b04a84616d13 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:10.599738659 +0000 UTC m=+82.087035314 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8dab96cc-78b2-4915-b417-b04a84616d13-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f4npq" (UID: "8dab96cc-78b2-4915-b417-b04a84616d13") : secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:06.709076 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:06.709047    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jhxl9_a0b30ace-19f7-4c3c-949e-7526c7a69d90/serve-healthcheck-canary/0.log"
Jul 25 01:12:06.991394 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:06.991365    2576 scope.go:117] "RemoveContainer" containerID="01c8272895cc65c353f1132e6e13ba677de9fb3c8091c85f9a76513fa122f7e4"
Jul 25 01:12:06.991553 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:06.991535    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-28cvc_openshift-console-operator(6acccfec-8c59-4af6-af57-fd9445830798)\"" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc" podUID="6acccfec-8c59-4af6-af57-fd9445830798"
Jul 25 01:12:10.628117 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:10.628080    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8dab96cc-78b2-4915-b417-b04a84616d13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:10.628519 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:10.628127    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8dab96cc-78b2-4915-b417-b04a84616d13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:10.628625 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:10.628606    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8dab96cc-78b2-4915-b417-b04a84616d13-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:10.630530 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:10.630495    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8dab96cc-78b2-4915-b417-b04a84616d13-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f4npq\" (UID: \"8dab96cc-78b2-4915-b417-b04a84616d13\") " pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:10.730484 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:10.730451    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f4npq"
Jul 25 01:12:10.855959 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:10.855928    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f4npq"]
Jul 25 01:12:10.858787 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:12:10.858756    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dab96cc_78b2_4915_b417_b04a84616d13.slice/crio-38735b8effec15086026a0ee695c0810529b331f5460f1eb7cad1337aea6accf WatchSource:0}: Error finding container 38735b8effec15086026a0ee695c0810529b331f5460f1eb7cad1337aea6accf: Status 404 returned error can't find the container with id 38735b8effec15086026a0ee695c0810529b331f5460f1eb7cad1337aea6accf
Jul 25 01:12:11.485469 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:11.485285    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f4npq" event={"ID":"8dab96cc-78b2-4915-b417-b04a84616d13","Type":"ContainerStarted","Data":"38735b8effec15086026a0ee695c0810529b331f5460f1eb7cad1337aea6accf"}
Jul 25 01:12:13.095458 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.095425    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rq6dv"]
Jul 25 01:12:13.102455 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.102429    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.106256 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.106231    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 25 01:12:13.106666 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.106492    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 25 01:12:13.106666 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.106510    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 25 01:12:13.106845 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.106694    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 25 01:12:13.107110 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.107084    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tqbdc\""
Jul 25 01:12:13.107198 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.107140    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 25 01:12:13.107198 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.107167    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 25 01:12:13.148800 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.148772    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-accelerators-collector-config\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.148951 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.148804    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.148951 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.148822    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kngvf\" (UniqueName: \"kubernetes.io/projected/2caa7660-548b-4bc2-bf73-446c447347ce-kube-api-access-kngvf\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.148951 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.148844    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2caa7660-548b-4bc2-bf73-446c447347ce-sys\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.148951 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.148930    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-tls\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.149154 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.148973    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2caa7660-548b-4bc2-bf73-446c447347ce-metrics-client-ca\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.149154 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.148999    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-wtmp\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.149154 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.149021    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-textfile\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.149154 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.149102    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2caa7660-548b-4bc2-bf73-446c447347ce-root\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.249840 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.249801    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2caa7660-548b-4bc2-bf73-446c447347ce-metrics-client-ca\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.250019 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.249850    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-wtmp\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.250019 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.249878    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-textfile\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.250019 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.249910    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2caa7660-548b-4bc2-bf73-446c447347ce-root\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.250019 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.249937    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-accelerators-collector-config\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.250019 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.249965    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.250019 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.249992    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kngvf\" (UniqueName: \"kubernetes.io/projected/2caa7660-548b-4bc2-bf73-446c447347ce-kube-api-access-kngvf\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.250019 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.250015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2caa7660-548b-4bc2-bf73-446c447347ce-sys\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.250401 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.250023    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-wtmp\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.250401 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.250030    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2caa7660-548b-4bc2-bf73-446c447347ce-root\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.250401 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.250177    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2caa7660-548b-4bc2-bf73-446c447347ce-sys\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.250401 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.250234    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-tls\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.250401 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:13.250367    2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 25 01:12:13.250635 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:13.250423    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-tls podName:2caa7660-548b-4bc2-bf73-446c447347ce nodeName:}" failed. No retries permitted until 2026-07-25 01:12:13.750404379 +0000 UTC m=+85.237701032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-tls") pod "node-exporter-rq6dv" (UID: "2caa7660-548b-4bc2-bf73-446c447347ce") : secret "node-exporter-tls" not found
Jul 25 01:12:13.250635 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.250456    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2caa7660-548b-4bc2-bf73-446c447347ce-metrics-client-ca\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.250777 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.250754    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-textfile\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.251145 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.250997    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-accelerators-collector-config\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.252696 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.252670    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.258447 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.258416    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kngvf\" (UniqueName: \"kubernetes.io/projected/2caa7660-548b-4bc2-bf73-446c447347ce-kube-api-access-kngvf\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.458584 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.458555    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tqwn9"
Jul 25 01:12:13.754543 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.754462    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-tls\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:13.756877 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:13.756854    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2caa7660-548b-4bc2-bf73-446c447347ce-node-exporter-tls\") pod \"node-exporter-rq6dv\" (UID: \"2caa7660-548b-4bc2-bf73-446c447347ce\") " pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:14.015087 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:14.015000    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rq6dv"
Jul 25 01:12:15.605834 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:12:15.605802    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2caa7660_548b_4bc2_bf73_446c447347ce.slice/crio-35199150a71c1df206795e04b00921988331b19975b6fbfa58cf87b640ed7a9b WatchSource:0}: Error finding container 35199150a71c1df206795e04b00921988331b19975b6fbfa58cf87b640ed7a9b: Status 404 returned error can't find the container with id 35199150a71c1df206795e04b00921988331b19975b6fbfa58cf87b640ed7a9b
Jul 25 01:12:16.514729 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:16.514617    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rq6dv" event={"ID":"2caa7660-548b-4bc2-bf73-446c447347ce","Type":"ContainerStarted","Data":"35199150a71c1df206795e04b00921988331b19975b6fbfa58cf87b640ed7a9b"}
Jul 25 01:12:16.518844 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:16.518747    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f4npq" event={"ID":"8dab96cc-78b2-4915-b417-b04a84616d13","Type":"ContainerStarted","Data":"7dd29079bcae168f02cfe3f148587f6145f6e30348baa88740188bebd3e00f24"}
Jul 25 01:12:17.524876 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:17.524842    2576 generic.go:356] "Generic (PLEG): container finished" podID="2caa7660-548b-4bc2-bf73-446c447347ce" containerID="d81740d097be00484e63a56825ec549ed64f45538690b61490bae0c6430a3f22" exitCode=0
Jul 25 01:12:17.525285 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:17.524929    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rq6dv" event={"ID":"2caa7660-548b-4bc2-bf73-446c447347ce","Type":"ContainerDied","Data":"d81740d097be00484e63a56825ec549ed64f45538690b61490bae0c6430a3f22"}
Jul 25 01:12:17.991199 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:17.991169    2576 scope.go:117] "RemoveContainer" containerID="01c8272895cc65c353f1132e6e13ba677de9fb3c8091c85f9a76513fa122f7e4"
Jul 25 01:12:18.427408 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:18.427387    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:12:19.535478 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:19.535451    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:12:19.535966 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:19.535548    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc" event={"ID":"6acccfec-8c59-4af6-af57-fd9445830798","Type":"ContainerStarted","Data":"56f3c7d3900c318232d17849733cc69627b97f05adef55b62fa5e443df5122b0"}
Jul 25 01:12:19.535966 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:19.535895    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:12:19.537500 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:19.537476    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f4npq" event={"ID":"8dab96cc-78b2-4915-b417-b04a84616d13","Type":"ContainerStarted","Data":"c3bc2fefc5b6fed13d4f9a1e44bcf48b3fdbe85f557c6ffaf9de81f8d8196032"}
Jul 25 01:12:19.539565 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:19.539541    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rq6dv" event={"ID":"2caa7660-548b-4bc2-bf73-446c447347ce","Type":"ContainerStarted","Data":"83bcbd38bda9d59d7faf5cf9decdfeff5bbd7ea62af5a05f366be5a577b07903"}
Jul 25 01:12:19.539689 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:19.539584    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rq6dv" event={"ID":"2caa7660-548b-4bc2-bf73-446c447347ce","Type":"ContainerStarted","Data":"74c7bbe413d2e92dbbfbca3ecc50e8ff113200d5e224410491ec1b9ecedec4db"}
Jul 25 01:12:19.550823 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:19.550779    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc" podStartSLOduration=80.921430382 podStartE2EDuration="1m27.550765498s" totalImagesPullingTime="6.629335116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:10:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:19.550146729 +0000 UTC m=+91.037443404" watchObservedRunningTime="2026-07-25 01:12:19.550765498 +0000 UTC m=+91.038062193"
Jul 25 01:12:19.567964 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:19.567915    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rq6dv" podStartSLOduration=5.491376145 podStartE2EDuration="6.567900952s" totalImagesPullingTime="1.003179946s" totalInitContainerRuntime="73.344861ms" isStatefulPod=true podCreationTimestamp="2026-07-25 01:12:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:19.567131992 +0000 UTC m=+91.054428658" watchObservedRunningTime="2026-07-25 01:12:19.567900952 +0000 UTC m=+91.055197627"
Jul 25 01:12:20.225666 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:20.225620    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-28cvc"
Jul 25 01:12:20.433533 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:20.433494    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-rswvx"]
Jul 25 01:12:20.436771 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:20.436746    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-rswvx"
Jul 25 01:12:20.438869 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:20.438844    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-h8t9g\""
Jul 25 01:12:20.438989 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:20.438884    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 25 01:12:20.438989 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:20.438846    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 25 01:12:20.451787 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:20.451767    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-rswvx"]
Jul 25 01:12:20.508534 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:20.508453    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wq79\" (UniqueName: \"kubernetes.io/projected/4b696896-09b1-4599-85a3-c14700d2fd08-kube-api-access-8wq79\") pod \"downloads-7c97dc4555-rswvx\" (UID: \"4b696896-09b1-4599-85a3-c14700d2fd08\") " pod="openshift-console/downloads-7c97dc4555-rswvx"
Jul 25 01:12:20.610046 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:20.610007    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wq79\" (UniqueName: \"kubernetes.io/projected/4b696896-09b1-4599-85a3-c14700d2fd08-kube-api-access-8wq79\") pod \"downloads-7c97dc4555-rswvx\" (UID: \"4b696896-09b1-4599-85a3-c14700d2fd08\") " pod="openshift-console/downloads-7c97dc4555-rswvx"
Jul 25 01:12:20.617940 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:20.617909    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wq79\" (UniqueName: \"kubernetes.io/projected/4b696896-09b1-4599-85a3-c14700d2fd08-kube-api-access-8wq79\") pod \"downloads-7c97dc4555-rswvx\" (UID: \"4b696896-09b1-4599-85a3-c14700d2fd08\") " pod="openshift-console/downloads-7c97dc4555-rswvx"
Jul 25 01:12:20.712779 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:20.712747    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-95f54d94c-n7p57"]
Jul 25 01:12:20.748190 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:20.748164    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-rswvx"
Jul 25 01:12:20.890797 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:20.890366    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-rswvx"]
Jul 25 01:12:20.896356 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:12:20.895999    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b696896_09b1_4599_85a3_c14700d2fd08.slice/crio-71324a20a1e99a1883f05939601ac97eab20dc4cb2330a1fdc5129029504bb8f WatchSource:0}: Error finding container 71324a20a1e99a1883f05939601ac97eab20dc4cb2330a1fdc5129029504bb8f: Status 404 returned error can't find the container with id 71324a20a1e99a1883f05939601ac97eab20dc4cb2330a1fdc5129029504bb8f
Jul 25 01:12:21.553947 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:21.553910    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-rswvx" event={"ID":"4b696896-09b1-4599-85a3-c14700d2fd08","Type":"ContainerStarted","Data":"71324a20a1e99a1883f05939601ac97eab20dc4cb2330a1fdc5129029504bb8f"}
Jul 25 01:12:23.564567 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:23.564525    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f4npq" event={"ID":"8dab96cc-78b2-4915-b417-b04a84616d13","Type":"ContainerStarted","Data":"1585a8cd58d209321cca5879a5848b6f2fee65331e4c071a3b081a4935b654e7"}
Jul 25 01:12:23.583604 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:23.583374    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-f4npq" podStartSLOduration=9.793377723 podStartE2EDuration="21.583355198s" totalImagesPullingTime="11.789977475s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:12:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:23.582740735 +0000 UTC m=+95.070037410" watchObservedRunningTime="2026-07-25 01:12:23.583355198 +0000 UTC m=+95.070651876"
Jul 25 01:12:25.410282 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.410251    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gc8gr"
Jul 25 01:12:25.561856 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.561826    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7956ffc966-k95nc"]
Jul 25 01:12:25.567774 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.567747    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.578644 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.578415    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 25 01:12:25.578644 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.578455    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 25 01:12:25.578644 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.578426    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 25 01:12:25.578644 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.578512    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-v28m6\""
Jul 25 01:12:25.578644 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.578420    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 25 01:12:25.578960 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.578783    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 25 01:12:25.582010 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.581988    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7956ffc966-k95nc"]
Jul 25 01:12:25.654593 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.654560    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-service-ca\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.654799 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.654607    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-oauth-config\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.654799 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.654648    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tthbz\" (UniqueName: \"kubernetes.io/projected/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-kube-api-access-tthbz\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.654799 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.654726    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-serving-cert\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.654799 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.654750    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-config\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.654799 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.654788    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-oauth-serving-cert\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.756160 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.756076    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-oauth-serving-cert\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.756160 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.756131    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-service-ca\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.756369 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.756164    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-oauth-config\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.756369 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.756202    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tthbz\" (UniqueName: \"kubernetes.io/projected/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-kube-api-access-tthbz\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.756369 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.756231    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-serving-cert\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.756695 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.756666    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-config\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.756971 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.756930    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-oauth-serving-cert\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.757104 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.757084    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-service-ca\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.757192 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.757173    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-config\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.759050 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.759023    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-oauth-config\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.759131 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.759089    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-serving-cert\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.764128 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.764087    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tthbz\" (UniqueName: \"kubernetes.io/projected/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-kube-api-access-tthbz\") pod \"console-7956ffc966-k95nc\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") " pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:25.878375 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:25.878323    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:26.025093 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:26.025017    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7956ffc966-k95nc"]
Jul 25 01:12:26.029837 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:12:26.029811    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e50738_dbe9_42d4_bd3c_d04c3b7574ca.slice/crio-dde7ba2c9f25f0602c41a432c19966fb354ea3af5986da65c639e73241ec4979 WatchSource:0}: Error finding container dde7ba2c9f25f0602c41a432c19966fb354ea3af5986da65c639e73241ec4979: Status 404 returned error can't find the container with id dde7ba2c9f25f0602c41a432c19966fb354ea3af5986da65c639e73241ec4979
Jul 25 01:12:26.579406 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:26.579367    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7956ffc966-k95nc" event={"ID":"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca","Type":"ContainerStarted","Data":"dde7ba2c9f25f0602c41a432c19966fb354ea3af5986da65c639e73241ec4979"}
Jul 25 01:12:29.594490 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:29.594457    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7956ffc966-k95nc" event={"ID":"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca","Type":"ContainerStarted","Data":"094b535e9388411af515dc2558feeeff7ff302777b55fc60903b15ea9e2a7e22"}
Jul 25 01:12:29.610537 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:29.610473    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7956ffc966-k95nc" podStartSLOduration=1.209776174 podStartE2EDuration="4.610454881s" totalImagesPullingTime="3.400678707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:12:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:29.609741612 +0000 UTC m=+101.097038289" watchObservedRunningTime="2026-07-25 01:12:29.610454881 +0000 UTC m=+101.097751557"
Jul 25 01:12:34.343290 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.343250    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56f77447bd-6hmxh"]
Jul 25 01:12:34.385509 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.385469    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56f77447bd-6hmxh"]
Jul 25 01:12:34.385676 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.385605    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.392341 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.392316    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 25 01:12:34.437208 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.437173    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-service-ca\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.437208 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.437211    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-trusted-ca-bundle\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.437424 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.437250    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec407e4f-8610-4865-a26e-747a10e4b473-console-serving-cert\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.437424 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.437332    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-console-config\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.437424 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.437360    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-oauth-serving-cert\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.437424 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.437380    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgw2w\" (UniqueName: \"kubernetes.io/projected/ec407e4f-8610-4865-a26e-747a10e4b473-kube-api-access-zgw2w\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.437627 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.437486    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec407e4f-8610-4865-a26e-747a10e4b473-console-oauth-config\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.538804 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.538776    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec407e4f-8610-4865-a26e-747a10e4b473-console-oauth-config\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.538804 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.538812    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-service-ca\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.539034 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.538831    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-trusted-ca-bundle\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.539034 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.538868    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec407e4f-8610-4865-a26e-747a10e4b473-console-serving-cert\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.539034 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.538892    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-console-config\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.539034 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.538910    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-oauth-serving-cert\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.539034 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.538935    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgw2w\" (UniqueName: \"kubernetes.io/projected/ec407e4f-8610-4865-a26e-747a10e4b473-kube-api-access-zgw2w\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.539627 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.539576    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-service-ca\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.539933 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.539908    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-console-config\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.540018 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.539932    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-oauth-serving-cert\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.540196 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.540169    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-trusted-ca-bundle\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.541614 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.541589    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec407e4f-8610-4865-a26e-747a10e4b473-console-oauth-config\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.541896 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.541874    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec407e4f-8610-4865-a26e-747a10e4b473-console-serving-cert\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.546154 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.546133    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgw2w\" (UniqueName: \"kubernetes.io/projected/ec407e4f-8610-4865-a26e-747a10e4b473-kube-api-access-zgw2w\") pod \"console-56f77447bd-6hmxh\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") " pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:34.696080 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:34.696042    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:35.878506 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:35.878464    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:35.878923 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:35.878567    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:35.884025 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:35.884000    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:36.626763 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:36.626730    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:12:38.835517 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:38.835484    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56f77447bd-6hmxh"]
Jul 25 01:12:38.986691 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:12:38.986603    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec407e4f_8610_4865_a26e_747a10e4b473.slice/crio-2a0156f6e637f8638c44aaedb271e99440c9a9a8023239e1841b84c13639b4c1 WatchSource:0}: Error finding container 2a0156f6e637f8638c44aaedb271e99440c9a9a8023239e1841b84c13639b4c1: Status 404 returned error can't find the container with id 2a0156f6e637f8638c44aaedb271e99440c9a9a8023239e1841b84c13639b4c1
Jul 25 01:12:39.636531 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:39.636493    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-rswvx" event={"ID":"4b696896-09b1-4599-85a3-c14700d2fd08","Type":"ContainerStarted","Data":"c1bee5458c335b55f56b9d51cb3e91eb4de130c9079dd0733d7f1768a5aeba81"}
Jul 25 01:12:39.636775 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:39.636754    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-rswvx"
Jul 25 01:12:39.638725 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:39.638648    2576 generic.go:356] "Generic (PLEG): container finished" podID="62b7cd6a-9bb4-4182-9c2b-db95bf27faca" containerID="246f6552a4844586397f63590cf24bf4b7a6f8d75aeceb0f0480d0b576d756ab" exitCode=0
Jul 25 01:12:39.638831 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:39.638768    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb" event={"ID":"62b7cd6a-9bb4-4182-9c2b-db95bf27faca","Type":"ContainerDied","Data":"246f6552a4844586397f63590cf24bf4b7a6f8d75aeceb0f0480d0b576d756ab"}
Jul 25 01:12:39.639081 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:39.639065    2576 scope.go:117] "RemoveContainer" containerID="246f6552a4844586397f63590cf24bf4b7a6f8d75aeceb0f0480d0b576d756ab"
Jul 25 01:12:39.640730 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:39.640699    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f77447bd-6hmxh" event={"ID":"ec407e4f-8610-4865-a26e-747a10e4b473","Type":"ContainerStarted","Data":"504f68ec93a4f22efff8b07ee84524994804c77f722c93842731efbd4ebbf67d"}
Jul 25 01:12:39.640830 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:39.640741    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f77447bd-6hmxh" event={"ID":"ec407e4f-8610-4865-a26e-747a10e4b473","Type":"ContainerStarted","Data":"2a0156f6e637f8638c44aaedb271e99440c9a9a8023239e1841b84c13639b4c1"}
Jul 25 01:12:39.642340 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:39.642320    2576 generic.go:356] "Generic (PLEG): container finished" podID="860680d1-a426-4fea-9c8f-892d44983420" containerID="a79f105a260f34062d13b1e02000bd8b110e50fb09df4ee8241523f67d8e965e" exitCode=0
Jul 25 01:12:39.642425 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:39.642348    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x" event={"ID":"860680d1-a426-4fea-9c8f-892d44983420","Type":"ContainerDied","Data":"a79f105a260f34062d13b1e02000bd8b110e50fb09df4ee8241523f67d8e965e"}
Jul 25 01:12:39.642584 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:39.642569    2576 scope.go:117] "RemoveContainer" containerID="a79f105a260f34062d13b1e02000bd8b110e50fb09df4ee8241523f67d8e965e"
Jul 25 01:12:39.652457 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:39.652388    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-rswvx" podStartSLOduration=1.49491792 podStartE2EDuration="19.652370298s" totalImagesPullingTime="18.157452378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:12:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:39.651050238 +0000 UTC m=+111.138346914" watchObservedRunningTime="2026-07-25 01:12:39.652370298 +0000 UTC m=+111.139666975"
Jul 25 01:12:39.657953 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:39.657932    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-rswvx"
Jul 25 01:12:39.669436 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:39.669388    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56f77447bd-6hmxh" podStartSLOduration=5.669374879 podStartE2EDuration="5.669374879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:12:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:39.668877847 +0000 UTC m=+111.156174524" watchObservedRunningTime="2026-07-25 01:12:39.669374879 +0000 UTC m=+111.156671554"
Jul 25 01:12:40.649711 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:40.649435    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-tr4jb" event={"ID":"62b7cd6a-9bb4-4182-9c2b-db95bf27faca","Type":"ContainerStarted","Data":"c64327ebba9bddfe1d58dc934e3e16ade4d1639e1fa1eda20071ab4897e71dbc"}
Jul 25 01:12:40.651725 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:40.651690    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x" event={"ID":"860680d1-a426-4fea-9c8f-892d44983420","Type":"ContainerStarted","Data":"e883f009e30a5cd3f3663002602f0680e3c400c8015c3dc4133674891b82041b"}
Jul 25 01:12:44.696973 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:44.696934    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:44.697546 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:44.697037    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:44.702731 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:44.702702    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:45.677345 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:45.677313    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:12:45.718931 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:45.718897    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7956ffc966-k95nc"]
Jul 25 01:12:45.735210 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:45.735168    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-95f54d94c-n7p57" podUID="5f4be3b2-eda3-4d6b-be6d-9806c73138ae" containerName="registry" containerID="cri-o://97f0e2f5a454553b29bbe8068aa95c2170eb67c87c26f7cdb9f940edbd3109a6" gracePeriod=30
Jul 25 01:12:46.071151 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.071120    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:12:46.251936 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.251846    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-installation-pull-secrets\") pod \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") "
Jul 25 01:12:46.251936 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.251907    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-trusted-ca\") pod \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") "
Jul 25 01:12:46.252158 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.252033    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-image-registry-private-configuration\") pod \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") "
Jul 25 01:12:46.252158 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.252077    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls\") pod \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") "
Jul 25 01:12:46.252158 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.252116    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-certificates\") pod \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") "
Jul 25 01:12:46.252158 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.252151    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-bound-sa-token\") pod \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") "
Jul 25 01:12:46.252378 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.252181    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdtjt\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-kube-api-access-jdtjt\") pod \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") "
Jul 25 01:12:46.252378 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.252207    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-ca-trust-extracted\") pod \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\" (UID: \"5f4be3b2-eda3-4d6b-be6d-9806c73138ae\") "
Jul 25 01:12:46.252378 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.252295    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5f4be3b2-eda3-4d6b-be6d-9806c73138ae" (UID: "5f4be3b2-eda3-4d6b-be6d-9806c73138ae"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:12:46.252523 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.252474    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-trusted-ca\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:12:46.252591 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.252518    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5f4be3b2-eda3-4d6b-be6d-9806c73138ae" (UID: "5f4be3b2-eda3-4d6b-be6d-9806c73138ae"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:12:46.255376 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.255317    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5f4be3b2-eda3-4d6b-be6d-9806c73138ae" (UID: "5f4be3b2-eda3-4d6b-be6d-9806c73138ae"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:12:46.255376 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.255345    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-kube-api-access-jdtjt" (OuterVolumeSpecName: "kube-api-access-jdtjt") pod "5f4be3b2-eda3-4d6b-be6d-9806c73138ae" (UID: "5f4be3b2-eda3-4d6b-be6d-9806c73138ae"). InnerVolumeSpecName "kube-api-access-jdtjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:12:46.255556 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.255408    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5f4be3b2-eda3-4d6b-be6d-9806c73138ae" (UID: "5f4be3b2-eda3-4d6b-be6d-9806c73138ae"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:12:46.255727 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.255691    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5f4be3b2-eda3-4d6b-be6d-9806c73138ae" (UID: "5f4be3b2-eda3-4d6b-be6d-9806c73138ae"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:12:46.255838 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.255798    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5f4be3b2-eda3-4d6b-be6d-9806c73138ae" (UID: "5f4be3b2-eda3-4d6b-be6d-9806c73138ae"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:12:46.263369 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.263342    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5f4be3b2-eda3-4d6b-be6d-9806c73138ae" (UID: "5f4be3b2-eda3-4d6b-be6d-9806c73138ae"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:12:46.353195 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.353159    2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-image-registry-private-configuration\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:12:46.353195 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.353196    2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-tls\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:12:46.353389 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.353209    2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-registry-certificates\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:12:46.353389 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.353218    2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-bound-sa-token\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:12:46.353389 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.353232    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdtjt\" (UniqueName: \"kubernetes.io/projected/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-kube-api-access-jdtjt\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:12:46.353389 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.353246    2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-ca-trust-extracted\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:12:46.353389 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.353258    2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f4be3b2-eda3-4d6b-be6d-9806c73138ae-installation-pull-secrets\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:12:46.678436 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.678402    2576 generic.go:356] "Generic (PLEG): container finished" podID="5f4be3b2-eda3-4d6b-be6d-9806c73138ae" containerID="97f0e2f5a454553b29bbe8068aa95c2170eb67c87c26f7cdb9f940edbd3109a6" exitCode=0
Jul 25 01:12:46.678614 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.678452    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-95f54d94c-n7p57"
Jul 25 01:12:46.678614 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.678491    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-95f54d94c-n7p57" event={"ID":"5f4be3b2-eda3-4d6b-be6d-9806c73138ae","Type":"ContainerDied","Data":"97f0e2f5a454553b29bbe8068aa95c2170eb67c87c26f7cdb9f940edbd3109a6"}
Jul 25 01:12:46.678614 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.678555    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-95f54d94c-n7p57" event={"ID":"5f4be3b2-eda3-4d6b-be6d-9806c73138ae","Type":"ContainerDied","Data":"bd053520c7308fa51d6bbc7ed99d955ffafc5baffb47210af41d81c91e7596a4"}
Jul 25 01:12:46.678614 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.678584    2576 scope.go:117] "RemoveContainer" containerID="97f0e2f5a454553b29bbe8068aa95c2170eb67c87c26f7cdb9f940edbd3109a6"
Jul 25 01:12:46.694164 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.694142    2576 scope.go:117] "RemoveContainer" containerID="97f0e2f5a454553b29bbe8068aa95c2170eb67c87c26f7cdb9f940edbd3109a6"
Jul 25 01:12:46.694470 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:12:46.694436    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97f0e2f5a454553b29bbe8068aa95c2170eb67c87c26f7cdb9f940edbd3109a6\": container with ID starting with 97f0e2f5a454553b29bbe8068aa95c2170eb67c87c26f7cdb9f940edbd3109a6 not found: ID does not exist" containerID="97f0e2f5a454553b29bbe8068aa95c2170eb67c87c26f7cdb9f940edbd3109a6"
Jul 25 01:12:46.694562 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.694472    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97f0e2f5a454553b29bbe8068aa95c2170eb67c87c26f7cdb9f940edbd3109a6"} err="failed to get container status \"97f0e2f5a454553b29bbe8068aa95c2170eb67c87c26f7cdb9f940edbd3109a6\": rpc error: code = NotFound desc = could not find container \"97f0e2f5a454553b29bbe8068aa95c2170eb67c87c26f7cdb9f940edbd3109a6\": container with ID starting with 97f0e2f5a454553b29bbe8068aa95c2170eb67c87c26f7cdb9f940edbd3109a6 not found: ID does not exist"
Jul 25 01:12:46.700970 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.700949    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-95f54d94c-n7p57"]
Jul 25 01:12:46.704594 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.704573    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-95f54d94c-n7p57"]
Jul 25 01:12:46.996436 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:12:46.996360    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f4be3b2-eda3-4d6b-be6d-9806c73138ae" path="/var/lib/kubelet/pods/5f4be3b2-eda3-4d6b-be6d-9806c73138ae/volumes"
Jul 25 01:13:10.747921 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:10.747876    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7956ffc966-k95nc" podUID="b4e50738-dbe9-42d4-bd3c-d04c3b7574ca" containerName="console" containerID="cri-o://094b535e9388411af515dc2558feeeff7ff302777b55fc60903b15ea9e2a7e22" gracePeriod=15
Jul 25 01:13:11.020069 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.020002    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:13:11.035486 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.035387    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-serving-cert\") pod \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") "
Jul 25 01:13:11.035486 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.035479    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-oauth-config\") pod \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") "
Jul 25 01:13:11.035700 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.035507    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-oauth-serving-cert\") pod \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") "
Jul 25 01:13:11.035700 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.035532    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-service-ca\") pod \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") "
Jul 25 01:13:11.035700 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.035554    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-config\") pod \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") "
Jul 25 01:13:11.035700 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.035583    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tthbz\" (UniqueName: \"kubernetes.io/projected/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-kube-api-access-tthbz\") pod \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\" (UID: \"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca\") "
Jul 25 01:13:11.036094 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.036065    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-service-ca" (OuterVolumeSpecName: "service-ca") pod "b4e50738-dbe9-42d4-bd3c-d04c3b7574ca" (UID: "b4e50738-dbe9-42d4-bd3c-d04c3b7574ca"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:13:11.036256 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.036112    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-config" (OuterVolumeSpecName: "console-config") pod "b4e50738-dbe9-42d4-bd3c-d04c3b7574ca" (UID: "b4e50738-dbe9-42d4-bd3c-d04c3b7574ca"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:13:11.036341 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.036262    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b4e50738-dbe9-42d4-bd3c-d04c3b7574ca" (UID: "b4e50738-dbe9-42d4-bd3c-d04c3b7574ca"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:13:11.038169 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.038144    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b4e50738-dbe9-42d4-bd3c-d04c3b7574ca" (UID: "b4e50738-dbe9-42d4-bd3c-d04c3b7574ca"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:13:11.038408 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.038361    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-kube-api-access-tthbz" (OuterVolumeSpecName: "kube-api-access-tthbz") pod "b4e50738-dbe9-42d4-bd3c-d04c3b7574ca" (UID: "b4e50738-dbe9-42d4-bd3c-d04c3b7574ca"). InnerVolumeSpecName "kube-api-access-tthbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:13:11.038609 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.038585    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b4e50738-dbe9-42d4-bd3c-d04c3b7574ca" (UID: "b4e50738-dbe9-42d4-bd3c-d04c3b7574ca"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:13:11.136449 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.136422    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-oauth-config\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:13:11.136449 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.136445    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-oauth-serving-cert\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:13:11.136618 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.136455    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-service-ca\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:13:11.136618 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.136463    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-config\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:13:11.136618 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.136471    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tthbz\" (UniqueName: \"kubernetes.io/projected/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-kube-api-access-tthbz\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:13:11.136618 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.136479    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca-console-serving-cert\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:13:11.766977 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.766938    2576 generic.go:356] "Generic (PLEG): container finished" podID="b4e50738-dbe9-42d4-bd3c-d04c3b7574ca" containerID="094b535e9388411af515dc2558feeeff7ff302777b55fc60903b15ea9e2a7e22" exitCode=0
Jul 25 01:13:11.767379 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.766989    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7956ffc966-k95nc"
Jul 25 01:13:11.767379 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.766996    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7956ffc966-k95nc" event={"ID":"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca","Type":"ContainerDied","Data":"094b535e9388411af515dc2558feeeff7ff302777b55fc60903b15ea9e2a7e22"}
Jul 25 01:13:11.767379 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.767043    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7956ffc966-k95nc" event={"ID":"b4e50738-dbe9-42d4-bd3c-d04c3b7574ca","Type":"ContainerDied","Data":"dde7ba2c9f25f0602c41a432c19966fb354ea3af5986da65c639e73241ec4979"}
Jul 25 01:13:11.767379 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.767070    2576 scope.go:117] "RemoveContainer" containerID="094b535e9388411af515dc2558feeeff7ff302777b55fc60903b15ea9e2a7e22"
Jul 25 01:13:11.775111 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.775094    2576 scope.go:117] "RemoveContainer" containerID="094b535e9388411af515dc2558feeeff7ff302777b55fc60903b15ea9e2a7e22"
Jul 25 01:13:11.775363 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:13:11.775342    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"094b535e9388411af515dc2558feeeff7ff302777b55fc60903b15ea9e2a7e22\": container with ID starting with 094b535e9388411af515dc2558feeeff7ff302777b55fc60903b15ea9e2a7e22 not found: ID does not exist" containerID="094b535e9388411af515dc2558feeeff7ff302777b55fc60903b15ea9e2a7e22"
Jul 25 01:13:11.775435 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.775372    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"094b535e9388411af515dc2558feeeff7ff302777b55fc60903b15ea9e2a7e22"} err="failed to get container status \"094b535e9388411af515dc2558feeeff7ff302777b55fc60903b15ea9e2a7e22\": rpc error: code = NotFound desc = could not find container \"094b535e9388411af515dc2558feeeff7ff302777b55fc60903b15ea9e2a7e22\": container with ID starting with 094b535e9388411af515dc2558feeeff7ff302777b55fc60903b15ea9e2a7e22 not found: ID does not exist"
Jul 25 01:13:11.785518 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.785496    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7956ffc966-k95nc"]
Jul 25 01:13:11.789688 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:11.789645    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7956ffc966-k95nc"]
Jul 25 01:13:12.995137 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:12.995101    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4e50738-dbe9-42d4-bd3c-d04c3b7574ca" path="/var/lib/kubelet/pods/b4e50738-dbe9-42d4-bd3c-d04c3b7574ca/volumes"
Jul 25 01:13:31.415236 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.415195    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-554bcbbf8c-mnrx9"]
Jul 25 01:13:31.415798 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.415566    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f4be3b2-eda3-4d6b-be6d-9806c73138ae" containerName="registry"
Jul 25 01:13:31.415798 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.415584    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f4be3b2-eda3-4d6b-be6d-9806c73138ae" containerName="registry"
Jul 25 01:13:31.415798 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.415610    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4e50738-dbe9-42d4-bd3c-d04c3b7574ca" containerName="console"
Jul 25 01:13:31.415798 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.415619    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4e50738-dbe9-42d4-bd3c-d04c3b7574ca" containerName="console"
Jul 25 01:13:31.415798 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.415714    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4e50738-dbe9-42d4-bd3c-d04c3b7574ca" containerName="console"
Jul 25 01:13:31.415798 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.415730    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f4be3b2-eda3-4d6b-be6d-9806c73138ae" containerName="registry"
Jul 25 01:13:31.421565 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.421540    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.429377 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.429326    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-554bcbbf8c-mnrx9"]
Jul 25 01:13:31.478919 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.478895    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-oauth-config\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.479055 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.478928    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-serving-cert\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.479055 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.478958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-service-ca\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.479055 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.478981    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-oauth-serving-cert\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.479055 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.479036    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-trusted-ca-bundle\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.479202 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.479071    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n2bh\" (UniqueName: \"kubernetes.io/projected/c026a2b2-72e8-4a81-88d5-60282d6b6e30-kube-api-access-6n2bh\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.479202 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.479087    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-config\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.580297 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.580265    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-service-ca\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.580297 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.580296    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-oauth-serving-cert\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.580466 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.580313    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-trusted-ca-bundle\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.580549 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.580527    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n2bh\" (UniqueName: \"kubernetes.io/projected/c026a2b2-72e8-4a81-88d5-60282d6b6e30-kube-api-access-6n2bh\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.580603 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.580571    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-config\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.580649 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.580634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-oauth-config\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.580719 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.580705    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-serving-cert\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.581070 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.581041    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-oauth-serving-cert\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.581070 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.581038    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-service-ca\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.581246 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.581226    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-config\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.581302 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.581246    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-trusted-ca-bundle\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.583137 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.583111    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-oauth-config\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.583223 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.583188    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-serving-cert\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.587527 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.587505    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n2bh\" (UniqueName: \"kubernetes.io/projected/c026a2b2-72e8-4a81-88d5-60282d6b6e30-kube-api-access-6n2bh\") pod \"console-554bcbbf8c-mnrx9\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") " pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.732303 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.732229    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:31.857783 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:31.857752    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-554bcbbf8c-mnrx9"]
Jul 25 01:13:31.860575 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:13:31.860549    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc026a2b2_72e8_4a81_88d5_60282d6b6e30.slice/crio-1dbb3c9e454fa28e5fc0cb58c7cd046d234d7669ddaa73c34562e2b3da79bb57 WatchSource:0}: Error finding container 1dbb3c9e454fa28e5fc0cb58c7cd046d234d7669ddaa73c34562e2b3da79bb57: Status 404 returned error can't find the container with id 1dbb3c9e454fa28e5fc0cb58c7cd046d234d7669ddaa73c34562e2b3da79bb57
Jul 25 01:13:32.842998 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:32.842960    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554bcbbf8c-mnrx9" event={"ID":"c026a2b2-72e8-4a81-88d5-60282d6b6e30","Type":"ContainerStarted","Data":"c23b432dda585f2b5695b7339fdc14b66f7094e83b543ed06718f24a5eeced03"}
Jul 25 01:13:32.843402 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:32.843021    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554bcbbf8c-mnrx9" event={"ID":"c026a2b2-72e8-4a81-88d5-60282d6b6e30","Type":"ContainerStarted","Data":"1dbb3c9e454fa28e5fc0cb58c7cd046d234d7669ddaa73c34562e2b3da79bb57"}
Jul 25 01:13:32.859206 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:32.859150    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-554bcbbf8c-mnrx9" podStartSLOduration=1.859132985 podStartE2EDuration="1.859132985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:13:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:13:32.858937933 +0000 UTC m=+164.346234608" watchObservedRunningTime="2026-07-25 01:13:32.859132985 +0000 UTC m=+164.346429662"
Jul 25 01:13:41.732567 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:41.732488    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:41.732567 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:41.732547    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:41.737073 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:41.737052    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:41.877731 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:41.877706    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:13:41.920571 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:13:41.920541    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56f77447bd-6hmxh"]
Jul 25 01:14:06.943518 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:06.943469    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56f77447bd-6hmxh" podUID="ec407e4f-8610-4865-a26e-747a10e4b473" containerName="console" containerID="cri-o://504f68ec93a4f22efff8b07ee84524994804c77f722c93842731efbd4ebbf67d" gracePeriod=15
Jul 25 01:14:07.180776 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.180746    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:14:07.358245 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.358160    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec407e4f-8610-4865-a26e-747a10e4b473-console-oauth-config\") pod \"ec407e4f-8610-4865-a26e-747a10e4b473\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") "
Jul 25 01:14:07.358245 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.358195    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-console-config\") pod \"ec407e4f-8610-4865-a26e-747a10e4b473\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") "
Jul 25 01:14:07.358245 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.358235    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-service-ca\") pod \"ec407e4f-8610-4865-a26e-747a10e4b473\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") "
Jul 25 01:14:07.358517 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.358255    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-oauth-serving-cert\") pod \"ec407e4f-8610-4865-a26e-747a10e4b473\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") "
Jul 25 01:14:07.358517 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.358282    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-trusted-ca-bundle\") pod \"ec407e4f-8610-4865-a26e-747a10e4b473\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") "
Jul 25 01:14:07.358517 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.358302    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec407e4f-8610-4865-a26e-747a10e4b473-console-serving-cert\") pod \"ec407e4f-8610-4865-a26e-747a10e4b473\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") "
Jul 25 01:14:07.358517 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.358322    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgw2w\" (UniqueName: \"kubernetes.io/projected/ec407e4f-8610-4865-a26e-747a10e4b473-kube-api-access-zgw2w\") pod \"ec407e4f-8610-4865-a26e-747a10e4b473\" (UID: \"ec407e4f-8610-4865-a26e-747a10e4b473\") "
Jul 25 01:14:07.358748 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.358698    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ec407e4f-8610-4865-a26e-747a10e4b473" (UID: "ec407e4f-8610-4865-a26e-747a10e4b473"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:14:07.358748 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.358701    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-service-ca" (OuterVolumeSpecName: "service-ca") pod "ec407e4f-8610-4865-a26e-747a10e4b473" (UID: "ec407e4f-8610-4865-a26e-747a10e4b473"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:14:07.358748 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.358725    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-console-config" (OuterVolumeSpecName: "console-config") pod "ec407e4f-8610-4865-a26e-747a10e4b473" (UID: "ec407e4f-8610-4865-a26e-747a10e4b473"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:14:07.358898 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.358789    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ec407e4f-8610-4865-a26e-747a10e4b473" (UID: "ec407e4f-8610-4865-a26e-747a10e4b473"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:14:07.360278 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.360249    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec407e4f-8610-4865-a26e-747a10e4b473-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ec407e4f-8610-4865-a26e-747a10e4b473" (UID: "ec407e4f-8610-4865-a26e-747a10e4b473"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:14:07.360392 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.360329    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec407e4f-8610-4865-a26e-747a10e4b473-kube-api-access-zgw2w" (OuterVolumeSpecName: "kube-api-access-zgw2w") pod "ec407e4f-8610-4865-a26e-747a10e4b473" (UID: "ec407e4f-8610-4865-a26e-747a10e4b473"). InnerVolumeSpecName "kube-api-access-zgw2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:14:07.360443 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.360417    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec407e4f-8610-4865-a26e-747a10e4b473-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ec407e4f-8610-4865-a26e-747a10e4b473" (UID: "ec407e4f-8610-4865-a26e-747a10e4b473"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:14:07.459186 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.459158    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-console-config\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:14:07.459186 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.459179    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-service-ca\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:14:07.459186 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.459188    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-oauth-serving-cert\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:14:07.459385 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.459197    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec407e4f-8610-4865-a26e-747a10e4b473-trusted-ca-bundle\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:14:07.459385 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.459206    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec407e4f-8610-4865-a26e-747a10e4b473-console-serving-cert\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:14:07.459385 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.459215    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgw2w\" (UniqueName: \"kubernetes.io/projected/ec407e4f-8610-4865-a26e-747a10e4b473-kube-api-access-zgw2w\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:14:07.459385 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.459223    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec407e4f-8610-4865-a26e-747a10e4b473-console-oauth-config\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:14:07.968088 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.968050    2576 generic.go:356] "Generic (PLEG): container finished" podID="ec407e4f-8610-4865-a26e-747a10e4b473" containerID="504f68ec93a4f22efff8b07ee84524994804c77f722c93842731efbd4ebbf67d" exitCode=0
Jul 25 01:14:07.968568 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.968100    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f77447bd-6hmxh"
Jul 25 01:14:07.968568 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.968139    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f77447bd-6hmxh" event={"ID":"ec407e4f-8610-4865-a26e-747a10e4b473","Type":"ContainerDied","Data":"504f68ec93a4f22efff8b07ee84524994804c77f722c93842731efbd4ebbf67d"}
Jul 25 01:14:07.968568 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.968194    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f77447bd-6hmxh" event={"ID":"ec407e4f-8610-4865-a26e-747a10e4b473","Type":"ContainerDied","Data":"2a0156f6e637f8638c44aaedb271e99440c9a9a8023239e1841b84c13639b4c1"}
Jul 25 01:14:07.968568 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.968215    2576 scope.go:117] "RemoveContainer" containerID="504f68ec93a4f22efff8b07ee84524994804c77f722c93842731efbd4ebbf67d"
Jul 25 01:14:07.977336 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.977305    2576 scope.go:117] "RemoveContainer" containerID="504f68ec93a4f22efff8b07ee84524994804c77f722c93842731efbd4ebbf67d"
Jul 25 01:14:07.977838 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:14:07.977812    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"504f68ec93a4f22efff8b07ee84524994804c77f722c93842731efbd4ebbf67d\": container with ID starting with 504f68ec93a4f22efff8b07ee84524994804c77f722c93842731efbd4ebbf67d not found: ID does not exist" containerID="504f68ec93a4f22efff8b07ee84524994804c77f722c93842731efbd4ebbf67d"
Jul 25 01:14:07.977938 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.977852    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"504f68ec93a4f22efff8b07ee84524994804c77f722c93842731efbd4ebbf67d"} err="failed to get container status \"504f68ec93a4f22efff8b07ee84524994804c77f722c93842731efbd4ebbf67d\": rpc error: code = NotFound desc = could not find container \"504f68ec93a4f22efff8b07ee84524994804c77f722c93842731efbd4ebbf67d\": container with ID starting with 504f68ec93a4f22efff8b07ee84524994804c77f722c93842731efbd4ebbf67d not found: ID does not exist"
Jul 25 01:14:07.990406 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.990383    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56f77447bd-6hmxh"]
Jul 25 01:14:07.995904 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:07.995884    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56f77447bd-6hmxh"]
Jul 25 01:14:08.994129 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:14:08.994093    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec407e4f-8610-4865-a26e-747a10e4b473" path="/var/lib/kubelet/pods/ec407e4f-8610-4865-a26e-747a10e4b473/volumes"
Jul 25 01:15:06.923206 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:06.923174    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-756f9c8dcb-z8jpg"]
Jul 25 01:15:06.923641 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:06.923463    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec407e4f-8610-4865-a26e-747a10e4b473" containerName="console"
Jul 25 01:15:06.923641 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:06.923479    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec407e4f-8610-4865-a26e-747a10e4b473" containerName="console"
Jul 25 01:15:06.923641 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:06.923530    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec407e4f-8610-4865-a26e-747a10e4b473" containerName="console"
Jul 25 01:15:06.926381 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:06.926359    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:06.939170 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:06.939148    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-756f9c8dcb-z8jpg"]
Jul 25 01:15:06.969401 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:06.969379    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42rdd\" (UniqueName: \"kubernetes.io/projected/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-kube-api-access-42rdd\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:06.969525 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:06.969408    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-oauth-serving-cert\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:06.969525 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:06.969435    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-service-ca\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:06.969525 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:06.969469    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-oauth-config\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:06.969716 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:06.969551    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-config\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:06.969716 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:06.969594    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-serving-cert\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:06.969716 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:06.969635    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-trusted-ca-bundle\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:07.070705 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:07.070671    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-config\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:07.070872 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:07.070718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-serving-cert\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:07.070872 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:07.070752    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-trusted-ca-bundle\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:07.070872 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:07.070823    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42rdd\" (UniqueName: \"kubernetes.io/projected/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-kube-api-access-42rdd\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:07.070872 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:07.070850    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-oauth-serving-cert\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:07.071070 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:07.070890    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-service-ca\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:07.071070 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:07.070919    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-oauth-config\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:07.071591 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:07.071556    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-oauth-serving-cert\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:07.071766 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:07.071740    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-service-ca\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:07.071937 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:07.071766    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-trusted-ca-bundle\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:07.072046 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:07.072023    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-config\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:07.073203 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:07.073185    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-serving-cert\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:07.073550 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:07.073533    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-oauth-config\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:07.081060 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:07.081038    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42rdd\" (UniqueName: \"kubernetes.io/projected/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-kube-api-access-42rdd\") pod \"console-756f9c8dcb-z8jpg\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") " pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:07.235942 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:07.235853    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:07.363417 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:07.363382    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-756f9c8dcb-z8jpg"]
Jul 25 01:15:07.367108 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:15:07.367082    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c4e717_ead1_4ca6_b73f_2be146f6e0e0.slice/crio-421cf6b0f42203555ba0f29a9615dce33f51117c93f9ed9ba29b0fbd41c8cec4 WatchSource:0}: Error finding container 421cf6b0f42203555ba0f29a9615dce33f51117c93f9ed9ba29b0fbd41c8cec4: Status 404 returned error can't find the container with id 421cf6b0f42203555ba0f29a9615dce33f51117c93f9ed9ba29b0fbd41c8cec4
Jul 25 01:15:08.180485 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:08.180386    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-756f9c8dcb-z8jpg" event={"ID":"70c4e717-ead1-4ca6-b73f-2be146f6e0e0","Type":"ContainerStarted","Data":"7e5d55ca25afbeccca264ff428a23540d8443028d43f91f812e85e6215d4ebbd"}
Jul 25 01:15:08.181136 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:08.181027    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-756f9c8dcb-z8jpg" event={"ID":"70c4e717-ead1-4ca6-b73f-2be146f6e0e0","Type":"ContainerStarted","Data":"421cf6b0f42203555ba0f29a9615dce33f51117c93f9ed9ba29b0fbd41c8cec4"}
Jul 25 01:15:08.198006 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:08.197964    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-756f9c8dcb-z8jpg" podStartSLOduration=2.197952074 podStartE2EDuration="2.197952074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:15:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:15:08.1968224 +0000 UTC m=+259.684119075" watchObservedRunningTime="2026-07-25 01:15:08.197952074 +0000 UTC m=+259.685248810"
Jul 25 01:15:17.236236 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:17.236150    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:17.236236 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:17.236214    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:17.241516 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:17.241490    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:18.222038 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:18.222004    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:15:18.266100 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:18.266073    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-554bcbbf8c-mnrx9"]
Jul 25 01:15:43.285463 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.285426    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-554bcbbf8c-mnrx9" podUID="c026a2b2-72e8-4a81-88d5-60282d6b6e30" containerName="console" containerID="cri-o://c23b432dda585f2b5695b7339fdc14b66f7094e83b543ed06718f24a5eeced03" gracePeriod=15
Jul 25 01:15:43.518931 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.518909    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:15:43.644976 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.644883    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-serving-cert\") pod \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") "
Jul 25 01:15:43.644976 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.644926    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-oauth-serving-cert\") pod \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") "
Jul 25 01:15:43.644976 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.644968    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6n2bh\" (UniqueName: \"kubernetes.io/projected/c026a2b2-72e8-4a81-88d5-60282d6b6e30-kube-api-access-6n2bh\") pod \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") "
Jul 25 01:15:43.645259 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.645076    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-trusted-ca-bundle\") pod \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") "
Jul 25 01:15:43.645259 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.645113    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-service-ca\") pod \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") "
Jul 25 01:15:43.645259 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.645136    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-oauth-config\") pod \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") "
Jul 25 01:15:43.645259 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.645192    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-config\") pod \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\" (UID: \"c026a2b2-72e8-4a81-88d5-60282d6b6e30\") "
Jul 25 01:15:43.645467 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.645404    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c026a2b2-72e8-4a81-88d5-60282d6b6e30" (UID: "c026a2b2-72e8-4a81-88d5-60282d6b6e30"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:15:43.645520 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.645459    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-service-ca" (OuterVolumeSpecName: "service-ca") pod "c026a2b2-72e8-4a81-88d5-60282d6b6e30" (UID: "c026a2b2-72e8-4a81-88d5-60282d6b6e30"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:15:43.645520 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.645468    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c026a2b2-72e8-4a81-88d5-60282d6b6e30" (UID: "c026a2b2-72e8-4a81-88d5-60282d6b6e30"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:15:43.645701 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.645676    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-config" (OuterVolumeSpecName: "console-config") pod "c026a2b2-72e8-4a81-88d5-60282d6b6e30" (UID: "c026a2b2-72e8-4a81-88d5-60282d6b6e30"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:15:43.647094 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.647065    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c026a2b2-72e8-4a81-88d5-60282d6b6e30" (UID: "c026a2b2-72e8-4a81-88d5-60282d6b6e30"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:15:43.647191 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.647096    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c026a2b2-72e8-4a81-88d5-60282d6b6e30-kube-api-access-6n2bh" (OuterVolumeSpecName: "kube-api-access-6n2bh") pod "c026a2b2-72e8-4a81-88d5-60282d6b6e30" (UID: "c026a2b2-72e8-4a81-88d5-60282d6b6e30"). InnerVolumeSpecName "kube-api-access-6n2bh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:15:43.647191 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.647133    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c026a2b2-72e8-4a81-88d5-60282d6b6e30" (UID: "c026a2b2-72e8-4a81-88d5-60282d6b6e30"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:15:43.746724 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.746696    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-config\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:15:43.746724 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.746718    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-serving-cert\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:15:43.746724 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.746728    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-oauth-serving-cert\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:15:43.746909 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.746738    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6n2bh\" (UniqueName: \"kubernetes.io/projected/c026a2b2-72e8-4a81-88d5-60282d6b6e30-kube-api-access-6n2bh\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:15:43.746909 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.746746    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-trusted-ca-bundle\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:15:43.746909 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.746754    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c026a2b2-72e8-4a81-88d5-60282d6b6e30-service-ca\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:15:43.746909 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:43.746762    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c026a2b2-72e8-4a81-88d5-60282d6b6e30-console-oauth-config\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:15:44.311968 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:44.311932    2576 generic.go:356] "Generic (PLEG): container finished" podID="c026a2b2-72e8-4a81-88d5-60282d6b6e30" containerID="c23b432dda585f2b5695b7339fdc14b66f7094e83b543ed06718f24a5eeced03" exitCode=0
Jul 25 01:15:44.312367 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:44.311984    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554bcbbf8c-mnrx9"
Jul 25 01:15:44.312367 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:44.311984    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554bcbbf8c-mnrx9" event={"ID":"c026a2b2-72e8-4a81-88d5-60282d6b6e30","Type":"ContainerDied","Data":"c23b432dda585f2b5695b7339fdc14b66f7094e83b543ed06718f24a5eeced03"}
Jul 25 01:15:44.312367 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:44.312035    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554bcbbf8c-mnrx9" event={"ID":"c026a2b2-72e8-4a81-88d5-60282d6b6e30","Type":"ContainerDied","Data":"1dbb3c9e454fa28e5fc0cb58c7cd046d234d7669ddaa73c34562e2b3da79bb57"}
Jul 25 01:15:44.312367 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:44.312061    2576 scope.go:117] "RemoveContainer" containerID="c23b432dda585f2b5695b7339fdc14b66f7094e83b543ed06718f24a5eeced03"
Jul 25 01:15:44.320492 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:44.320474    2576 scope.go:117] "RemoveContainer" containerID="c23b432dda585f2b5695b7339fdc14b66f7094e83b543ed06718f24a5eeced03"
Jul 25 01:15:44.320758 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:15:44.320738    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c23b432dda585f2b5695b7339fdc14b66f7094e83b543ed06718f24a5eeced03\": container with ID starting with c23b432dda585f2b5695b7339fdc14b66f7094e83b543ed06718f24a5eeced03 not found: ID does not exist" containerID="c23b432dda585f2b5695b7339fdc14b66f7094e83b543ed06718f24a5eeced03"
Jul 25 01:15:44.320811 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:44.320765    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c23b432dda585f2b5695b7339fdc14b66f7094e83b543ed06718f24a5eeced03"} err="failed to get container status \"c23b432dda585f2b5695b7339fdc14b66f7094e83b543ed06718f24a5eeced03\": rpc error: code = NotFound desc = could not find container \"c23b432dda585f2b5695b7339fdc14b66f7094e83b543ed06718f24a5eeced03\": container with ID starting with c23b432dda585f2b5695b7339fdc14b66f7094e83b543ed06718f24a5eeced03 not found: ID does not exist"
Jul 25 01:15:44.331687 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:44.331667    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-554bcbbf8c-mnrx9"]
Jul 25 01:15:44.335135 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:44.335115    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-554bcbbf8c-mnrx9"]
Jul 25 01:15:44.995027 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:44.994994    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c026a2b2-72e8-4a81-88d5-60282d6b6e30" path="/var/lib/kubelet/pods/c026a2b2-72e8-4a81-88d5-60282d6b6e30/volumes"
Jul 25 01:15:48.927098 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:48.927072    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:15:48.933736 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:48.933716    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:15:48.935159 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:48.935132    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:15:48.937807 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:48.937790    2576 kubelet.go:1648] "Image garbage collection succeeded"
Jul 25 01:15:48.941364 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:48.941347    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:15:51.343350 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.343319    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t927f"]
Jul 25 01:15:51.345725 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.343589    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c026a2b2-72e8-4a81-88d5-60282d6b6e30" containerName="console"
Jul 25 01:15:51.345725 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.343599    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c026a2b2-72e8-4a81-88d5-60282d6b6e30" containerName="console"
Jul 25 01:15:51.345725 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.343684    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c026a2b2-72e8-4a81-88d5-60282d6b6e30" containerName="console"
Jul 25 01:15:51.346598 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.346582    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t927f"
Jul 25 01:15:51.349514 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.349491    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-4swg2\""
Jul 25 01:15:51.350670 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.350631    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 25 01:15:51.350905 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.350889    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 25 01:15:51.379546 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.379526    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t927f"]
Jul 25 01:15:51.500170 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.500141    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjw72\" (UniqueName: \"kubernetes.io/projected/c549cdf8-5569-4af7-92ba-86e5522d514d-kube-api-access-cjw72\") pod \"cert-manager-operator-controller-manager-55c444bf86-t927f\" (UID: \"c549cdf8-5569-4af7-92ba-86e5522d514d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t927f"
Jul 25 01:15:51.500291 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.500180    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c549cdf8-5569-4af7-92ba-86e5522d514d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-t927f\" (UID: \"c549cdf8-5569-4af7-92ba-86e5522d514d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t927f"
Jul 25 01:15:51.600513 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.600443    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjw72\" (UniqueName: \"kubernetes.io/projected/c549cdf8-5569-4af7-92ba-86e5522d514d-kube-api-access-cjw72\") pod \"cert-manager-operator-controller-manager-55c444bf86-t927f\" (UID: \"c549cdf8-5569-4af7-92ba-86e5522d514d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t927f"
Jul 25 01:15:51.600513 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.600482    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c549cdf8-5569-4af7-92ba-86e5522d514d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-t927f\" (UID: \"c549cdf8-5569-4af7-92ba-86e5522d514d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t927f"
Jul 25 01:15:51.600976 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.600956    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c549cdf8-5569-4af7-92ba-86e5522d514d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-t927f\" (UID: \"c549cdf8-5569-4af7-92ba-86e5522d514d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t927f"
Jul 25 01:15:51.608588 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.608571    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjw72\" (UniqueName: \"kubernetes.io/projected/c549cdf8-5569-4af7-92ba-86e5522d514d-kube-api-access-cjw72\") pod \"cert-manager-operator-controller-manager-55c444bf86-t927f\" (UID: \"c549cdf8-5569-4af7-92ba-86e5522d514d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t927f"
Jul 25 01:15:51.655558 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.655539    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t927f"
Jul 25 01:15:51.783157 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.783134    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t927f"]
Jul 25 01:15:51.785780 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:15:51.785753    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc549cdf8_5569_4af7_92ba_86e5522d514d.slice/crio-72872d0e683c1a81435e3178030c56e51cae4dc929aad98e60e57fea5e8c1384 WatchSource:0}: Error finding container 72872d0e683c1a81435e3178030c56e51cae4dc929aad98e60e57fea5e8c1384: Status 404 returned error can't find the container with id 72872d0e683c1a81435e3178030c56e51cae4dc929aad98e60e57fea5e8c1384
Jul 25 01:15:51.788325 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:51.788307    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 01:15:52.342366 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:15:52.342327    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t927f" event={"ID":"c549cdf8-5569-4af7-92ba-86e5522d514d","Type":"ContainerStarted","Data":"72872d0e683c1a81435e3178030c56e51cae4dc929aad98e60e57fea5e8c1384"}
Jul 25 01:16:03.388665 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:03.388606    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t927f" event={"ID":"c549cdf8-5569-4af7-92ba-86e5522d514d","Type":"ContainerStarted","Data":"b6a8123bab6b07c8fa68dd5ac999797598cba287ff846b3185fb337f21abbcf9"}
Jul 25 01:16:03.416454 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:03.416407    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-t927f" podStartSLOduration=1.424458629 podStartE2EDuration="12.416395711s" totalImagesPullingTime="10.991937082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:16:03.41453826 +0000 UTC m=+314.901834935" watchObservedRunningTime="2026-07-25 01:16:03.416395711 +0000 UTC m=+314.903692385"
Jul 25 01:16:15.346558 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:15.346522    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tmxkw"]
Jul 25 01:16:15.349803 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:15.349781    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-tmxkw"
Jul 25 01:16:15.351685 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:15.351667    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 25 01:16:15.351685 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:15.351709    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-2d5t6\""
Jul 25 01:16:15.352157 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:15.352136    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 25 01:16:15.356635 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:15.356404    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tmxkw"]
Jul 25 01:16:15.373168 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:15.373148    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23c8f177-b010-496e-8b8d-475f391f1fb9-bound-sa-token\") pod \"cert-manager-55b4d954d5-tmxkw\" (UID: \"23c8f177-b010-496e-8b8d-475f391f1fb9\") " pod="cert-manager/cert-manager-55b4d954d5-tmxkw"
Jul 25 01:16:15.373257 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:15.373204    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nzxj\" (UniqueName: \"kubernetes.io/projected/23c8f177-b010-496e-8b8d-475f391f1fb9-kube-api-access-6nzxj\") pod \"cert-manager-55b4d954d5-tmxkw\" (UID: \"23c8f177-b010-496e-8b8d-475f391f1fb9\") " pod="cert-manager/cert-manager-55b4d954d5-tmxkw"
Jul 25 01:16:15.474227 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:15.474202    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23c8f177-b010-496e-8b8d-475f391f1fb9-bound-sa-token\") pod \"cert-manager-55b4d954d5-tmxkw\" (UID: \"23c8f177-b010-496e-8b8d-475f391f1fb9\") " pod="cert-manager/cert-manager-55b4d954d5-tmxkw"
Jul 25 01:16:15.474328 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:15.474254    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nzxj\" (UniqueName: \"kubernetes.io/projected/23c8f177-b010-496e-8b8d-475f391f1fb9-kube-api-access-6nzxj\") pod \"cert-manager-55b4d954d5-tmxkw\" (UID: \"23c8f177-b010-496e-8b8d-475f391f1fb9\") " pod="cert-manager/cert-manager-55b4d954d5-tmxkw"
Jul 25 01:16:15.481852 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:15.481826    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nzxj\" (UniqueName: \"kubernetes.io/projected/23c8f177-b010-496e-8b8d-475f391f1fb9-kube-api-access-6nzxj\") pod \"cert-manager-55b4d954d5-tmxkw\" (UID: \"23c8f177-b010-496e-8b8d-475f391f1fb9\") " pod="cert-manager/cert-manager-55b4d954d5-tmxkw"
Jul 25 01:16:15.481852 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:15.481836    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23c8f177-b010-496e-8b8d-475f391f1fb9-bound-sa-token\") pod \"cert-manager-55b4d954d5-tmxkw\" (UID: \"23c8f177-b010-496e-8b8d-475f391f1fb9\") " pod="cert-manager/cert-manager-55b4d954d5-tmxkw"
Jul 25 01:16:15.659633 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:15.659611    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-tmxkw"
Jul 25 01:16:15.781380 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:15.781346    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tmxkw"]
Jul 25 01:16:15.784870 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:16:15.784844    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23c8f177_b010_496e_8b8d_475f391f1fb9.slice/crio-8d91fc721d420881b54886657de6d8d927c99a140dc63888986704df593e063a WatchSource:0}: Error finding container 8d91fc721d420881b54886657de6d8d927c99a140dc63888986704df593e063a: Status 404 returned error can't find the container with id 8d91fc721d420881b54886657de6d8d927c99a140dc63888986704df593e063a
Jul 25 01:16:16.443315 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:16.443277    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-tmxkw" event={"ID":"23c8f177-b010-496e-8b8d-475f391f1fb9","Type":"ContainerStarted","Data":"8d91fc721d420881b54886657de6d8d927c99a140dc63888986704df593e063a"}
Jul 25 01:16:19.460450 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:19.460419    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-tmxkw" event={"ID":"23c8f177-b010-496e-8b8d-475f391f1fb9","Type":"ContainerStarted","Data":"476fdea6a96f1831a36e11720921b292867cb3e57dd56549a5923c34223681f1"}
Jul 25 01:16:19.475393 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:19.475350    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-tmxkw" podStartSLOduration=1.126120325 podStartE2EDuration="4.475335331s" totalImagesPullingTime="3.349215006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:16:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:16:19.473886775 +0000 UTC m=+330.961183450" watchObservedRunningTime="2026-07-25 01:16:19.475335331 +0000 UTC m=+330.962632005"
Jul 25 01:16:40.815110 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.815073    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"]
Jul 25 01:16:40.823795 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.823770    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:40.825936 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.825907    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 25 01:16:40.826278 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.826261    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 25 01:16:40.826344 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.826307    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 25 01:16:40.826452 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.826339    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 25 01:16:40.826452 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.826394    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lrsw7\""
Jul 25 01:16:40.832066 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.832047    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"]
Jul 25 01:16:40.867663 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.867632    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a852dc15-df04-4625-baa2-98e0a63b621a-tmp\") pod \"opendatahub-operator-controller-manager-9844cf678-ms57s\" (UID: \"a852dc15-df04-4625-baa2-98e0a63b621a\") " pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:40.867749 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.867689    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a852dc15-df04-4625-baa2-98e0a63b621a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9844cf678-ms57s\" (UID: \"a852dc15-df04-4625-baa2-98e0a63b621a\") " pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:40.867793 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.867774    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a852dc15-df04-4625-baa2-98e0a63b621a-webhook-cert\") pod \"opendatahub-operator-controller-manager-9844cf678-ms57s\" (UID: \"a852dc15-df04-4625-baa2-98e0a63b621a\") " pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:40.867830 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.867804    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfscv\" (UniqueName: \"kubernetes.io/projected/a852dc15-df04-4625-baa2-98e0a63b621a-kube-api-access-cfscv\") pod \"opendatahub-operator-controller-manager-9844cf678-ms57s\" (UID: \"a852dc15-df04-4625-baa2-98e0a63b621a\") " pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:40.867871 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.867855    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a852dc15-df04-4625-baa2-98e0a63b621a-manifests\") pod \"opendatahub-operator-controller-manager-9844cf678-ms57s\" (UID: \"a852dc15-df04-4625-baa2-98e0a63b621a\") " pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:40.968553 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.968520    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a852dc15-df04-4625-baa2-98e0a63b621a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9844cf678-ms57s\" (UID: \"a852dc15-df04-4625-baa2-98e0a63b621a\") " pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:40.968730 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.968571    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a852dc15-df04-4625-baa2-98e0a63b621a-webhook-cert\") pod \"opendatahub-operator-controller-manager-9844cf678-ms57s\" (UID: \"a852dc15-df04-4625-baa2-98e0a63b621a\") " pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:40.968730 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.968604    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfscv\" (UniqueName: \"kubernetes.io/projected/a852dc15-df04-4625-baa2-98e0a63b621a-kube-api-access-cfscv\") pod \"opendatahub-operator-controller-manager-9844cf678-ms57s\" (UID: \"a852dc15-df04-4625-baa2-98e0a63b621a\") " pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:40.968730 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.968682    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a852dc15-df04-4625-baa2-98e0a63b621a-manifests\") pod \"opendatahub-operator-controller-manager-9844cf678-ms57s\" (UID: \"a852dc15-df04-4625-baa2-98e0a63b621a\") " pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:40.968730 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.968722    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a852dc15-df04-4625-baa2-98e0a63b621a-tmp\") pod \"opendatahub-operator-controller-manager-9844cf678-ms57s\" (UID: \"a852dc15-df04-4625-baa2-98e0a63b621a\") " pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:40.969091 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.969069    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a852dc15-df04-4625-baa2-98e0a63b621a-manifests\") pod \"opendatahub-operator-controller-manager-9844cf678-ms57s\" (UID: \"a852dc15-df04-4625-baa2-98e0a63b621a\") " pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:40.970875 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.970847    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a852dc15-df04-4625-baa2-98e0a63b621a-tmp\") pod \"opendatahub-operator-controller-manager-9844cf678-ms57s\" (UID: \"a852dc15-df04-4625-baa2-98e0a63b621a\") " pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:40.971084 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.971067    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a852dc15-df04-4625-baa2-98e0a63b621a-webhook-cert\") pod \"opendatahub-operator-controller-manager-9844cf678-ms57s\" (UID: \"a852dc15-df04-4625-baa2-98e0a63b621a\") " pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:40.971126 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.971103    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a852dc15-df04-4625-baa2-98e0a63b621a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9844cf678-ms57s\" (UID: \"a852dc15-df04-4625-baa2-98e0a63b621a\") " pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:40.979538 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:40.979515    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfscv\" (UniqueName: \"kubernetes.io/projected/a852dc15-df04-4625-baa2-98e0a63b621a-kube-api-access-cfscv\") pod \"opendatahub-operator-controller-manager-9844cf678-ms57s\" (UID: \"a852dc15-df04-4625-baa2-98e0a63b621a\") " pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:41.134553 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:41.134478    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:41.270976 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:41.270947    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"]
Jul 25 01:16:41.274028 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:16:41.273995    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda852dc15_df04_4625_baa2_98e0a63b621a.slice/crio-9bfa7c2cfa720b02cedb1a45f77dd7f8156027cb54793d0ba00d57c2e318e32d WatchSource:0}: Error finding container 9bfa7c2cfa720b02cedb1a45f77dd7f8156027cb54793d0ba00d57c2e318e32d: Status 404 returned error can't find the container with id 9bfa7c2cfa720b02cedb1a45f77dd7f8156027cb54793d0ba00d57c2e318e32d
Jul 25 01:16:41.547124 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:41.547090    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s" event={"ID":"a852dc15-df04-4625-baa2-98e0a63b621a","Type":"ContainerStarted","Data":"9bfa7c2cfa720b02cedb1a45f77dd7f8156027cb54793d0ba00d57c2e318e32d"}
Jul 25 01:16:44.561935 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:44.561855    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s" event={"ID":"a852dc15-df04-4625-baa2-98e0a63b621a","Type":"ContainerStarted","Data":"7fb26c3df454737e991bda65c66fff236e818068e82520592b5eb5cc4534a6e2"}
Jul 25 01:16:45.567321 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:45.567286    2576 generic.go:356] "Generic (PLEG): container finished" podID="a852dc15-df04-4625-baa2-98e0a63b621a" containerID="7fb26c3df454737e991bda65c66fff236e818068e82520592b5eb5cc4534a6e2" exitCode=0
Jul 25 01:16:45.567762 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:45.567372    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s" event={"ID":"a852dc15-df04-4625-baa2-98e0a63b621a","Type":"ContainerDied","Data":"7fb26c3df454737e991bda65c66fff236e818068e82520592b5eb5cc4534a6e2"}
Jul 25 01:16:46.573582 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:46.573543    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s" event={"ID":"a852dc15-df04-4625-baa2-98e0a63b621a","Type":"ContainerStarted","Data":"3403fe0a62a3c6a4f533ccf9d6f54aa172e24a76645d8ef51e52d46489e2b7c6"}
Jul 25 01:16:46.574085 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:46.573668    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:16:46.590824 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:46.590770    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s" podStartSLOduration=2.17078489 podStartE2EDuration="6.590751527s" totalImagesPullingTime="3.347944522s" totalInitContainerRuntime="1.072022115s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:16:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:16:46.589616268 +0000 UTC m=+358.076912954" watchObservedRunningTime="2026-07-25 01:16:46.590751527 +0000 UTC m=+358.078048203"
Jul 25 01:16:57.580677 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:16:57.580632    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9844cf678-ms57s"
Jul 25 01:17:00.498109 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.498075    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c6bd6744b-nwrb5"]
Jul 25 01:17:00.506569 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.506547    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c6bd6744b-nwrb5"
Jul 25 01:17:00.508852 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.508824    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 25 01:17:00.509004 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.508987    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 25 01:17:00.509503 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.509482    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 25 01:17:00.509595 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.509541    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 25 01:17:00.509595 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.509564    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-pzbvk\""
Jul 25 01:17:00.513488 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.513467    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c6bd6744b-nwrb5"]
Jul 25 01:17:00.583127 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.583096    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7bz4g"]
Jul 25 01:17:00.586875 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.586855    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7bz4g"
Jul 25 01:17:00.588938 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.588915    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 25 01:17:00.589062 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.588964    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tdmrs\""
Jul 25 01:17:00.610632 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.610597    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7bz4g"]
Jul 25 01:17:00.624948 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.624921    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6d8e3b8-9f54-4208-9295-e601f6c5da1d-tmp\") pod \"kube-auth-proxy-7c6bd6744b-nwrb5\" (UID: \"a6d8e3b8-9f54-4208-9295-e601f6c5da1d\") " pod="openshift-ingress/kube-auth-proxy-7c6bd6744b-nwrb5"
Jul 25 01:17:00.625088 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.624961    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2dlw\" (UniqueName: \"kubernetes.io/projected/a6d8e3b8-9f54-4208-9295-e601f6c5da1d-kube-api-access-c2dlw\") pod \"kube-auth-proxy-7c6bd6744b-nwrb5\" (UID: \"a6d8e3b8-9f54-4208-9295-e601f6c5da1d\") " pod="openshift-ingress/kube-auth-proxy-7c6bd6744b-nwrb5"
Jul 25 01:17:00.625088 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.625053    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6d8e3b8-9f54-4208-9295-e601f6c5da1d-tls-certs\") pod \"kube-auth-proxy-7c6bd6744b-nwrb5\" (UID: \"a6d8e3b8-9f54-4208-9295-e601f6c5da1d\") " pod="openshift-ingress/kube-auth-proxy-7c6bd6744b-nwrb5"
Jul 25 01:17:00.726276 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.726241    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plpqp\" (UniqueName: \"kubernetes.io/projected/d7221467-0e5d-4b32-ba02-9f57785a77c6-kube-api-access-plpqp\") pod \"odh-model-controller-7998c67dc4-7bz4g\" (UID: \"d7221467-0e5d-4b32-ba02-9f57785a77c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-7bz4g"
Jul 25 01:17:00.726445 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.726293    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6d8e3b8-9f54-4208-9295-e601f6c5da1d-tls-certs\") pod \"kube-auth-proxy-7c6bd6744b-nwrb5\" (UID: \"a6d8e3b8-9f54-4208-9295-e601f6c5da1d\") " pod="openshift-ingress/kube-auth-proxy-7c6bd6744b-nwrb5"
Jul 25 01:17:00.726445 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.726319    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7221467-0e5d-4b32-ba02-9f57785a77c6-cert\") pod \"odh-model-controller-7998c67dc4-7bz4g\" (UID: \"d7221467-0e5d-4b32-ba02-9f57785a77c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-7bz4g"
Jul 25 01:17:00.726445 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.726353    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6d8e3b8-9f54-4208-9295-e601f6c5da1d-tmp\") pod \"kube-auth-proxy-7c6bd6744b-nwrb5\" (UID: \"a6d8e3b8-9f54-4208-9295-e601f6c5da1d\") " pod="openshift-ingress/kube-auth-proxy-7c6bd6744b-nwrb5"
Jul 25 01:17:00.726445 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.726376    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2dlw\" (UniqueName: \"kubernetes.io/projected/a6d8e3b8-9f54-4208-9295-e601f6c5da1d-kube-api-access-c2dlw\") pod \"kube-auth-proxy-7c6bd6744b-nwrb5\" (UID: \"a6d8e3b8-9f54-4208-9295-e601f6c5da1d\") " pod="openshift-ingress/kube-auth-proxy-7c6bd6744b-nwrb5"
Jul 25 01:17:00.728577 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.728543    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6d8e3b8-9f54-4208-9295-e601f6c5da1d-tmp\") pod \"kube-auth-proxy-7c6bd6744b-nwrb5\" (UID: \"a6d8e3b8-9f54-4208-9295-e601f6c5da1d\") " pod="openshift-ingress/kube-auth-proxy-7c6bd6744b-nwrb5"
Jul 25 01:17:00.728796 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.728778    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6d8e3b8-9f54-4208-9295-e601f6c5da1d-tls-certs\") pod \"kube-auth-proxy-7c6bd6744b-nwrb5\" (UID: \"a6d8e3b8-9f54-4208-9295-e601f6c5da1d\") " pod="openshift-ingress/kube-auth-proxy-7c6bd6744b-nwrb5"
Jul 25 01:17:00.733547 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.733527    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2dlw\" (UniqueName: \"kubernetes.io/projected/a6d8e3b8-9f54-4208-9295-e601f6c5da1d-kube-api-access-c2dlw\") pod \"kube-auth-proxy-7c6bd6744b-nwrb5\" (UID: \"a6d8e3b8-9f54-4208-9295-e601f6c5da1d\") " pod="openshift-ingress/kube-auth-proxy-7c6bd6744b-nwrb5"
Jul 25 01:17:00.817662 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.817564    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c6bd6744b-nwrb5"
Jul 25 01:17:00.826956 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.826930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plpqp\" (UniqueName: \"kubernetes.io/projected/d7221467-0e5d-4b32-ba02-9f57785a77c6-kube-api-access-plpqp\") pod \"odh-model-controller-7998c67dc4-7bz4g\" (UID: \"d7221467-0e5d-4b32-ba02-9f57785a77c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-7bz4g"
Jul 25 01:17:00.827047 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.826975    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7221467-0e5d-4b32-ba02-9f57785a77c6-cert\") pod \"odh-model-controller-7998c67dc4-7bz4g\" (UID: \"d7221467-0e5d-4b32-ba02-9f57785a77c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-7bz4g"
Jul 25 01:17:00.829739 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.829708    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7221467-0e5d-4b32-ba02-9f57785a77c6-cert\") pod \"odh-model-controller-7998c67dc4-7bz4g\" (UID: \"d7221467-0e5d-4b32-ba02-9f57785a77c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-7bz4g"
Jul 25 01:17:00.838029 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.838005    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plpqp\" (UniqueName: \"kubernetes.io/projected/d7221467-0e5d-4b32-ba02-9f57785a77c6-kube-api-access-plpqp\") pod \"odh-model-controller-7998c67dc4-7bz4g\" (UID: \"d7221467-0e5d-4b32-ba02-9f57785a77c6\") " pod="opendatahub/odh-model-controller-7998c67dc4-7bz4g"
Jul 25 01:17:00.898341 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.898314    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7bz4g"
Jul 25 01:17:00.967487 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:00.966972    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c6bd6744b-nwrb5"]
Jul 25 01:17:00.971698 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:17:00.971631    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6d8e3b8_9f54_4208_9295_e601f6c5da1d.slice/crio-7de1f8e4e41e6db3410aad4328af3debdeb812bae4c851e661e5d121732ee6e8 WatchSource:0}: Error finding container 7de1f8e4e41e6db3410aad4328af3debdeb812bae4c851e661e5d121732ee6e8: Status 404 returned error can't find the container with id 7de1f8e4e41e6db3410aad4328af3debdeb812bae4c851e661e5d121732ee6e8
Jul 25 01:17:01.054300 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:01.054268    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7bz4g"]
Jul 25 01:17:01.057834 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:17:01.057808    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7221467_0e5d_4b32_ba02_9f57785a77c6.slice/crio-4f6b7deff5cd995b8f4eed3a8d7db8533f6f6bc634801aa2201e08b36895181f WatchSource:0}: Error finding container 4f6b7deff5cd995b8f4eed3a8d7db8533f6f6bc634801aa2201e08b36895181f: Status 404 returned error can't find the container with id 4f6b7deff5cd995b8f4eed3a8d7db8533f6f6bc634801aa2201e08b36895181f
Jul 25 01:17:01.393620 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:01.393587    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cwscn"]
Jul 25 01:17:01.398392 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:01.398368    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cwscn"
Jul 25 01:17:01.400322 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:01.400302    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-p6s2v\""
Jul 25 01:17:01.400434 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:01.400334    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 25 01:17:01.408830 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:01.408808    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cwscn"]
Jul 25 01:17:01.533194 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:01.533158    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkl4n\" (UniqueName: \"kubernetes.io/projected/e927bf51-6950-41c6-9a76-a578c5c782f2-kube-api-access-tkl4n\") pod \"model-serving-api-54d89bd5fc-cwscn\" (UID: \"e927bf51-6950-41c6-9a76-a578c5c782f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-cwscn"
Jul 25 01:17:01.533194 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:01.533198    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e927bf51-6950-41c6-9a76-a578c5c782f2-tls-certs\") pod \"model-serving-api-54d89bd5fc-cwscn\" (UID: \"e927bf51-6950-41c6-9a76-a578c5c782f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-cwscn"
Jul 25 01:17:01.634286 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:01.634253    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e927bf51-6950-41c6-9a76-a578c5c782f2-tls-certs\") pod \"model-serving-api-54d89bd5fc-cwscn\" (UID: \"e927bf51-6950-41c6-9a76-a578c5c782f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-cwscn"
Jul 25 01:17:01.634470 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:01.634383    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkl4n\" (UniqueName: \"kubernetes.io/projected/e927bf51-6950-41c6-9a76-a578c5c782f2-kube-api-access-tkl4n\") pod \"model-serving-api-54d89bd5fc-cwscn\" (UID: \"e927bf51-6950-41c6-9a76-a578c5c782f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-cwscn"
Jul 25 01:17:01.637223 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:01.637194    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e927bf51-6950-41c6-9a76-a578c5c782f2-tls-certs\") pod \"model-serving-api-54d89bd5fc-cwscn\" (UID: \"e927bf51-6950-41c6-9a76-a578c5c782f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-cwscn"
Jul 25 01:17:01.642840 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:01.642806    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7bz4g" event={"ID":"d7221467-0e5d-4b32-ba02-9f57785a77c6","Type":"ContainerStarted","Data":"4f6b7deff5cd995b8f4eed3a8d7db8533f6f6bc634801aa2201e08b36895181f"}
Jul 25 01:17:01.643364 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:01.643337    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkl4n\" (UniqueName: \"kubernetes.io/projected/e927bf51-6950-41c6-9a76-a578c5c782f2-kube-api-access-tkl4n\") pod \"model-serving-api-54d89bd5fc-cwscn\" (UID: \"e927bf51-6950-41c6-9a76-a578c5c782f2\") " pod="opendatahub/model-serving-api-54d89bd5fc-cwscn"
Jul 25 01:17:01.644443 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:01.644386    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c6bd6744b-nwrb5" event={"ID":"a6d8e3b8-9f54-4208-9295-e601f6c5da1d","Type":"ContainerStarted","Data":"7de1f8e4e41e6db3410aad4328af3debdeb812bae4c851e661e5d121732ee6e8"}
Jul 25 01:17:01.709068 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:01.709033    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cwscn"
Jul 25 01:17:01.881069 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:01.872124    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cwscn"]
Jul 25 01:17:02.653119 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:02.653002    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cwscn" event={"ID":"e927bf51-6950-41c6-9a76-a578c5c782f2","Type":"ContainerStarted","Data":"628f71f783e75b7ec1a2be5b92566dd4dde8aec855ebe7d271a42c60a4b45613"}
Jul 25 01:17:05.671187 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:05.671151    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cwscn" event={"ID":"e927bf51-6950-41c6-9a76-a578c5c782f2","Type":"ContainerStarted","Data":"a09e45d4f187e87e923ba43ad9d6a675cc610f2e3ba3b0cff965c34ffa943d17"}
Jul 25 01:17:05.671678 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:05.671345    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-cwscn"
Jul 25 01:17:05.672767 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:05.672734    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7bz4g" event={"ID":"d7221467-0e5d-4b32-ba02-9f57785a77c6","Type":"ContainerStarted","Data":"ecec5d0a2a90774c162147d693ace3dd9947ae69eaf9d943d75be51a615a62a8"}
Jul 25 01:17:05.672882 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:05.672861    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-7bz4g"
Jul 25 01:17:05.674321 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:05.674299    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c6bd6744b-nwrb5" event={"ID":"a6d8e3b8-9f54-4208-9295-e601f6c5da1d","Type":"ContainerStarted","Data":"e98738391d5a0abae3a86add662fe83c198edd242b7b479a3382c3e2a52deb22"}
Jul 25 01:17:05.689354 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:05.689313    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-cwscn" podStartSLOduration=1.3841191720000001 podStartE2EDuration="4.689299905s" totalImagesPullingTime="3.305180733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:17:05.687102914 +0000 UTC m=+377.174399590" watchObservedRunningTime="2026-07-25 01:17:05.689299905 +0000 UTC m=+377.176596580"
Jul 25 01:17:05.701520 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:05.701478    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c6bd6744b-nwrb5" podStartSLOduration=1.488894877 podStartE2EDuration="5.701461333s" totalImagesPullingTime="4.212566456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:17:05.701021308 +0000 UTC m=+377.188317983" watchObservedRunningTime="2026-07-25 01:17:05.701461333 +0000 UTC m=+377.188758009"
Jul 25 01:17:05.714550 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:05.714502    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-7bz4g" podStartSLOduration=1.57674329 podStartE2EDuration="5.71449144s" totalImagesPullingTime="4.13774815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:17:05.713877246 +0000 UTC m=+377.201173926" watchObservedRunningTime="2026-07-25 01:17:05.71449144 +0000 UTC m=+377.201788116"
Jul 25 01:17:06.711493 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.711462    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"]
Jul 25 01:17:06.714814 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.714795    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"
Jul 25 01:17:06.717373 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.717353    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 25 01:17:06.717477 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.717388    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 25 01:17:06.717477 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.717388    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lfs94\""
Jul 25 01:17:06.717585 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.717443    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 25 01:17:06.717585 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.717443    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 25 01:17:06.717824 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.717806    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 25 01:17:06.723031 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.723012    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"]
Jul 25 01:17:06.883379 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.883337    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ce0ca022-f93f-4269-85c1-5cc9b49abf24-manager-config\") pod \"lws-controller-manager-66889bf7cd-rc5rx\" (UID: \"ce0ca022-f93f-4269-85c1-5cc9b49abf24\") " pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"
Jul 25 01:17:06.883379 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.883386    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce0ca022-f93f-4269-85c1-5cc9b49abf24-cert\") pod \"lws-controller-manager-66889bf7cd-rc5rx\" (UID: \"ce0ca022-f93f-4269-85c1-5cc9b49abf24\") " pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"
Jul 25 01:17:06.883624 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.883403    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce0ca022-f93f-4269-85c1-5cc9b49abf24-metrics-cert\") pod \"lws-controller-manager-66889bf7cd-rc5rx\" (UID: \"ce0ca022-f93f-4269-85c1-5cc9b49abf24\") " pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"
Jul 25 01:17:06.883624 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.883422    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g242\" (UniqueName: \"kubernetes.io/projected/ce0ca022-f93f-4269-85c1-5cc9b49abf24-kube-api-access-5g242\") pod \"lws-controller-manager-66889bf7cd-rc5rx\" (UID: \"ce0ca022-f93f-4269-85c1-5cc9b49abf24\") " pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"
Jul 25 01:17:06.984435 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.984340    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce0ca022-f93f-4269-85c1-5cc9b49abf24-cert\") pod \"lws-controller-manager-66889bf7cd-rc5rx\" (UID: \"ce0ca022-f93f-4269-85c1-5cc9b49abf24\") " pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"
Jul 25 01:17:06.984435 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.984376    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce0ca022-f93f-4269-85c1-5cc9b49abf24-metrics-cert\") pod \"lws-controller-manager-66889bf7cd-rc5rx\" (UID: \"ce0ca022-f93f-4269-85c1-5cc9b49abf24\") " pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"
Jul 25 01:17:06.984435 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.984398    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g242\" (UniqueName: \"kubernetes.io/projected/ce0ca022-f93f-4269-85c1-5cc9b49abf24-kube-api-access-5g242\") pod \"lws-controller-manager-66889bf7cd-rc5rx\" (UID: \"ce0ca022-f93f-4269-85c1-5cc9b49abf24\") " pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"
Jul 25 01:17:06.984816 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.984475    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ce0ca022-f93f-4269-85c1-5cc9b49abf24-manager-config\") pod \"lws-controller-manager-66889bf7cd-rc5rx\" (UID: \"ce0ca022-f93f-4269-85c1-5cc9b49abf24\") " pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"
Jul 25 01:17:06.985478 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.985449    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ce0ca022-f93f-4269-85c1-5cc9b49abf24-manager-config\") pod \"lws-controller-manager-66889bf7cd-rc5rx\" (UID: \"ce0ca022-f93f-4269-85c1-5cc9b49abf24\") " pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"
Jul 25 01:17:06.987141 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.987113    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce0ca022-f93f-4269-85c1-5cc9b49abf24-metrics-cert\") pod \"lws-controller-manager-66889bf7cd-rc5rx\" (UID: \"ce0ca022-f93f-4269-85c1-5cc9b49abf24\") " pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"
Jul 25 01:17:06.987228 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.987113    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce0ca022-f93f-4269-85c1-5cc9b49abf24-cert\") pod \"lws-controller-manager-66889bf7cd-rc5rx\" (UID: \"ce0ca022-f93f-4269-85c1-5cc9b49abf24\") " pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"
Jul 25 01:17:06.992048 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:06.992019    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g242\" (UniqueName: \"kubernetes.io/projected/ce0ca022-f93f-4269-85c1-5cc9b49abf24-kube-api-access-5g242\") pod \"lws-controller-manager-66889bf7cd-rc5rx\" (UID: \"ce0ca022-f93f-4269-85c1-5cc9b49abf24\") " pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"
Jul 25 01:17:07.024227 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:07.024198    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"
Jul 25 01:17:07.159459 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:07.159293    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"]
Jul 25 01:17:07.162854 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:17:07.162828    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce0ca022_f93f_4269_85c1_5cc9b49abf24.slice/crio-07e189cfeead2a9722bf003b63de0dde53fb9eb0c59f2d90826c20d2c78fb391 WatchSource:0}: Error finding container 07e189cfeead2a9722bf003b63de0dde53fb9eb0c59f2d90826c20d2c78fb391: Status 404 returned error can't find the container with id 07e189cfeead2a9722bf003b63de0dde53fb9eb0c59f2d90826c20d2c78fb391
Jul 25 01:17:07.684310 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:07.684274    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx" event={"ID":"ce0ca022-f93f-4269-85c1-5cc9b49abf24","Type":"ContainerStarted","Data":"07e189cfeead2a9722bf003b63de0dde53fb9eb0c59f2d90826c20d2c78fb391"}
Jul 25 01:17:10.701407 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:10.701366    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx" event={"ID":"ce0ca022-f93f-4269-85c1-5cc9b49abf24","Type":"ContainerStarted","Data":"88717f248cc304d47bac640ffc45beba571dc869f1d33c42435aa4537c7000b7"}
Jul 25 01:17:10.701889 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:10.701489    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"
Jul 25 01:17:10.728354 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:10.728303    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx" podStartSLOduration=2.068862224 podStartE2EDuration="4.72828717s" totalImagesPullingTime="2.659424946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:17:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:17:10.726950269 +0000 UTC m=+382.214246945" watchObservedRunningTime="2026-07-25 01:17:10.72828717 +0000 UTC m=+382.215583917"
Jul 25 01:17:16.681270 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:16.681236    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-7bz4g"
Jul 25 01:17:16.683194 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:16.683171    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-cwscn"
Jul 25 01:17:21.708646 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:17:21.708615    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-66889bf7cd-rc5rx"
Jul 25 01:18:38.821956 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:38.821923    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dc9685f5d-mmbth"]
Jul 25 01:18:38.825678 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:38.825633    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:38.837862 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:38.837841    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dc9685f5d-mmbth"]
Jul 25 01:18:38.913378 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:38.913348    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bbba638f-b747-4abf-871f-6c393e2e05b7-console-serving-cert\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:38.913378 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:38.913381    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bbba638f-b747-4abf-871f-6c393e2e05b7-oauth-serving-cert\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:38.913547 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:38.913401    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwnsr\" (UniqueName: \"kubernetes.io/projected/bbba638f-b747-4abf-871f-6c393e2e05b7-kube-api-access-bwnsr\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:38.913547 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:38.913433    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bbba638f-b747-4abf-871f-6c393e2e05b7-service-ca\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:38.913547 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:38.913461    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bbba638f-b747-4abf-871f-6c393e2e05b7-console-config\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:38.913547 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:38.913477    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbba638f-b747-4abf-871f-6c393e2e05b7-trusted-ca-bundle\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:38.913743 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:38.913552    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bbba638f-b747-4abf-871f-6c393e2e05b7-console-oauth-config\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:39.014416 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:39.014387    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bbba638f-b747-4abf-871f-6c393e2e05b7-service-ca\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:39.014416 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:39.014420    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bbba638f-b747-4abf-871f-6c393e2e05b7-console-config\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:39.014625 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:39.014436    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbba638f-b747-4abf-871f-6c393e2e05b7-trusted-ca-bundle\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:39.014625 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:39.014485    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bbba638f-b747-4abf-871f-6c393e2e05b7-console-oauth-config\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:39.014625 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:39.014533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bbba638f-b747-4abf-871f-6c393e2e05b7-console-serving-cert\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:39.014625 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:39.014554    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bbba638f-b747-4abf-871f-6c393e2e05b7-oauth-serving-cert\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:39.014625 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:39.014575    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwnsr\" (UniqueName: \"kubernetes.io/projected/bbba638f-b747-4abf-871f-6c393e2e05b7-kube-api-access-bwnsr\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:39.015194 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:39.015166    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bbba638f-b747-4abf-871f-6c393e2e05b7-service-ca\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:39.015307 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:39.015240    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bbba638f-b747-4abf-871f-6c393e2e05b7-console-config\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:39.015388 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:39.015362    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bbba638f-b747-4abf-871f-6c393e2e05b7-oauth-serving-cert\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:39.015515 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:39.015496    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbba638f-b747-4abf-871f-6c393e2e05b7-trusted-ca-bundle\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:39.017065 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:39.017025    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bbba638f-b747-4abf-871f-6c393e2e05b7-console-serving-cert\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:39.017184 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:39.017104    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bbba638f-b747-4abf-871f-6c393e2e05b7-console-oauth-config\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:39.021412 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:39.021390    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwnsr\" (UniqueName: \"kubernetes.io/projected/bbba638f-b747-4abf-871f-6c393e2e05b7-kube-api-access-bwnsr\") pod \"console-6dc9685f5d-mmbth\" (UID: \"bbba638f-b747-4abf-871f-6c393e2e05b7\") " pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:39.135404 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:39.135323    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:39.265238 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:39.265206    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dc9685f5d-mmbth"]
Jul 25 01:18:39.268758 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:18:39.268725    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbba638f_b747_4abf_871f_6c393e2e05b7.slice/crio-3ced8d19e8a61df436a8e954f4dd5b8736246ad07a7b8e790efdf43214c38684 WatchSource:0}: Error finding container 3ced8d19e8a61df436a8e954f4dd5b8736246ad07a7b8e790efdf43214c38684: Status 404 returned error can't find the container with id 3ced8d19e8a61df436a8e954f4dd5b8736246ad07a7b8e790efdf43214c38684
Jul 25 01:18:40.063022 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:40.062991    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dc9685f5d-mmbth" event={"ID":"bbba638f-b747-4abf-871f-6c393e2e05b7","Type":"ContainerStarted","Data":"86f982934c5ab9bf98507a54e461c8d07f74b92585de394c723ebb457cfce3c3"}
Jul 25 01:18:40.063367 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:40.063035    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dc9685f5d-mmbth" event={"ID":"bbba638f-b747-4abf-871f-6c393e2e05b7","Type":"ContainerStarted","Data":"3ced8d19e8a61df436a8e954f4dd5b8736246ad07a7b8e790efdf43214c38684"}
Jul 25 01:18:40.079593 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:40.079549    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6dc9685f5d-mmbth" podStartSLOduration=2.079530379 podStartE2EDuration="2.079530379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:18:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:18:40.07887087 +0000 UTC m=+471.566167544" watchObservedRunningTime="2026-07-25 01:18:40.079530379 +0000 UTC m=+471.566827054"
Jul 25 01:18:49.135973 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:49.135943    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:49.136400 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:49.135993    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:49.140475 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:49.140454    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:50.117522 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:50.117493    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6dc9685f5d-mmbth"
Jul 25 01:18:50.157927 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:18:50.157895    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-756f9c8dcb-z8jpg"]
Jul 25 01:19:15.177026 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.176962    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-756f9c8dcb-z8jpg" podUID="70c4e717-ead1-4ca6-b73f-2be146f6e0e0" containerName="console" containerID="cri-o://7e5d55ca25afbeccca264ff428a23540d8443028d43f91f812e85e6215d4ebbd" gracePeriod=15
Jul 25 01:19:15.426389 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.426364    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:19:15.509436 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.509364    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-oauth-config\") pod \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") "
Jul 25 01:19:15.509436 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.509398    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-trusted-ca-bundle\") pod \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") "
Jul 25 01:19:15.509636 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.509455    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-serving-cert\") pod \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") "
Jul 25 01:19:15.509636 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.509507    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-service-ca\") pod \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") "
Jul 25 01:19:15.509636 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.509536    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42rdd\" (UniqueName: \"kubernetes.io/projected/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-kube-api-access-42rdd\") pod \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") "
Jul 25 01:19:15.509636 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.509568    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-oauth-serving-cert\") pod \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") "
Jul 25 01:19:15.509636 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.509607    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-config\") pod \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\" (UID: \"70c4e717-ead1-4ca6-b73f-2be146f6e0e0\") "
Jul 25 01:19:15.509946 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.509919    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-service-ca" (OuterVolumeSpecName: "service-ca") pod "70c4e717-ead1-4ca6-b73f-2be146f6e0e0" (UID: "70c4e717-ead1-4ca6-b73f-2be146f6e0e0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:19:15.510117 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.510086    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "70c4e717-ead1-4ca6-b73f-2be146f6e0e0" (UID: "70c4e717-ead1-4ca6-b73f-2be146f6e0e0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:19:15.510117 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.509950    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "70c4e717-ead1-4ca6-b73f-2be146f6e0e0" (UID: "70c4e717-ead1-4ca6-b73f-2be146f6e0e0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:19:15.510256 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.510151    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-config" (OuterVolumeSpecName: "console-config") pod "70c4e717-ead1-4ca6-b73f-2be146f6e0e0" (UID: "70c4e717-ead1-4ca6-b73f-2be146f6e0e0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:19:15.511498 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.511475    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "70c4e717-ead1-4ca6-b73f-2be146f6e0e0" (UID: "70c4e717-ead1-4ca6-b73f-2be146f6e0e0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:19:15.511648 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.511631    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "70c4e717-ead1-4ca6-b73f-2be146f6e0e0" (UID: "70c4e717-ead1-4ca6-b73f-2be146f6e0e0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:19:15.511648 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.511675    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-kube-api-access-42rdd" (OuterVolumeSpecName: "kube-api-access-42rdd") pod "70c4e717-ead1-4ca6-b73f-2be146f6e0e0" (UID: "70c4e717-ead1-4ca6-b73f-2be146f6e0e0"). InnerVolumeSpecName "kube-api-access-42rdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:19:15.611220 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.611186    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-config\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:19:15.611220 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.611215    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-oauth-config\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:19:15.611220 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.611225    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-trusted-ca-bundle\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:19:15.611430 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.611234    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-console-serving-cert\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:19:15.611430 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.611242    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-service-ca\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:19:15.611430 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.611251    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42rdd\" (UniqueName: \"kubernetes.io/projected/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-kube-api-access-42rdd\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:19:15.611430 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:15.611259    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/70c4e717-ead1-4ca6-b73f-2be146f6e0e0-oauth-serving-cert\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:19:16.228285 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:16.228251    2576 generic.go:356] "Generic (PLEG): container finished" podID="70c4e717-ead1-4ca6-b73f-2be146f6e0e0" containerID="7e5d55ca25afbeccca264ff428a23540d8443028d43f91f812e85e6215d4ebbd" exitCode=0
Jul 25 01:19:16.228710 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:16.228289    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-756f9c8dcb-z8jpg"
Jul 25 01:19:16.228710 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:16.228315    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-756f9c8dcb-z8jpg" event={"ID":"70c4e717-ead1-4ca6-b73f-2be146f6e0e0","Type":"ContainerDied","Data":"7e5d55ca25afbeccca264ff428a23540d8443028d43f91f812e85e6215d4ebbd"}
Jul 25 01:19:16.228710 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:16.228359    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-756f9c8dcb-z8jpg" event={"ID":"70c4e717-ead1-4ca6-b73f-2be146f6e0e0","Type":"ContainerDied","Data":"421cf6b0f42203555ba0f29a9615dce33f51117c93f9ed9ba29b0fbd41c8cec4"}
Jul 25 01:19:16.228710 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:16.228381    2576 scope.go:117] "RemoveContainer" containerID="7e5d55ca25afbeccca264ff428a23540d8443028d43f91f812e85e6215d4ebbd"
Jul 25 01:19:16.237220 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:16.237203    2576 scope.go:117] "RemoveContainer" containerID="7e5d55ca25afbeccca264ff428a23540d8443028d43f91f812e85e6215d4ebbd"
Jul 25 01:19:16.237473 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:19:16.237457    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e5d55ca25afbeccca264ff428a23540d8443028d43f91f812e85e6215d4ebbd\": container with ID starting with 7e5d55ca25afbeccca264ff428a23540d8443028d43f91f812e85e6215d4ebbd not found: ID does not exist" containerID="7e5d55ca25afbeccca264ff428a23540d8443028d43f91f812e85e6215d4ebbd"
Jul 25 01:19:16.237534 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:16.237485    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e5d55ca25afbeccca264ff428a23540d8443028d43f91f812e85e6215d4ebbd"} err="failed to get container status \"7e5d55ca25afbeccca264ff428a23540d8443028d43f91f812e85e6215d4ebbd\": rpc error: code = NotFound desc = could not find container \"7e5d55ca25afbeccca264ff428a23540d8443028d43f91f812e85e6215d4ebbd\": container with ID starting with 7e5d55ca25afbeccca264ff428a23540d8443028d43f91f812e85e6215d4ebbd not found: ID does not exist"
Jul 25 01:19:16.250426 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:16.250402    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-756f9c8dcb-z8jpg"]
Jul 25 01:19:16.255189 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:16.255168    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-756f9c8dcb-z8jpg"]
Jul 25 01:19:16.995114 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:19:16.995082    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70c4e717-ead1-4ca6-b73f-2be146f6e0e0" path="/var/lib/kubelet/pods/70c4e717-ead1-4ca6-b73f-2be146f6e0e0/volumes"
Jul 25 01:20:04.577846 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.577814    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:20:04.578295 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.578108    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c4e717-ead1-4ca6-b73f-2be146f6e0e0" containerName="console"
Jul 25 01:20:04.578295 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.578119    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c4e717-ead1-4ca6-b73f-2be146f6e0e0" containerName="console"
Jul 25 01:20:04.578295 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.578176    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c4e717-ead1-4ca6-b73f-2be146f6e0e0" containerName="console"
Jul 25 01:20:04.582340 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.582321    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"
Jul 25 01:20:04.584197 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.584182    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 25 01:20:04.584304 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.584202    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cq4b8\""
Jul 25 01:20:04.584304 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.584208    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 25 01:20:04.584786 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.584769    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 25 01:20:04.590592 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.590570    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:20:04.614005 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.613981    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:20:04.693161 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.693129    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvxwd\" (UniqueName: \"kubernetes.io/projected/4e0bf5fd-a6bd-40ff-ac8f-db9f5247fb73-kube-api-access-vvxwd\") pod \"limitador-limitador-54d6d988d5-6kg9k\" (UID: \"4e0bf5fd-a6bd-40ff-ac8f-db9f5247fb73\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"
Jul 25 01:20:04.693291 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.693178    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4e0bf5fd-a6bd-40ff-ac8f-db9f5247fb73-config-file\") pod \"limitador-limitador-54d6d988d5-6kg9k\" (UID: \"4e0bf5fd-a6bd-40ff-ac8f-db9f5247fb73\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"
Jul 25 01:20:04.793826 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.793800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4e0bf5fd-a6bd-40ff-ac8f-db9f5247fb73-config-file\") pod \"limitador-limitador-54d6d988d5-6kg9k\" (UID: \"4e0bf5fd-a6bd-40ff-ac8f-db9f5247fb73\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"
Jul 25 01:20:04.793970 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.793862    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvxwd\" (UniqueName: \"kubernetes.io/projected/4e0bf5fd-a6bd-40ff-ac8f-db9f5247fb73-kube-api-access-vvxwd\") pod \"limitador-limitador-54d6d988d5-6kg9k\" (UID: \"4e0bf5fd-a6bd-40ff-ac8f-db9f5247fb73\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"
Jul 25 01:20:04.794461 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.794443    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4e0bf5fd-a6bd-40ff-ac8f-db9f5247fb73-config-file\") pod \"limitador-limitador-54d6d988d5-6kg9k\" (UID: \"4e0bf5fd-a6bd-40ff-ac8f-db9f5247fb73\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"
Jul 25 01:20:04.800843 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.800825    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvxwd\" (UniqueName: \"kubernetes.io/projected/4e0bf5fd-a6bd-40ff-ac8f-db9f5247fb73-kube-api-access-vvxwd\") pod \"limitador-limitador-54d6d988d5-6kg9k\" (UID: \"4e0bf5fd-a6bd-40ff-ac8f-db9f5247fb73\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"
Jul 25 01:20:04.893460 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:04.893399    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"
Jul 25 01:20:05.024034 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:05.024008    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:20:05.026430 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:20:05.026405    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e0bf5fd_a6bd_40ff_ac8f_db9f5247fb73.slice/crio-f85dd5ab11287a8281ccecd5f2c5644f1a7c545d450626d1412500db042abd38 WatchSource:0}: Error finding container f85dd5ab11287a8281ccecd5f2c5644f1a7c545d450626d1412500db042abd38: Status 404 returned error can't find the container with id f85dd5ab11287a8281ccecd5f2c5644f1a7c545d450626d1412500db042abd38
Jul 25 01:20:05.131382 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:05.131358    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-jrx9j"]
Jul 25 01:20:05.135539 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:05.135523    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jrx9j"
Jul 25 01:20:05.137437 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:05.137417    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bwb92\""
Jul 25 01:20:05.141474 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:05.141453    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jrx9j"]
Jul 25 01:20:05.197560 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:05.197529    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd4g2\" (UniqueName: \"kubernetes.io/projected/36f79987-b05d-4ac0-9db6-4238a851bc20-kube-api-access-gd4g2\") pod \"authorino-75d556bd77-jrx9j\" (UID: \"36f79987-b05d-4ac0-9db6-4238a851bc20\") " pod="kuadrant-system/authorino-75d556bd77-jrx9j"
Jul 25 01:20:05.298815 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:05.298788    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd4g2\" (UniqueName: \"kubernetes.io/projected/36f79987-b05d-4ac0-9db6-4238a851bc20-kube-api-access-gd4g2\") pod \"authorino-75d556bd77-jrx9j\" (UID: \"36f79987-b05d-4ac0-9db6-4238a851bc20\") " pod="kuadrant-system/authorino-75d556bd77-jrx9j"
Jul 25 01:20:05.305809 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:05.305784    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd4g2\" (UniqueName: \"kubernetes.io/projected/36f79987-b05d-4ac0-9db6-4238a851bc20-kube-api-access-gd4g2\") pod \"authorino-75d556bd77-jrx9j\" (UID: \"36f79987-b05d-4ac0-9db6-4238a851bc20\") " pod="kuadrant-system/authorino-75d556bd77-jrx9j"
Jul 25 01:20:05.432386 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:05.432354    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-6kg9k" event={"ID":"4e0bf5fd-a6bd-40ff-ac8f-db9f5247fb73","Type":"ContainerStarted","Data":"f85dd5ab11287a8281ccecd5f2c5644f1a7c545d450626d1412500db042abd38"}
Jul 25 01:20:05.445532 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:05.445508    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jrx9j"
Jul 25 01:20:05.602259 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:05.602226    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jrx9j"]
Jul 25 01:20:05.606498 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:20:05.606466    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36f79987_b05d_4ac0_9db6_4238a851bc20.slice/crio-115a292b1a4ea9bd64634264385986ae7307376cbd46fec8ed0dd2e8f7df9123 WatchSource:0}: Error finding container 115a292b1a4ea9bd64634264385986ae7307376cbd46fec8ed0dd2e8f7df9123: Status 404 returned error can't find the container with id 115a292b1a4ea9bd64634264385986ae7307376cbd46fec8ed0dd2e8f7df9123
Jul 25 01:20:06.440861 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:06.440820    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jrx9j" event={"ID":"36f79987-b05d-4ac0-9db6-4238a851bc20","Type":"ContainerStarted","Data":"115a292b1a4ea9bd64634264385986ae7307376cbd46fec8ed0dd2e8f7df9123"}
Jul 25 01:20:09.457607 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:09.457576    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-6kg9k" event={"ID":"4e0bf5fd-a6bd-40ff-ac8f-db9f5247fb73","Type":"ContainerStarted","Data":"489d1e944d272818a94a98f9b847b75e41f1fd7c0489c9b15a355325b950aa2d"}
Jul 25 01:20:09.459215 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:09.459183    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jrx9j" event={"ID":"36f79987-b05d-4ac0-9db6-4238a851bc20","Type":"ContainerStarted","Data":"ac6183a87f8818652fe638435353f274f69a0b071a3ff0b93a5324eec7290e90"}
Jul 25 01:20:09.473625 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:09.473576    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-jrx9j" podStartSLOduration=0.702199979 podStartE2EDuration="4.473557731s" totalImagesPullingTime="3.771357752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:09.47172377 +0000 UTC m=+560.959020444" watchObservedRunningTime="2026-07-25 01:20:09.473557731 +0000 UTC m=+560.960854406"
Jul 25 01:20:10.482200 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:10.482103    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-6kg9k" podStartSLOduration=2.13151224 podStartE2EDuration="6.482087358s" totalImagesPullingTime="4.350575118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:10.481278435 +0000 UTC m=+561.968575109" watchObservedRunningTime="2026-07-25 01:20:10.482087358 +0000 UTC m=+561.969384032"
Jul 25 01:20:20.464502 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:20.464464    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"
Jul 25 01:20:20.465157 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:20.465134    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"
Jul 25 01:20:38.519006 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.518970    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ck9tk"]
Jul 25 01:20:38.528926 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.528900    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ck9tk"
Jul 25 01:20:38.529085 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.529040    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ck9tk"]
Jul 25 01:20:38.576444 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.576414    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vskbq\" (UniqueName: \"kubernetes.io/projected/c039bc88-c488-4f8b-869e-a512fe52a417-kube-api-access-vskbq\") pod \"authorino-5d4b67c446-ck9tk\" (UID: \"c039bc88-c488-4f8b-869e-a512fe52a417\") " pod="kuadrant-system/authorino-5d4b67c446-ck9tk"
Jul 25 01:20:38.677803 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.677766    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vskbq\" (UniqueName: \"kubernetes.io/projected/c039bc88-c488-4f8b-869e-a512fe52a417-kube-api-access-vskbq\") pod \"authorino-5d4b67c446-ck9tk\" (UID: \"c039bc88-c488-4f8b-869e-a512fe52a417\") " pod="kuadrant-system/authorino-5d4b67c446-ck9tk"
Jul 25 01:20:38.685117 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.685089    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vskbq\" (UniqueName: \"kubernetes.io/projected/c039bc88-c488-4f8b-869e-a512fe52a417-kube-api-access-vskbq\") pod \"authorino-5d4b67c446-ck9tk\" (UID: \"c039bc88-c488-4f8b-869e-a512fe52a417\") " pod="kuadrant-system/authorino-5d4b67c446-ck9tk"
Jul 25 01:20:38.702414 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.702389    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ck9tk"]
Jul 25 01:20:38.702612 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.702600    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ck9tk"
Jul 25 01:20:38.730382 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.730359    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-kn8bf"]
Jul 25 01:20:38.734030 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.734014    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-kn8bf"
Jul 25 01:20:38.735951 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.735930    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 25 01:20:38.742149 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.742126    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-kn8bf"]
Jul 25 01:20:38.778689 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.778585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5aecff77-f974-4444-a8ad-1cdebfb654ce-tls-cert\") pod \"authorino-76877994-kn8bf\" (UID: \"5aecff77-f974-4444-a8ad-1cdebfb654ce\") " pod="kuadrant-system/authorino-76877994-kn8bf"
Jul 25 01:20:38.778689 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.778628    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vbnj\" (UniqueName: \"kubernetes.io/projected/5aecff77-f974-4444-a8ad-1cdebfb654ce-kube-api-access-2vbnj\") pod \"authorino-76877994-kn8bf\" (UID: \"5aecff77-f974-4444-a8ad-1cdebfb654ce\") " pod="kuadrant-system/authorino-76877994-kn8bf"
Jul 25 01:20:38.784081 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.784057    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-kn8bf"]
Jul 25 01:20:38.784288 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:20:38.784267    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-2vbnj tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-kn8bf" podUID="5aecff77-f974-4444-a8ad-1cdebfb654ce"
Jul 25 01:20:38.809981 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.809956    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5556db4fb9-t5f4j"]
Jul 25 01:20:38.813381 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.813364    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5556db4fb9-t5f4j"
Jul 25 01:20:38.819230 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.819203    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5556db4fb9-t5f4j"]
Jul 25 01:20:38.835165 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.835143    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ck9tk"]
Jul 25 01:20:38.838288 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:20:38.838266    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc039bc88_c488_4f8b_869e_a512fe52a417.slice/crio-df22e7802a2d28f01ed84d027c95b8b781e6b0b3b69e3e4c76b299610882f3bb WatchSource:0}: Error finding container df22e7802a2d28f01ed84d027c95b8b781e6b0b3b69e3e4c76b299610882f3bb: Status 404 returned error can't find the container with id df22e7802a2d28f01ed84d027c95b8b781e6b0b3b69e3e4c76b299610882f3bb
Jul 25 01:20:38.879367 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.879340    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5aecff77-f974-4444-a8ad-1cdebfb654ce-tls-cert\") pod \"authorino-76877994-kn8bf\" (UID: \"5aecff77-f974-4444-a8ad-1cdebfb654ce\") " pod="kuadrant-system/authorino-76877994-kn8bf"
Jul 25 01:20:38.879471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.879379    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb26l\" (UniqueName: \"kubernetes.io/projected/c48a52cc-d575-47a6-a331-a65a62b75c38-kube-api-access-bb26l\") pod \"authorino-5556db4fb9-t5f4j\" (UID: \"c48a52cc-d575-47a6-a331-a65a62b75c38\") " pod="kuadrant-system/authorino-5556db4fb9-t5f4j"
Jul 25 01:20:38.879471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.879429    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c48a52cc-d575-47a6-a331-a65a62b75c38-tls-cert\") pod \"authorino-5556db4fb9-t5f4j\" (UID: \"c48a52cc-d575-47a6-a331-a65a62b75c38\") " pod="kuadrant-system/authorino-5556db4fb9-t5f4j"
Jul 25 01:20:38.879471 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.879456    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vbnj\" (UniqueName: \"kubernetes.io/projected/5aecff77-f974-4444-a8ad-1cdebfb654ce-kube-api-access-2vbnj\") pod \"authorino-76877994-kn8bf\" (UID: \"5aecff77-f974-4444-a8ad-1cdebfb654ce\") " pod="kuadrant-system/authorino-76877994-kn8bf"
Jul 25 01:20:38.881581 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.881562    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5aecff77-f974-4444-a8ad-1cdebfb654ce-tls-cert\") pod \"authorino-76877994-kn8bf\" (UID: \"5aecff77-f974-4444-a8ad-1cdebfb654ce\") " pod="kuadrant-system/authorino-76877994-kn8bf"
Jul 25 01:20:38.886709 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.886689    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vbnj\" (UniqueName: \"kubernetes.io/projected/5aecff77-f974-4444-a8ad-1cdebfb654ce-kube-api-access-2vbnj\") pod \"authorino-76877994-kn8bf\" (UID: \"5aecff77-f974-4444-a8ad-1cdebfb654ce\") " pod="kuadrant-system/authorino-76877994-kn8bf"
Jul 25 01:20:38.980412 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.980389    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb26l\" (UniqueName: \"kubernetes.io/projected/c48a52cc-d575-47a6-a331-a65a62b75c38-kube-api-access-bb26l\") pod \"authorino-5556db4fb9-t5f4j\" (UID: \"c48a52cc-d575-47a6-a331-a65a62b75c38\") " pod="kuadrant-system/authorino-5556db4fb9-t5f4j"
Jul 25 01:20:38.980506 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.980423    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c48a52cc-d575-47a6-a331-a65a62b75c38-tls-cert\") pod \"authorino-5556db4fb9-t5f4j\" (UID: \"c48a52cc-d575-47a6-a331-a65a62b75c38\") " pod="kuadrant-system/authorino-5556db4fb9-t5f4j"
Jul 25 01:20:38.982433 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.982418    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c48a52cc-d575-47a6-a331-a65a62b75c38-tls-cert\") pod \"authorino-5556db4fb9-t5f4j\" (UID: \"c48a52cc-d575-47a6-a331-a65a62b75c38\") " pod="kuadrant-system/authorino-5556db4fb9-t5f4j"
Jul 25 01:20:38.992642 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:38.992625    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb26l\" (UniqueName: \"kubernetes.io/projected/c48a52cc-d575-47a6-a331-a65a62b75c38-kube-api-access-bb26l\") pod \"authorino-5556db4fb9-t5f4j\" (UID: \"c48a52cc-d575-47a6-a331-a65a62b75c38\") " pod="kuadrant-system/authorino-5556db4fb9-t5f4j"
Jul 25 01:20:39.123794 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.123724    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5556db4fb9-t5f4j"
Jul 25 01:20:39.252296 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.252270    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5556db4fb9-t5f4j"]
Jul 25 01:20:39.255073 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:20:39.255036    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc48a52cc_d575_47a6_a331_a65a62b75c38.slice/crio-790269c5c695690ca6b183b49bf762956d4efdcd1b3a4df59d1dac9d5b05abc9 WatchSource:0}: Error finding container 790269c5c695690ca6b183b49bf762956d4efdcd1b3a4df59d1dac9d5b05abc9: Status 404 returned error can't find the container with id 790269c5c695690ca6b183b49bf762956d4efdcd1b3a4df59d1dac9d5b05abc9
Jul 25 01:20:39.586144 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.586101    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ck9tk" event={"ID":"c039bc88-c488-4f8b-869e-a512fe52a417","Type":"ContainerStarted","Data":"eb32b31f70a182dbd9b5b85f02d3c7ef3404e77434175baee2f84c879c491910"}
Jul 25 01:20:39.586598 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.586153    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ck9tk" event={"ID":"c039bc88-c488-4f8b-869e-a512fe52a417","Type":"ContainerStarted","Data":"df22e7802a2d28f01ed84d027c95b8b781e6b0b3b69e3e4c76b299610882f3bb"}
Jul 25 01:20:39.586598 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.586144    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-ck9tk" podUID="c039bc88-c488-4f8b-869e-a512fe52a417" containerName="authorino" containerID="cri-o://eb32b31f70a182dbd9b5b85f02d3c7ef3404e77434175baee2f84c879c491910" gracePeriod=30
Jul 25 01:20:39.587422 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.587398    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-kn8bf"
Jul 25 01:20:39.587536 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.587424    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5556db4fb9-t5f4j" event={"ID":"c48a52cc-d575-47a6-a331-a65a62b75c38","Type":"ContainerStarted","Data":"790269c5c695690ca6b183b49bf762956d4efdcd1b3a4df59d1dac9d5b05abc9"}
Jul 25 01:20:39.594104 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.594086    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-kn8bf"
Jul 25 01:20:39.600411 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.600369    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-ck9tk" podStartSLOduration=1.248160062 podStartE2EDuration="1.600356961s" totalImagesPullingTime="352.196899ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:39.598695463 +0000 UTC m=+591.085992137" watchObservedRunningTime="2026-07-25 01:20:39.600356961 +0000 UTC m=+591.087653635"
Jul 25 01:20:39.686237 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.686212    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vbnj\" (UniqueName: \"kubernetes.io/projected/5aecff77-f974-4444-a8ad-1cdebfb654ce-kube-api-access-2vbnj\") pod \"5aecff77-f974-4444-a8ad-1cdebfb654ce\" (UID: \"5aecff77-f974-4444-a8ad-1cdebfb654ce\") "
Jul 25 01:20:39.686353 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.686292    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5aecff77-f974-4444-a8ad-1cdebfb654ce-tls-cert\") pod \"5aecff77-f974-4444-a8ad-1cdebfb654ce\" (UID: \"5aecff77-f974-4444-a8ad-1cdebfb654ce\") "
Jul 25 01:20:39.688264 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.688240    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5aecff77-f974-4444-a8ad-1cdebfb654ce-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5aecff77-f974-4444-a8ad-1cdebfb654ce" (UID: "5aecff77-f974-4444-a8ad-1cdebfb654ce"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:20:39.688264 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.688250    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5aecff77-f974-4444-a8ad-1cdebfb654ce-kube-api-access-2vbnj" (OuterVolumeSpecName: "kube-api-access-2vbnj") pod "5aecff77-f974-4444-a8ad-1cdebfb654ce" (UID: "5aecff77-f974-4444-a8ad-1cdebfb654ce"). InnerVolumeSpecName "kube-api-access-2vbnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:20:39.787322 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.787286    2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5aecff77-f974-4444-a8ad-1cdebfb654ce-tls-cert\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:20:39.787322 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.787313    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vbnj\" (UniqueName: \"kubernetes.io/projected/5aecff77-f974-4444-a8ad-1cdebfb654ce-kube-api-access-2vbnj\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:20:39.834486 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.834464    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ck9tk"
Jul 25 01:20:39.888122 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.888094    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vskbq\" (UniqueName: \"kubernetes.io/projected/c039bc88-c488-4f8b-869e-a512fe52a417-kube-api-access-vskbq\") pod \"c039bc88-c488-4f8b-869e-a512fe52a417\" (UID: \"c039bc88-c488-4f8b-869e-a512fe52a417\") "
Jul 25 01:20:39.890116 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.890093    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c039bc88-c488-4f8b-869e-a512fe52a417-kube-api-access-vskbq" (OuterVolumeSpecName: "kube-api-access-vskbq") pod "c039bc88-c488-4f8b-869e-a512fe52a417" (UID: "c039bc88-c488-4f8b-869e-a512fe52a417"). InnerVolumeSpecName "kube-api-access-vskbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:20:39.988938 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:39.988905    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vskbq\" (UniqueName: \"kubernetes.io/projected/c039bc88-c488-4f8b-869e-a512fe52a417-kube-api-access-vskbq\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:20:40.593260 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.593220    2576 generic.go:356] "Generic (PLEG): container finished" podID="c039bc88-c488-4f8b-869e-a512fe52a417" containerID="eb32b31f70a182dbd9b5b85f02d3c7ef3404e77434175baee2f84c879c491910" exitCode=0
Jul 25 01:20:40.593713 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.593284    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-ck9tk"
Jul 25 01:20:40.593713 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.593315    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ck9tk" event={"ID":"c039bc88-c488-4f8b-869e-a512fe52a417","Type":"ContainerDied","Data":"eb32b31f70a182dbd9b5b85f02d3c7ef3404e77434175baee2f84c879c491910"}
Jul 25 01:20:40.593713 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.593359    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-ck9tk" event={"ID":"c039bc88-c488-4f8b-869e-a512fe52a417","Type":"ContainerDied","Data":"df22e7802a2d28f01ed84d027c95b8b781e6b0b3b69e3e4c76b299610882f3bb"}
Jul 25 01:20:40.593713 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.593376    2576 scope.go:117] "RemoveContainer" containerID="eb32b31f70a182dbd9b5b85f02d3c7ef3404e77434175baee2f84c879c491910"
Jul 25 01:20:40.594921 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.594899    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-kn8bf"
Jul 25 01:20:40.594921 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.594909    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5556db4fb9-t5f4j" event={"ID":"c48a52cc-d575-47a6-a331-a65a62b75c38","Type":"ContainerStarted","Data":"2c749055f5f020337786f6731b000ff34cc8f0ee8b71e0a866a0d23c655413ce"}
Jul 25 01:20:40.602336 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.602321    2576 scope.go:117] "RemoveContainer" containerID="eb32b31f70a182dbd9b5b85f02d3c7ef3404e77434175baee2f84c879c491910"
Jul 25 01:20:40.602623 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:20:40.602602    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb32b31f70a182dbd9b5b85f02d3c7ef3404e77434175baee2f84c879c491910\": container with ID starting with eb32b31f70a182dbd9b5b85f02d3c7ef3404e77434175baee2f84c879c491910 not found: ID does not exist" containerID="eb32b31f70a182dbd9b5b85f02d3c7ef3404e77434175baee2f84c879c491910"
Jul 25 01:20:40.602755 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.602633    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb32b31f70a182dbd9b5b85f02d3c7ef3404e77434175baee2f84c879c491910"} err="failed to get container status \"eb32b31f70a182dbd9b5b85f02d3c7ef3404e77434175baee2f84c879c491910\": rpc error: code = NotFound desc = could not find container \"eb32b31f70a182dbd9b5b85f02d3c7ef3404e77434175baee2f84c879c491910\": container with ID starting with eb32b31f70a182dbd9b5b85f02d3c7ef3404e77434175baee2f84c879c491910 not found: ID does not exist"
Jul 25 01:20:40.611340 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.611297    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5556db4fb9-t5f4j" podStartSLOduration=2.057347001 podStartE2EDuration="2.611283312s" totalImagesPullingTime="553.936311ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:40.610925343 +0000 UTC m=+592.098222017" watchObservedRunningTime="2026-07-25 01:20:40.611283312 +0000 UTC m=+592.098579990"
Jul 25 01:20:40.635611 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.635586    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jrx9j"]
Jul 25 01:20:40.635791 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.635770    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-jrx9j" podUID="36f79987-b05d-4ac0-9db6-4238a851bc20" containerName="authorino" containerID="cri-o://ac6183a87f8818652fe638435353f274f69a0b071a3ff0b93a5324eec7290e90" gracePeriod=30
Jul 25 01:20:40.639861 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.639838    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-kn8bf"]
Jul 25 01:20:40.644519 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.644498    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-kn8bf"]
Jul 25 01:20:40.654188 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.654162    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ck9tk"]
Jul 25 01:20:40.658584 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.658563    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-ck9tk"]
Jul 25 01:20:40.873825 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.873798    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jrx9j"
Jul 25 01:20:40.896134 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.896105    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gd4g2\" (UniqueName: \"kubernetes.io/projected/36f79987-b05d-4ac0-9db6-4238a851bc20-kube-api-access-gd4g2\") pod \"36f79987-b05d-4ac0-9db6-4238a851bc20\" (UID: \"36f79987-b05d-4ac0-9db6-4238a851bc20\") "
Jul 25 01:20:40.898109 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.898084    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36f79987-b05d-4ac0-9db6-4238a851bc20-kube-api-access-gd4g2" (OuterVolumeSpecName: "kube-api-access-gd4g2") pod "36f79987-b05d-4ac0-9db6-4238a851bc20" (UID: "36f79987-b05d-4ac0-9db6-4238a851bc20"). InnerVolumeSpecName "kube-api-access-gd4g2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:20:40.996039 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.996008    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5aecff77-f974-4444-a8ad-1cdebfb654ce" path="/var/lib/kubelet/pods/5aecff77-f974-4444-a8ad-1cdebfb654ce/volumes"
Jul 25 01:20:40.996255 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.996242    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c039bc88-c488-4f8b-869e-a512fe52a417" path="/var/lib/kubelet/pods/c039bc88-c488-4f8b-869e-a512fe52a417/volumes"
Jul 25 01:20:40.996841 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:40.996824    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gd4g2\" (UniqueName: \"kubernetes.io/projected/36f79987-b05d-4ac0-9db6-4238a851bc20-kube-api-access-gd4g2\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:20:41.601503 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:41.601471    2576 generic.go:356] "Generic (PLEG): container finished" podID="36f79987-b05d-4ac0-9db6-4238a851bc20" containerID="ac6183a87f8818652fe638435353f274f69a0b071a3ff0b93a5324eec7290e90" exitCode=0
Jul 25 01:20:41.601503 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:41.601507    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-jrx9j"
Jul 25 01:20:41.602060 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:41.601562    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jrx9j" event={"ID":"36f79987-b05d-4ac0-9db6-4238a851bc20","Type":"ContainerDied","Data":"ac6183a87f8818652fe638435353f274f69a0b071a3ff0b93a5324eec7290e90"}
Jul 25 01:20:41.602060 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:41.601611    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-jrx9j" event={"ID":"36f79987-b05d-4ac0-9db6-4238a851bc20","Type":"ContainerDied","Data":"115a292b1a4ea9bd64634264385986ae7307376cbd46fec8ed0dd2e8f7df9123"}
Jul 25 01:20:41.602060 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:41.601635    2576 scope.go:117] "RemoveContainer" containerID="ac6183a87f8818652fe638435353f274f69a0b071a3ff0b93a5324eec7290e90"
Jul 25 01:20:41.609662 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:41.609639    2576 scope.go:117] "RemoveContainer" containerID="ac6183a87f8818652fe638435353f274f69a0b071a3ff0b93a5324eec7290e90"
Jul 25 01:20:41.609914 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:20:41.609899    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac6183a87f8818652fe638435353f274f69a0b071a3ff0b93a5324eec7290e90\": container with ID starting with ac6183a87f8818652fe638435353f274f69a0b071a3ff0b93a5324eec7290e90 not found: ID does not exist" containerID="ac6183a87f8818652fe638435353f274f69a0b071a3ff0b93a5324eec7290e90"
Jul 25 01:20:41.609952 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:41.609923    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac6183a87f8818652fe638435353f274f69a0b071a3ff0b93a5324eec7290e90"} err="failed to get container status \"ac6183a87f8818652fe638435353f274f69a0b071a3ff0b93a5324eec7290e90\": rpc error: code = NotFound desc = could not find container \"ac6183a87f8818652fe638435353f274f69a0b071a3ff0b93a5324eec7290e90\": container with ID starting with ac6183a87f8818652fe638435353f274f69a0b071a3ff0b93a5324eec7290e90 not found: ID does not exist"
Jul 25 01:20:41.615614 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:41.615594    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jrx9j"]
Jul 25 01:20:41.619756 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:41.619737    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-jrx9j"]
Jul 25 01:20:42.995948 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:42.995914    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36f79987-b05d-4ac0-9db6-4238a851bc20" path="/var/lib/kubelet/pods/36f79987-b05d-4ac0-9db6-4238a851bc20/volumes"
Jul 25 01:20:48.954098 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:48.954066    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:20:48.958590 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:48.958570    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:20:48.962132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:48.962115    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:20:48.966615 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:20:48.966598    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:21:21.681177 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:21:21.681097    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:21:49.535566 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:21:49.535530    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:22:01.219707 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:22:01.219647    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:22:20.326366 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:22:20.326326    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:22:29.257768 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:22:29.257735    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:22:51.023828 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:22:51.023748    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:23:03.830820 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.830779    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"]
Jul 25 01:23:03.831218 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.831172    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c039bc88-c488-4f8b-869e-a512fe52a417" containerName="authorino"
Jul 25 01:23:03.831218 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.831189    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c039bc88-c488-4f8b-869e-a512fe52a417" containerName="authorino"
Jul 25 01:23:03.831218 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.831214    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36f79987-b05d-4ac0-9db6-4238a851bc20" containerName="authorino"
Jul 25 01:23:03.831325 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.831223    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="36f79987-b05d-4ac0-9db6-4238a851bc20" containerName="authorino"
Jul 25 01:23:03.831325 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.831306    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c039bc88-c488-4f8b-869e-a512fe52a417" containerName="authorino"
Jul 25 01:23:03.831388 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.831324    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="36f79987-b05d-4ac0-9db6-4238a851bc20" containerName="authorino"
Jul 25 01:23:03.834340 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.834322    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:03.836248 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.836226    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 25 01:23:03.836453 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.836436    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 25 01:23:03.836529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.836438    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-rj5g5\""
Jul 25 01:23:03.836529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.836445    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 25 01:23:03.845132 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.845110    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"]
Jul 25 01:23:03.907911 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.907880    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x99s5\" (UniqueName: \"kubernetes.io/projected/9998c918-0a90-4f65-92a2-8fe3d8dcf622-kube-api-access-x99s5\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:03.907911 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.907914    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9998c918-0a90-4f65-92a2-8fe3d8dcf622-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:03.908101 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.907939    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9998c918-0a90-4f65-92a2-8fe3d8dcf622-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:03.908101 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.907996    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9998c918-0a90-4f65-92a2-8fe3d8dcf622-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:03.908101 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.908030    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9998c918-0a90-4f65-92a2-8fe3d8dcf622-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:03.908101 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:03.908045    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9998c918-0a90-4f65-92a2-8fe3d8dcf622-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:04.009216 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:04.009183    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9998c918-0a90-4f65-92a2-8fe3d8dcf622-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:04.009379 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:04.009242    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9998c918-0a90-4f65-92a2-8fe3d8dcf622-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:04.009379 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:04.009262    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9998c918-0a90-4f65-92a2-8fe3d8dcf622-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:04.009379 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:04.009299    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x99s5\" (UniqueName: \"kubernetes.io/projected/9998c918-0a90-4f65-92a2-8fe3d8dcf622-kube-api-access-x99s5\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:04.009379 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:04.009321    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9998c918-0a90-4f65-92a2-8fe3d8dcf622-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:04.009379 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:04.009361    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9998c918-0a90-4f65-92a2-8fe3d8dcf622-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:04.009769 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:04.009743    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9998c918-0a90-4f65-92a2-8fe3d8dcf622-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:04.009846 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:04.009769    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9998c918-0a90-4f65-92a2-8fe3d8dcf622-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:04.009846 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:04.009816    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9998c918-0a90-4f65-92a2-8fe3d8dcf622-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:04.011385 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:04.011362    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9998c918-0a90-4f65-92a2-8fe3d8dcf622-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:04.011683 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:04.011666    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9998c918-0a90-4f65-92a2-8fe3d8dcf622-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:04.017601 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:04.017576    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x99s5\" (UniqueName: \"kubernetes.io/projected/9998c918-0a90-4f65-92a2-8fe3d8dcf622-kube-api-access-x99s5\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j77dk\" (UID: \"9998c918-0a90-4f65-92a2-8fe3d8dcf622\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:04.143428 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:04.143348    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:04.270727 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:04.270700    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"]
Jul 25 01:23:04.273682 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:23:04.273633    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9998c918_0a90_4f65_92a2_8fe3d8dcf622.slice/crio-dd490f49ca87a90aa41ef1dfc97131a8035651515295aa490f7420ca1260fc28 WatchSource:0}: Error finding container dd490f49ca87a90aa41ef1dfc97131a8035651515295aa490f7420ca1260fc28: Status 404 returned error can't find the container with id dd490f49ca87a90aa41ef1dfc97131a8035651515295aa490f7420ca1260fc28
Jul 25 01:23:04.274940 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:04.274926    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 01:23:05.210260 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:05.210220    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk" event={"ID":"9998c918-0a90-4f65-92a2-8fe3d8dcf622","Type":"ContainerStarted","Data":"dd490f49ca87a90aa41ef1dfc97131a8035651515295aa490f7420ca1260fc28"}
Jul 25 01:23:06.216365 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:06.216323    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk" event={"ID":"9998c918-0a90-4f65-92a2-8fe3d8dcf622","Type":"ContainerStarted","Data":"db736cd0343154e954dbdaf4301c789f422ff11dfa4940e2d55970269d6ba677"}
Jul 25 01:23:06.232424 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:06.232379    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk" podStartSLOduration=2.252109497 podStartE2EDuration="3.232363011s" totalImagesPullingTime="980.253514ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:23:06.231247736 +0000 UTC m=+737.718544410" watchObservedRunningTime="2026-07-25 01:23:06.232363011 +0000 UTC m=+737.719659732"
Jul 25 01:23:07.141631 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:07.141592    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:23:14.144416 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:14.144367    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:14.144823 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:14.144433    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:14.156551 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:14.156530    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:14.264727 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:14.264702    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j77dk"
Jul 25 01:23:57.188332 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:57.188299    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c997bfc44-rzjrt"]
Jul 25 01:23:57.191627 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:57.191609    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c997bfc44-rzjrt"
Jul 25 01:23:57.197246 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:57.197220    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c997bfc44-rzjrt"]
Jul 25 01:23:57.334468 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:57.334441    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bca994f4-be7f-4182-b54a-3a3685227b88-tls-cert\") pod \"authorino-6c997bfc44-rzjrt\" (UID: \"bca994f4-be7f-4182-b54a-3a3685227b88\") " pod="kuadrant-system/authorino-6c997bfc44-rzjrt"
Jul 25 01:23:57.334616 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:57.334472    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plfk9\" (UniqueName: \"kubernetes.io/projected/bca994f4-be7f-4182-b54a-3a3685227b88-kube-api-access-plfk9\") pod \"authorino-6c997bfc44-rzjrt\" (UID: \"bca994f4-be7f-4182-b54a-3a3685227b88\") " pod="kuadrant-system/authorino-6c997bfc44-rzjrt"
Jul 25 01:23:57.435306 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:57.435276    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bca994f4-be7f-4182-b54a-3a3685227b88-tls-cert\") pod \"authorino-6c997bfc44-rzjrt\" (UID: \"bca994f4-be7f-4182-b54a-3a3685227b88\") " pod="kuadrant-system/authorino-6c997bfc44-rzjrt"
Jul 25 01:23:57.435426 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:57.435310    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plfk9\" (UniqueName: \"kubernetes.io/projected/bca994f4-be7f-4182-b54a-3a3685227b88-kube-api-access-plfk9\") pod \"authorino-6c997bfc44-rzjrt\" (UID: \"bca994f4-be7f-4182-b54a-3a3685227b88\") " pod="kuadrant-system/authorino-6c997bfc44-rzjrt"
Jul 25 01:23:57.437587 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:57.437559    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bca994f4-be7f-4182-b54a-3a3685227b88-tls-cert\") pod \"authorino-6c997bfc44-rzjrt\" (UID: \"bca994f4-be7f-4182-b54a-3a3685227b88\") " pod="kuadrant-system/authorino-6c997bfc44-rzjrt"
Jul 25 01:23:57.442914 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:57.442859    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plfk9\" (UniqueName: \"kubernetes.io/projected/bca994f4-be7f-4182-b54a-3a3685227b88-kube-api-access-plfk9\") pod \"authorino-6c997bfc44-rzjrt\" (UID: \"bca994f4-be7f-4182-b54a-3a3685227b88\") " pod="kuadrant-system/authorino-6c997bfc44-rzjrt"
Jul 25 01:23:57.501821 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:57.501788    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c997bfc44-rzjrt"
Jul 25 01:23:57.635512 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:57.635483    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c997bfc44-rzjrt"]
Jul 25 01:23:57.638777 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:23:57.638751    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbca994f4_be7f_4182_b54a_3a3685227b88.slice/crio-3af28e148f55d107703ad6ddfb0557e4afa9cff83dae890ae77492a7de6a568d WatchSource:0}: Error finding container 3af28e148f55d107703ad6ddfb0557e4afa9cff83dae890ae77492a7de6a568d: Status 404 returned error can't find the container with id 3af28e148f55d107703ad6ddfb0557e4afa9cff83dae890ae77492a7de6a568d
Jul 25 01:23:58.448446 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:58.448404    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c997bfc44-rzjrt" event={"ID":"bca994f4-be7f-4182-b54a-3a3685227b88","Type":"ContainerStarted","Data":"fbec5a23e270e40dc4c6ac89eb7dbe307be49eadb246e01c6e0c48a0f785f259"}
Jul 25 01:23:58.448974 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:58.448456    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c997bfc44-rzjrt" event={"ID":"bca994f4-be7f-4182-b54a-3a3685227b88","Type":"ContainerStarted","Data":"3af28e148f55d107703ad6ddfb0557e4afa9cff83dae890ae77492a7de6a568d"}
Jul 25 01:23:58.473299 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:58.473242    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c997bfc44-rzjrt" podStartSLOduration=1.087595004 podStartE2EDuration="1.473222459s" totalImagesPullingTime="385.627455ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:23:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:23:58.472617569 +0000 UTC m=+789.959914298" watchObservedRunningTime="2026-07-25 01:23:58.473222459 +0000 UTC m=+789.960519126"
Jul 25 01:23:58.534400 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:58.534360    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5556db4fb9-t5f4j"]
Jul 25 01:23:58.534602 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:58.534563    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5556db4fb9-t5f4j" podUID="c48a52cc-d575-47a6-a331-a65a62b75c38" containerName="authorino" containerID="cri-o://2c749055f5f020337786f6731b000ff34cc8f0ee8b71e0a866a0d23c655413ce" gracePeriod=30
Jul 25 01:23:58.808142 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:58.808113    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5556db4fb9-t5f4j"
Jul 25 01:23:58.946901 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:58.946860    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bb26l\" (UniqueName: \"kubernetes.io/projected/c48a52cc-d575-47a6-a331-a65a62b75c38-kube-api-access-bb26l\") pod \"c48a52cc-d575-47a6-a331-a65a62b75c38\" (UID: \"c48a52cc-d575-47a6-a331-a65a62b75c38\") "
Jul 25 01:23:58.947063 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:58.946935    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c48a52cc-d575-47a6-a331-a65a62b75c38-tls-cert\") pod \"c48a52cc-d575-47a6-a331-a65a62b75c38\" (UID: \"c48a52cc-d575-47a6-a331-a65a62b75c38\") "
Jul 25 01:23:58.949136 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:58.949098    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c48a52cc-d575-47a6-a331-a65a62b75c38-kube-api-access-bb26l" (OuterVolumeSpecName: "kube-api-access-bb26l") pod "c48a52cc-d575-47a6-a331-a65a62b75c38" (UID: "c48a52cc-d575-47a6-a331-a65a62b75c38"). InnerVolumeSpecName "kube-api-access-bb26l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:23:58.957827 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:58.957793    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c48a52cc-d575-47a6-a331-a65a62b75c38-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c48a52cc-d575-47a6-a331-a65a62b75c38" (UID: "c48a52cc-d575-47a6-a331-a65a62b75c38"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:23:59.048094 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:59.048063    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb26l\" (UniqueName: \"kubernetes.io/projected/c48a52cc-d575-47a6-a331-a65a62b75c38-kube-api-access-bb26l\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:23:59.048094 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:59.048095    2576 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c48a52cc-d575-47a6-a331-a65a62b75c38-tls-cert\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:23:59.454264 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:59.454227    2576 generic.go:356] "Generic (PLEG): container finished" podID="c48a52cc-d575-47a6-a331-a65a62b75c38" containerID="2c749055f5f020337786f6731b000ff34cc8f0ee8b71e0a866a0d23c655413ce" exitCode=0
Jul 25 01:23:59.454630 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:59.454294    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5556db4fb9-t5f4j"
Jul 25 01:23:59.454630 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:59.454311    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5556db4fb9-t5f4j" event={"ID":"c48a52cc-d575-47a6-a331-a65a62b75c38","Type":"ContainerDied","Data":"2c749055f5f020337786f6731b000ff34cc8f0ee8b71e0a866a0d23c655413ce"}
Jul 25 01:23:59.454630 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:59.454366    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5556db4fb9-t5f4j" event={"ID":"c48a52cc-d575-47a6-a331-a65a62b75c38","Type":"ContainerDied","Data":"790269c5c695690ca6b183b49bf762956d4efdcd1b3a4df59d1dac9d5b05abc9"}
Jul 25 01:23:59.454630 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:59.454383    2576 scope.go:117] "RemoveContainer" containerID="2c749055f5f020337786f6731b000ff34cc8f0ee8b71e0a866a0d23c655413ce"
Jul 25 01:23:59.463761 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:59.463745    2576 scope.go:117] "RemoveContainer" containerID="2c749055f5f020337786f6731b000ff34cc8f0ee8b71e0a866a0d23c655413ce"
Jul 25 01:23:59.464023 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:23:59.464003    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c749055f5f020337786f6731b000ff34cc8f0ee8b71e0a866a0d23c655413ce\": container with ID starting with 2c749055f5f020337786f6731b000ff34cc8f0ee8b71e0a866a0d23c655413ce not found: ID does not exist" containerID="2c749055f5f020337786f6731b000ff34cc8f0ee8b71e0a866a0d23c655413ce"
Jul 25 01:23:59.464092 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:59.464033    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c749055f5f020337786f6731b000ff34cc8f0ee8b71e0a866a0d23c655413ce"} err="failed to get container status \"2c749055f5f020337786f6731b000ff34cc8f0ee8b71e0a866a0d23c655413ce\": rpc error: code = NotFound desc = could not find container \"2c749055f5f020337786f6731b000ff34cc8f0ee8b71e0a866a0d23c655413ce\": container with ID starting with 2c749055f5f020337786f6731b000ff34cc8f0ee8b71e0a866a0d23c655413ce not found: ID does not exist"
Jul 25 01:23:59.470713 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:59.470691    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5556db4fb9-t5f4j"]
Jul 25 01:23:59.474245 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:23:59.474224    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5556db4fb9-t5f4j"]
Jul 25 01:24:00.996068 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:24:00.996038    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c48a52cc-d575-47a6-a331-a65a62b75c38" path="/var/lib/kubelet/pods/c48a52cc-d575-47a6-a331-a65a62b75c38/volumes"
Jul 25 01:24:34.128066 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:24:34.128028    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:24:44.926669 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:24:44.926624    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:24:57.332859 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:24:57.332822    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:25:48.980778 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:25:48.980698    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:25:48.983331 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:25:48.983310    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:25:48.989147 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:25:48.989123    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:25:48.991549 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:25:48.991526    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:26:38.030446 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:26:38.030403    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:26:47.227935 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:26:47.227896    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:27:15.230686 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:27:15.230583    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:27:23.719854 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:27:23.719813    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:27:34.224259 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:27:34.224226    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:27:41.822583 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:27:41.822548    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:27:51.230798 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:27:51.230767    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:28:02.728851 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:28:02.728813    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:28:12.823593 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:28:12.823555    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:28:22.937750 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:28:22.937709    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:28:44.425791 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:28:44.425715    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:28:53.229929 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:28:53.229892    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:29:16.632221 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:29:16.632177    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:29:28.725390 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:29:28.725348    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:29:44.331976 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:29:44.331939    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:29:53.629110 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:29:53.629078    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:30:03.235745 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:30:03.235709    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:30:12.228902 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:30:12.228811    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:30:21.922498 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:30:21.922460    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:30:49.008084 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:30:49.008052    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:30:49.008627 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:30:49.008184    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:30:49.018987 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:30:49.018966    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:30:49.019158 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:30:49.019141    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:32:09.333250 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:32:09.333218    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:32:18.328630 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:32:18.328584    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:32:33.324967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:32:33.324931    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:32:46.321323 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:32:46.321283    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:32:55.520253 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:32:55.520221    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:33:06.126685 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:33:06.126638    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:33:16.923665 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:33:16.923570    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:33:26.324834 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:33:26.324799    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:33:36.822807 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:33:36.822773    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:33:47.127693 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:33:47.127646    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:33:57.223496 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:33:57.223464    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:34:19.225030 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:19.224997    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:34:27.429006 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.428967    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"]
Jul 25 01:34:27.429557 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.429275    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c48a52cc-d575-47a6-a331-a65a62b75c38" containerName="authorino"
Jul 25 01:34:27.429557 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.429286    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c48a52cc-d575-47a6-a331-a65a62b75c38" containerName="authorino"
Jul 25 01:34:27.429557 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.429344    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c48a52cc-d575-47a6-a331-a65a62b75c38" containerName="authorino"
Jul 25 01:34:27.432269 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.432250    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.434153 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.434131    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-ca5a4e04-kserve-self-signed-certs\""
Jul 25 01:34:27.445353 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.445333    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"]
Jul 25 01:34:27.484614 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.484582    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hps9h\" (UniqueName: \"kubernetes.io/projected/b525dac2-5ce8-412f-b581-e0f861fd7874-kube-api-access-hps9h\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.484747 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.484617    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b525dac2-5ce8-412f-b581-e0f861fd7874-tls-certs\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.484747 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.484645    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-home\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.484747 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.484731    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-model-cache\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.484852 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.484755    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-tmp-dir\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.484852 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.484790    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-dshm\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.585918 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.585883    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-dshm\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.586095 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.586018    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hps9h\" (UniqueName: \"kubernetes.io/projected/b525dac2-5ce8-412f-b581-e0f861fd7874-kube-api-access-hps9h\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.586095 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.586050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b525dac2-5ce8-412f-b581-e0f861fd7874-tls-certs\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.586095 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.586077    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-home\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.586258 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.586185    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-model-cache\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.586258 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.586219    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-tmp-dir\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.586443 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.586412    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-home\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.586571 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.586477    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-model-cache\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.586571 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.586561    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-tmp-dir\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.588149 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.588125    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-dshm\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.588383 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.588367    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b525dac2-5ce8-412f-b581-e0f861fd7874-tls-certs\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.592864 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.592844    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hps9h\" (UniqueName: \"kubernetes.io/projected/b525dac2-5ce8-412f-b581-e0f861fd7874-kube-api-access-hps9h\") pod \"e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") " pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.741078 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.740989    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:27.871201 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:27.871127    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"]
Jul 25 01:34:27.874008 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:34:27.873971    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb525dac2_5ce8_412f_b581_e0f861fd7874.slice/crio-4ca8f8f90d2585019295af818f39f37adf890e3073e284f27a071b88028edb4a WatchSource:0}: Error finding container 4ca8f8f90d2585019295af818f39f37adf890e3073e284f27a071b88028edb4a: Status 404 returned error can't find the container with id 4ca8f8f90d2585019295af818f39f37adf890e3073e284f27a071b88028edb4a
Jul 25 01:34:28.204611 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:28.204560    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw" event={"ID":"b525dac2-5ce8-412f-b581-e0f861fd7874","Type":"ContainerStarted","Data":"2dfb9277448bcf7eb9501d11fb03e02238a7d6ee77a0932d7df7d5789f9423cf"}
Jul 25 01:34:28.204611 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:28.204610    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw" event={"ID":"b525dac2-5ce8-412f-b581-e0f861fd7874","Type":"ContainerStarted","Data":"4ca8f8f90d2585019295af818f39f37adf890e3073e284f27a071b88028edb4a"}
Jul 25 01:34:28.219904 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:28.219858    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw" podStartSLOduration=1.219845284 podStartE2EDuration="1.219845284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:34:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:34:28.218676126 +0000 UTC m=+1419.705972806" watchObservedRunningTime="2026-07-25 01:34:28.219845284 +0000 UTC m=+1419.707141960"
Jul 25 01:34:28.330225 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:28.330190    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:34:31.416903 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.416868    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"]
Jul 25 01:34:31.420213 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.420192    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.422192 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.422171    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-ca5a4e04-kserve-self-signed-certs\""
Jul 25 01:34:31.429706 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.429684    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"]
Jul 25 01:34:31.519507 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.519474    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-model-cache\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.519650 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.519518    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cee828a-7f49-4298-91a9-f9daced8ba64-tls-certs\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.519650 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.519628    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-home\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.519777 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.519683    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-tmp-dir\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.519777 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.519715    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-dshm\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.519777 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.519741    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6rsr\" (UniqueName: \"kubernetes.io/projected/5cee828a-7f49-4298-91a9-f9daced8ba64-kube-api-access-f6rsr\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.620625 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.620589    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-home\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.620625 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.620624    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-tmp-dir\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.620846 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.620687    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-dshm\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.620846 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.620724    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6rsr\" (UniqueName: \"kubernetes.io/projected/5cee828a-7f49-4298-91a9-f9daced8ba64-kube-api-access-f6rsr\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.620919 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.620870    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-model-cache\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.620967 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.620932    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cee828a-7f49-4298-91a9-f9daced8ba64-tls-certs\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.621071 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.621050    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-home\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.621154 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.621091    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-tmp-dir\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.621231 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.621212    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-model-cache\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.622970 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.622938    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-dshm\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.623370 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.623348    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cee828a-7f49-4298-91a9-f9daced8ba64-tls-certs\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.627869 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.627852    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6rsr\" (UniqueName: \"kubernetes.io/projected/5cee828a-7f49-4298-91a9-f9daced8ba64-kube-api-access-f6rsr\") pod \"e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") " pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.729760 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.729680    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:31.857296 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:31.857267    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"]
Jul 25 01:34:31.860977 ip-10-0-134-238 kubenswrapper[2576]: W0725 01:34:31.860943    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5cee828a_7f49_4298_91a9_f9daced8ba64.slice/crio-ad6ba8609b7e38bd14f41a8172e09ce940c9c618c720325e1bc3d0b897bd6d70 WatchSource:0}: Error finding container ad6ba8609b7e38bd14f41a8172e09ce940c9c618c720325e1bc3d0b897bd6d70: Status 404 returned error can't find the container with id ad6ba8609b7e38bd14f41a8172e09ce940c9c618c720325e1bc3d0b897bd6d70
Jul 25 01:34:32.226124 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:32.226095    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn" event={"ID":"5cee828a-7f49-4298-91a9-f9daced8ba64","Type":"ContainerStarted","Data":"56159f0cfc5f34c60f6d877fdb1655b21559ab440049d9774c48ea433119eac5"}
Jul 25 01:34:32.226124 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:32.226132    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn" event={"ID":"5cee828a-7f49-4298-91a9-f9daced8ba64","Type":"ContainerStarted","Data":"ad6ba8609b7e38bd14f41a8172e09ce940c9c618c720325e1bc3d0b897bd6d70"}
Jul 25 01:34:32.242956 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:32.242912    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn" podStartSLOduration=1.242897554 podStartE2EDuration="1.242897554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:34:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:34:32.241464529 +0000 UTC m=+1423.728761216" watchObservedRunningTime="2026-07-25 01:34:32.242897554 +0000 UTC m=+1423.730194229"
Jul 25 01:34:37.742136 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:37.742055    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:37.742136 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:37.742126    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:37.754453 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:37.754429    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:38.263748 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:38.263719    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:34:41.730454 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:41.730384    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:41.730826 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:41.730454    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:41.742855 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:41.742833    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:42.281453 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:42.281423    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:34:45.723494 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:34:45.723463    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:35:12.024887 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:12.024851    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"]
Jul 25 01:35:13.134160 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:13.134130    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"]
Jul 25 01:35:16.125193 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:16.125156    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:35:27.043884 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.043840    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw" podUID="b525dac2-5ce8-412f-b581-e0f861fd7874" containerName="main" containerID="cri-o://2dfb9277448bcf7eb9501d11fb03e02238a7d6ee77a0932d7df7d5789f9423cf" gracePeriod=45
Jul 25 01:35:27.288304 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.288282    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:35:27.365747 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.365683    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-home\") pod \"b525dac2-5ce8-412f-b581-e0f861fd7874\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") "
Jul 25 01:35:27.365747 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.365727    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-model-cache\") pod \"b525dac2-5ce8-412f-b581-e0f861fd7874\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") "
Jul 25 01:35:27.365896 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.365761    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-tmp-dir\") pod \"b525dac2-5ce8-412f-b581-e0f861fd7874\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") "
Jul 25 01:35:27.365896 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.365782    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b525dac2-5ce8-412f-b581-e0f861fd7874-tls-certs\") pod \"b525dac2-5ce8-412f-b581-e0f861fd7874\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") "
Jul 25 01:35:27.365896 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.365797    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-dshm\") pod \"b525dac2-5ce8-412f-b581-e0f861fd7874\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") "
Jul 25 01:35:27.365896 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.365857    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hps9h\" (UniqueName: \"kubernetes.io/projected/b525dac2-5ce8-412f-b581-e0f861fd7874-kube-api-access-hps9h\") pod \"b525dac2-5ce8-412f-b581-e0f861fd7874\" (UID: \"b525dac2-5ce8-412f-b581-e0f861fd7874\") "
Jul 25 01:35:27.366064 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.366009    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b525dac2-5ce8-412f-b581-e0f861fd7874" (UID: "b525dac2-5ce8-412f-b581-e0f861fd7874"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:35:27.366570 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.365978    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-home" (OuterVolumeSpecName: "home") pod "b525dac2-5ce8-412f-b581-e0f861fd7874" (UID: "b525dac2-5ce8-412f-b581-e0f861fd7874"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:35:27.366570 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.366327    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-model-cache" (OuterVolumeSpecName: "model-cache") pod "b525dac2-5ce8-412f-b581-e0f861fd7874" (UID: "b525dac2-5ce8-412f-b581-e0f861fd7874"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:35:27.368687 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.366996    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-tmp-dir\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:35:27.370350 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.369147    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b525dac2-5ce8-412f-b581-e0f861fd7874-kube-api-access-hps9h" (OuterVolumeSpecName: "kube-api-access-hps9h") pod "b525dac2-5ce8-412f-b581-e0f861fd7874" (UID: "b525dac2-5ce8-412f-b581-e0f861fd7874"). InnerVolumeSpecName "kube-api-access-hps9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:35:27.373580 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.373414    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-dshm" (OuterVolumeSpecName: "dshm") pod "b525dac2-5ce8-412f-b581-e0f861fd7874" (UID: "b525dac2-5ce8-412f-b581-e0f861fd7874"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:35:27.373943 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.373920    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b525dac2-5ce8-412f-b581-e0f861fd7874-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b525dac2-5ce8-412f-b581-e0f861fd7874" (UID: "b525dac2-5ce8-412f-b581-e0f861fd7874"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:35:27.468133 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.468110    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b525dac2-5ce8-412f-b581-e0f861fd7874-tls-certs\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:35:27.468133 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.468134    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-dshm\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:35:27.468133 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.468143    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hps9h\" (UniqueName: \"kubernetes.io/projected/b525dac2-5ce8-412f-b581-e0f861fd7874-kube-api-access-hps9h\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:35:27.468322 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.468152    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-home\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:35:27.468322 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.468162    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b525dac2-5ce8-412f-b581-e0f861fd7874-model-cache\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:35:27.472937 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.472910    2576 generic.go:356] "Generic (PLEG): container finished" podID="b525dac2-5ce8-412f-b581-e0f861fd7874" containerID="2dfb9277448bcf7eb9501d11fb03e02238a7d6ee77a0932d7df7d5789f9423cf" exitCode=0
Jul 25 01:35:27.473029 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.472972    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"
Jul 25 01:35:27.473029 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.473010    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw" event={"ID":"b525dac2-5ce8-412f-b581-e0f861fd7874","Type":"ContainerDied","Data":"2dfb9277448bcf7eb9501d11fb03e02238a7d6ee77a0932d7df7d5789f9423cf"}
Jul 25 01:35:27.473102 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.473049    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw" event={"ID":"b525dac2-5ce8-412f-b581-e0f861fd7874","Type":"ContainerDied","Data":"4ca8f8f90d2585019295af818f39f37adf890e3073e284f27a071b88028edb4a"}
Jul 25 01:35:27.473102 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.473069    2576 scope.go:117] "RemoveContainer" containerID="2dfb9277448bcf7eb9501d11fb03e02238a7d6ee77a0932d7df7d5789f9423cf"
Jul 25 01:35:27.481316 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.481297    2576 scope.go:117] "RemoveContainer" containerID="2dfb9277448bcf7eb9501d11fb03e02238a7d6ee77a0932d7df7d5789f9423cf"
Jul 25 01:35:27.481552 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:35:27.481534    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dfb9277448bcf7eb9501d11fb03e02238a7d6ee77a0932d7df7d5789f9423cf\": container with ID starting with 2dfb9277448bcf7eb9501d11fb03e02238a7d6ee77a0932d7df7d5789f9423cf not found: ID does not exist" containerID="2dfb9277448bcf7eb9501d11fb03e02238a7d6ee77a0932d7df7d5789f9423cf"
Jul 25 01:35:27.481613 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.481563    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dfb9277448bcf7eb9501d11fb03e02238a7d6ee77a0932d7df7d5789f9423cf"} err="failed to get container status \"2dfb9277448bcf7eb9501d11fb03e02238a7d6ee77a0932d7df7d5789f9423cf\": rpc error: code = NotFound desc = could not find container \"2dfb9277448bcf7eb9501d11fb03e02238a7d6ee77a0932d7df7d5789f9423cf\": container with ID starting with 2dfb9277448bcf7eb9501d11fb03e02238a7d6ee77a0932d7df7d5789f9423cf not found: ID does not exist"
Jul 25 01:35:27.491505 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.491485    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"]
Jul 25 01:35:27.495683 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:27.495646    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-ca5a4e04-kserve-5948db9799-j56sw"]
Jul 25 01:35:28.153776 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.153738    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn" podUID="5cee828a-7f49-4298-91a9-f9daced8ba64" containerName="main" containerID="cri-o://56159f0cfc5f34c60f6d877fdb1655b21559ab440049d9774c48ea433119eac5" gracePeriod=45
Jul 25 01:35:28.393929 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.393900    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:35:28.475085 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.475018    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cee828a-7f49-4298-91a9-f9daced8ba64-tls-certs\") pod \"5cee828a-7f49-4298-91a9-f9daced8ba64\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") "
Jul 25 01:35:28.475085 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.475058    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6rsr\" (UniqueName: \"kubernetes.io/projected/5cee828a-7f49-4298-91a9-f9daced8ba64-kube-api-access-f6rsr\") pod \"5cee828a-7f49-4298-91a9-f9daced8ba64\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") "
Jul 25 01:35:28.475262 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.475113    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-home\") pod \"5cee828a-7f49-4298-91a9-f9daced8ba64\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") "
Jul 25 01:35:28.475262 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.475136    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-dshm\") pod \"5cee828a-7f49-4298-91a9-f9daced8ba64\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") "
Jul 25 01:35:28.475262 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.475164    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-tmp-dir\") pod \"5cee828a-7f49-4298-91a9-f9daced8ba64\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") "
Jul 25 01:35:28.475262 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.475198    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-model-cache\") pod \"5cee828a-7f49-4298-91a9-f9daced8ba64\" (UID: \"5cee828a-7f49-4298-91a9-f9daced8ba64\") "
Jul 25 01:35:28.475468 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.475374    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-home" (OuterVolumeSpecName: "home") pod "5cee828a-7f49-4298-91a9-f9daced8ba64" (UID: "5cee828a-7f49-4298-91a9-f9daced8ba64"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:35:28.475468 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.475461    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-home\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:35:28.475578 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.475521    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5cee828a-7f49-4298-91a9-f9daced8ba64" (UID: "5cee828a-7f49-4298-91a9-f9daced8ba64"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:35:28.475578 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.475545    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-model-cache" (OuterVolumeSpecName: "model-cache") pod "5cee828a-7f49-4298-91a9-f9daced8ba64" (UID: "5cee828a-7f49-4298-91a9-f9daced8ba64"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:35:28.477312 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.477286    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cee828a-7f49-4298-91a9-f9daced8ba64-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5cee828a-7f49-4298-91a9-f9daced8ba64" (UID: "5cee828a-7f49-4298-91a9-f9daced8ba64"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:35:28.477312 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.477305    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-dshm" (OuterVolumeSpecName: "dshm") pod "5cee828a-7f49-4298-91a9-f9daced8ba64" (UID: "5cee828a-7f49-4298-91a9-f9daced8ba64"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:35:28.477459 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.477408    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cee828a-7f49-4298-91a9-f9daced8ba64-kube-api-access-f6rsr" (OuterVolumeSpecName: "kube-api-access-f6rsr") pod "5cee828a-7f49-4298-91a9-f9daced8ba64" (UID: "5cee828a-7f49-4298-91a9-f9daced8ba64"). InnerVolumeSpecName "kube-api-access-f6rsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:35:28.479955 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.479929    2576 generic.go:356] "Generic (PLEG): container finished" podID="5cee828a-7f49-4298-91a9-f9daced8ba64" containerID="56159f0cfc5f34c60f6d877fdb1655b21559ab440049d9774c48ea433119eac5" exitCode=0
Jul 25 01:35:28.480067 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.479977    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"
Jul 25 01:35:28.480067 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.480010    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn" event={"ID":"5cee828a-7f49-4298-91a9-f9daced8ba64","Type":"ContainerDied","Data":"56159f0cfc5f34c60f6d877fdb1655b21559ab440049d9774c48ea433119eac5"}
Jul 25 01:35:28.480067 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.480046    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn" event={"ID":"5cee828a-7f49-4298-91a9-f9daced8ba64","Type":"ContainerDied","Data":"ad6ba8609b7e38bd14f41a8172e09ce940c9c618c720325e1bc3d0b897bd6d70"}
Jul 25 01:35:28.480067 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.480063    2576 scope.go:117] "RemoveContainer" containerID="56159f0cfc5f34c60f6d877fdb1655b21559ab440049d9774c48ea433119eac5"
Jul 25 01:35:28.494082 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.494068    2576 scope.go:117] "RemoveContainer" containerID="56159f0cfc5f34c60f6d877fdb1655b21559ab440049d9774c48ea433119eac5"
Jul 25 01:35:28.494332 ip-10-0-134-238 kubenswrapper[2576]: E0725 01:35:28.494313    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56159f0cfc5f34c60f6d877fdb1655b21559ab440049d9774c48ea433119eac5\": container with ID starting with 56159f0cfc5f34c60f6d877fdb1655b21559ab440049d9774c48ea433119eac5 not found: ID does not exist" containerID="56159f0cfc5f34c60f6d877fdb1655b21559ab440049d9774c48ea433119eac5"
Jul 25 01:35:28.494405 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.494345    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56159f0cfc5f34c60f6d877fdb1655b21559ab440049d9774c48ea433119eac5"} err="failed to get container status \"56159f0cfc5f34c60f6d877fdb1655b21559ab440049d9774c48ea433119eac5\": rpc error: code = NotFound desc = could not find container \"56159f0cfc5f34c60f6d877fdb1655b21559ab440049d9774c48ea433119eac5\": container with ID starting with 56159f0cfc5f34c60f6d877fdb1655b21559ab440049d9774c48ea433119eac5 not found: ID does not exist"
Jul 25 01:35:28.503415 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.503394    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"]
Jul 25 01:35:28.508987 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.508968    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-ca5a4e04-kserve-d7b587759-f6qhn"]
Jul 25 01:35:28.576510 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.576486    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cee828a-7f49-4298-91a9-f9daced8ba64-tls-certs\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:35:28.576510 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.576509    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6rsr\" (UniqueName: \"kubernetes.io/projected/5cee828a-7f49-4298-91a9-f9daced8ba64-kube-api-access-f6rsr\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:35:28.576680 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.576520    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-dshm\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:35:28.576680 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.576529    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-tmp-dir\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:35:28.576680 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.576538    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cee828a-7f49-4298-91a9-f9daced8ba64-model-cache\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 01:35:28.997185 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.997142    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cee828a-7f49-4298-91a9-f9daced8ba64" path="/var/lib/kubelet/pods/5cee828a-7f49-4298-91a9-f9daced8ba64/volumes"
Jul 25 01:35:28.997614 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:28.997593    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b525dac2-5ce8-412f-b581-e0f861fd7874" path="/var/lib/kubelet/pods/b525dac2-5ce8-412f-b581-e0f861fd7874/volumes"
Jul 25 01:35:29.927361 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:29.927332    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:35:36.624767 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:36.624732    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:35:49.035610 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:49.035581    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:35:49.037781 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:49.037760    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:35:49.043757 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:49.043736    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:35:49.045175 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:49.045157    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:35:54.220441 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:35:54.220406    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:36:03.425468 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:36:03.425433    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:36:12.726929 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:36:12.726849    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:36:21.726683 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:36:21.726636    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:36:32.425957 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:36:32.425920    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:36:41.621480 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:36:41.621442    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:36:54.733730 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:36:54.733690    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:37:04.323368 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:37:04.323331    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:37:13.734529 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:37:13.734489    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:37:23.427006 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:37:23.426967    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:37:35.928564 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:37:35.928525    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:37:44.828912 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:37:44.828838    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:37:55.631141 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:37:55.631102    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:38:06.423828 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:38:06.423795    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:38:15.520196 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:38:15.520160    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:38:23.523037 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:38:23.523002    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:38:36.924752 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:38:36.924703    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:38:49.429931 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:38:49.429895    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:39:04.528312 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:39:04.528279    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:40:49.060352 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:40:49.060322    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:40:49.065203 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:40:49.065178    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:40:49.071580 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:40:49.071556    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:40:49.073718 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:40:49.073701    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:44:17.520782 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:44:17.520749    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:44:29.626419 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:44:29.626384    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:45:49.088983 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:45:49.088956    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:45:49.091234 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:45:49.091213    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:45:49.097209 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:45:49.097182    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:45:49.099416 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:45:49.099401    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:47:44.520765 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:47:44.520728    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:47:56.423640 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:47:56.423602    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:50:49.114355 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:50:49.114315    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:50:49.117615 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:50:49.117586    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:50:49.122391 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:50:49.122371    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:50:49.125494 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:50:49.125477    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:53:09.526640 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:53:09.526605    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:53:21.231625 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:53:21.231593    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:55:49.142935 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:55:49.142833    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:55:49.146902 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:55:49.145316    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 01:55:49.150900 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:55:49.150878    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:55:49.153287 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:55:49.153271    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 01:58:34.122941 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:58:34.122907    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 01:59:59.630826 ip-10-0-134-238 kubenswrapper[2576]: I0725 01:59:59.630784    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:00:08.826495 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:00:08.826461    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:00:19.127641 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:00:19.127567    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:00:40.725268 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:00:40.725229    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:00:49.171419 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:00:49.171391    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 02:00:49.174778 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:00:49.174758    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 02:00:49.179546 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:00:49.179526    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 02:00:49.182805 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:00:49.182789    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 02:01:17.725341 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:17.725306    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:01:24.420021 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.419989    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"]
Jul 25 02:01:24.420398 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.420293    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b525dac2-5ce8-412f-b581-e0f861fd7874" containerName="main"
Jul 25 02:01:24.420398 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.420303    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b525dac2-5ce8-412f-b581-e0f861fd7874" containerName="main"
Jul 25 02:01:24.420398 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.420322    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cee828a-7f49-4298-91a9-f9daced8ba64" containerName="main"
Jul 25 02:01:24.420398 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.420327    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cee828a-7f49-4298-91a9-f9daced8ba64" containerName="main"
Jul 25 02:01:24.420398 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.420378    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cee828a-7f49-4298-91a9-f9daced8ba64" containerName="main"
Jul 25 02:01:24.420398 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.420385    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b525dac2-5ce8-412f-b581-e0f861fd7874" containerName="main"
Jul 25 02:01:24.423451 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.423431    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.426171 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.426137    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-dc5514f6\"/\"e2e-lifecycle-model-dc5514f6-kserve-self-signed-certs\""
Jul 25 02:01:24.426280 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.426182    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-dc5514f6\"/\"default-dockercfg-6nb45\""
Jul 25 02:01:24.426280 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.426186    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-dc5514f6\"/\"kube-root-ca.crt\""
Jul 25 02:01:24.426280 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.426269    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-dc5514f6\"/\"openshift-service-ca.crt\""
Jul 25 02:01:24.434065 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.434044    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"]
Jul 25 02:01:24.455782 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.455759    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef101070-1c8c-494a-84fb-c26457ce4e3d-tls-certs\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.455886 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.455797    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndwwx\" (UniqueName: \"kubernetes.io/projected/ef101070-1c8c-494a-84fb-c26457ce4e3d-kube-api-access-ndwwx\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.455886 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.455819    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-dshm\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.455886 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.455858    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-model-cache\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.456004 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.455903    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-tmp-dir\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.456004 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.455935    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-home\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.556501 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.556465    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-home\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.556709 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.556514    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef101070-1c8c-494a-84fb-c26457ce4e3d-tls-certs\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.556709 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.556557    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndwwx\" (UniqueName: \"kubernetes.io/projected/ef101070-1c8c-494a-84fb-c26457ce4e3d-kube-api-access-ndwwx\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.556709 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.556596    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-dshm\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.556709 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.556630    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-model-cache\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.556709 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.556699    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-tmp-dir\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.556983 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.556960    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-home\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.557057 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.557037    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-model-cache\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.557116 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.557054    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-tmp-dir\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.558839 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.558811    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-dshm\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.559118 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.559097    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef101070-1c8c-494a-84fb-c26457ce4e3d-tls-certs\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.563564 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.563543    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndwwx\" (UniqueName: \"kubernetes.io/projected/ef101070-1c8c-494a-84fb-c26457ce4e3d-kube-api-access-ndwwx\") pod \"e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") " pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.734884 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.734810    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:24.863771 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:24.863738    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"]
Jul 25 02:01:24.867161 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:01:24.867133    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef101070_1c8c_494a_84fb_c26457ce4e3d.slice/crio-e027ce967013e815c0212fb0d475f63d35ab6b2d378cd327cfebb3dd11ae76ec WatchSource:0}: Error finding container e027ce967013e815c0212fb0d475f63d35ab6b2d378cd327cfebb3dd11ae76ec: Status 404 returned error can't find the container with id e027ce967013e815c0212fb0d475f63d35ab6b2d378cd327cfebb3dd11ae76ec
Jul 25 02:01:25.205614 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:25.205577    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7" event={"ID":"ef101070-1c8c-494a-84fb-c26457ce4e3d","Type":"ContainerStarted","Data":"fbf031fb011e213db36838bbc8668772a59c9d2c0f1a0cc19375b1ec9ded7472"}
Jul 25 02:01:25.205807 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:25.205624    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7" event={"ID":"ef101070-1c8c-494a-84fb-c26457ce4e3d","Type":"ContainerStarted","Data":"e027ce967013e815c0212fb0d475f63d35ab6b2d378cd327cfebb3dd11ae76ec"}
Jul 25 02:01:25.221463 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:25.221420    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7" podStartSLOduration=1.221405184 podStartE2EDuration="1.221405184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 02:01:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:01:25.219683318 +0000 UTC m=+3036.706979993" watchObservedRunningTime="2026-07-25 02:01:25.221405184 +0000 UTC m=+3036.708701858"
Jul 25 02:01:29.331292 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:29.331258    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:01:34.735054 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:34.735015    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:34.735054 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:34.735064    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:34.747947 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:34.747922    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:35.264528 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:35.264501    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:01:41.328954 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:41.328913    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:01:59.727827 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:01:59.727792    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:02:09.929526 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:09.929490    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:02:14.011953 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:14.011919    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"]
Jul 25 02:02:29.030820 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.030763    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7" podUID="ef101070-1c8c-494a-84fb-c26457ce4e3d" containerName="main" containerID="cri-o://fbf031fb011e213db36838bbc8668772a59c9d2c0f1a0cc19375b1ec9ded7472" gracePeriod=45
Jul 25 02:02:29.277103 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.277080    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:02:29.372414 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.372335    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef101070-1c8c-494a-84fb-c26457ce4e3d-tls-certs\") pod \"ef101070-1c8c-494a-84fb-c26457ce4e3d\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") "
Jul 25 02:02:29.372414 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.372376    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndwwx\" (UniqueName: \"kubernetes.io/projected/ef101070-1c8c-494a-84fb-c26457ce4e3d-kube-api-access-ndwwx\") pod \"ef101070-1c8c-494a-84fb-c26457ce4e3d\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") "
Jul 25 02:02:29.372414 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.372393    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-model-cache\") pod \"ef101070-1c8c-494a-84fb-c26457ce4e3d\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") "
Jul 25 02:02:29.372709 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.372453    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-home\") pod \"ef101070-1c8c-494a-84fb-c26457ce4e3d\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") "
Jul 25 02:02:29.372709 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.372488    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-dshm\") pod \"ef101070-1c8c-494a-84fb-c26457ce4e3d\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") "
Jul 25 02:02:29.372709 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.372643    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-tmp-dir\") pod \"ef101070-1c8c-494a-84fb-c26457ce4e3d\" (UID: \"ef101070-1c8c-494a-84fb-c26457ce4e3d\") "
Jul 25 02:02:29.372858 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.372760    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-model-cache" (OuterVolumeSpecName: "model-cache") pod "ef101070-1c8c-494a-84fb-c26457ce4e3d" (UID: "ef101070-1c8c-494a-84fb-c26457ce4e3d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:02:29.372858 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.372771    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-home" (OuterVolumeSpecName: "home") pod "ef101070-1c8c-494a-84fb-c26457ce4e3d" (UID: "ef101070-1c8c-494a-84fb-c26457ce4e3d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:02:29.372949 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.372906    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ef101070-1c8c-494a-84fb-c26457ce4e3d" (UID: "ef101070-1c8c-494a-84fb-c26457ce4e3d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:02:29.373024 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.373005    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-tmp-dir\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:02:29.373071 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.373031    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-model-cache\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:02:29.373071 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.373043    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-home\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:02:29.374638 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.374611    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-dshm" (OuterVolumeSpecName: "dshm") pod "ef101070-1c8c-494a-84fb-c26457ce4e3d" (UID: "ef101070-1c8c-494a-84fb-c26457ce4e3d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:02:29.374778 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.374635    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef101070-1c8c-494a-84fb-c26457ce4e3d-kube-api-access-ndwwx" (OuterVolumeSpecName: "kube-api-access-ndwwx") pod "ef101070-1c8c-494a-84fb-c26457ce4e3d" (UID: "ef101070-1c8c-494a-84fb-c26457ce4e3d"). InnerVolumeSpecName "kube-api-access-ndwwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:02:29.374778 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.374637    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef101070-1c8c-494a-84fb-c26457ce4e3d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ef101070-1c8c-494a-84fb-c26457ce4e3d" (UID: "ef101070-1c8c-494a-84fb-c26457ce4e3d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 02:02:29.474337 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.474305    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef101070-1c8c-494a-84fb-c26457ce4e3d-tls-certs\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:02:29.474337 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.474331    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndwwx\" (UniqueName: \"kubernetes.io/projected/ef101070-1c8c-494a-84fb-c26457ce4e3d-kube-api-access-ndwwx\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:02:29.474337 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.474342    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ef101070-1c8c-494a-84fb-c26457ce4e3d-dshm\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:02:29.491061 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.491029    2576 generic.go:356] "Generic (PLEG): container finished" podID="ef101070-1c8c-494a-84fb-c26457ce4e3d" containerID="fbf031fb011e213db36838bbc8668772a59c9d2c0f1a0cc19375b1ec9ded7472" exitCode=0
Jul 25 02:02:29.491192 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.491076    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"
Jul 25 02:02:29.491192 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.491119    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7" event={"ID":"ef101070-1c8c-494a-84fb-c26457ce4e3d","Type":"ContainerDied","Data":"fbf031fb011e213db36838bbc8668772a59c9d2c0f1a0cc19375b1ec9ded7472"}
Jul 25 02:02:29.491192 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.491176    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7" event={"ID":"ef101070-1c8c-494a-84fb-c26457ce4e3d","Type":"ContainerDied","Data":"e027ce967013e815c0212fb0d475f63d35ab6b2d378cd327cfebb3dd11ae76ec"}
Jul 25 02:02:29.491354 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.491199    2576 scope.go:117] "RemoveContainer" containerID="fbf031fb011e213db36838bbc8668772a59c9d2c0f1a0cc19375b1ec9ded7472"
Jul 25 02:02:29.500402 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.500378    2576 scope.go:117] "RemoveContainer" containerID="fbf031fb011e213db36838bbc8668772a59c9d2c0f1a0cc19375b1ec9ded7472"
Jul 25 02:02:29.500680 ip-10-0-134-238 kubenswrapper[2576]: E0725 02:02:29.500635    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbf031fb011e213db36838bbc8668772a59c9d2c0f1a0cc19375b1ec9ded7472\": container with ID starting with fbf031fb011e213db36838bbc8668772a59c9d2c0f1a0cc19375b1ec9ded7472 not found: ID does not exist" containerID="fbf031fb011e213db36838bbc8668772a59c9d2c0f1a0cc19375b1ec9ded7472"
Jul 25 02:02:29.500742 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.500674    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf031fb011e213db36838bbc8668772a59c9d2c0f1a0cc19375b1ec9ded7472"} err="failed to get container status \"fbf031fb011e213db36838bbc8668772a59c9d2c0f1a0cc19375b1ec9ded7472\": rpc error: code = NotFound desc = could not find container \"fbf031fb011e213db36838bbc8668772a59c9d2c0f1a0cc19375b1ec9ded7472\": container with ID starting with fbf031fb011e213db36838bbc8668772a59c9d2c0f1a0cc19375b1ec9ded7472 not found: ID does not exist"
Jul 25 02:02:29.511343 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.511322    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"]
Jul 25 02:02:29.516567 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:29.516543    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-dc5514f6/e2e-lifecycle-model-dc5514f6-kserve-7c748ddddc-zcdd7"]
Jul 25 02:02:30.996194 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:02:30.996119    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef101070-1c8c-494a-84fb-c26457ce4e3d" path="/var/lib/kubelet/pods/ef101070-1c8c-494a-84fb-c26457ce4e3d/volumes"
Jul 25 02:03:01.029632 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:03:01.029597    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:03:12.737786 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:03:12.737699    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:04:01.129887 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:01.129854    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:04:10.358679 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.358636    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"]
Jul 25 02:04:10.359135 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.358963    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef101070-1c8c-494a-84fb-c26457ce4e3d" containerName="main"
Jul 25 02:04:10.359135 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.358974    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef101070-1c8c-494a-84fb-c26457ce4e3d" containerName="main"
Jul 25 02:04:10.359135 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.359032    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef101070-1c8c-494a-84fb-c26457ce4e3d" containerName="main"
Jul 25 02:04:10.361967 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.361950    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.366079 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.366056    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c13f56\"/\"openshift-service-ca.crt\""
Jul 25 02:04:10.366209 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.366106    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c13f56\"/\"kube-root-ca.crt\""
Jul 25 02:04:10.366209 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.366076    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c13f56\"/\"default-dockercfg-9pqpx\""
Jul 25 02:04:10.366335 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.366067    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c13f56\"/\"test-model-c13f56-kserve-self-signed-certs\""
Jul 25 02:04:10.384230 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.384208    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"]
Jul 25 02:04:10.398816 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.398789    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-model-cache\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.398920 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.398825    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-dshm\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.398920 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.398846    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc906577-ba91-4d16-ba4b-071043ddbe43-tls-certs\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.398920 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.398870    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r89vs\" (UniqueName: \"kubernetes.io/projected/dc906577-ba91-4d16-ba4b-071043ddbe43-kube-api-access-r89vs\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.399078 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.398945    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-tmp-dir\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.399078 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.399023    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-home\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.500181 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.500143    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-home\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.500367 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.500202    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-model-cache\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.500367 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.500232    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-dshm\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.500367 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.500260    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc906577-ba91-4d16-ba4b-071043ddbe43-tls-certs\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.500367 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.500294    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r89vs\" (UniqueName: \"kubernetes.io/projected/dc906577-ba91-4d16-ba4b-071043ddbe43-kube-api-access-r89vs\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.500367 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.500320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-tmp-dir\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.500634 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.500574    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-home\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.500634 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.500590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-model-cache\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.500776 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.500728    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-tmp-dir\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.502592 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.502565    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-dshm\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.502754 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.502737    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc906577-ba91-4d16-ba4b-071043ddbe43-tls-certs\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.508676 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.508642    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r89vs\" (UniqueName: \"kubernetes.io/projected/dc906577-ba91-4d16-ba4b-071043ddbe43-kube-api-access-r89vs\") pod \"test-model-c13f56-kserve-544cc89cd9-46c5v\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") " pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.670835 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.670805    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:10.810089 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.810057    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"]
Jul 25 02:04:10.815705 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:04:10.815677    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc906577_ba91_4d16_ba4b_071043ddbe43.slice/crio-ac5cef90498f6ba3eba58c7e86d88ad246d55673ef9080e13676e6480f98cc3e WatchSource:0}: Error finding container ac5cef90498f6ba3eba58c7e86d88ad246d55673ef9080e13676e6480f98cc3e: Status 404 returned error can't find the container with id ac5cef90498f6ba3eba58c7e86d88ad246d55673ef9080e13676e6480f98cc3e
Jul 25 02:04:10.941487 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.941410    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v" event={"ID":"dc906577-ba91-4d16-ba4b-071043ddbe43","Type":"ContainerStarted","Data":"267e47ed65741fce4708ac6c11ae94b12c51d643b6350c494da682d7ec48dcc1"}
Jul 25 02:04:10.941487 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.941450    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v" event={"ID":"dc906577-ba91-4d16-ba4b-071043ddbe43","Type":"ContainerStarted","Data":"ac5cef90498f6ba3eba58c7e86d88ad246d55673ef9080e13676e6480f98cc3e"}
Jul 25 02:04:10.960078 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:10.960033    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v" podStartSLOduration=0.960019916 podStartE2EDuration="960.019916ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 02:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:04:10.959169468 +0000 UTC m=+3202.446466142" watchObservedRunningTime="2026-07-25 02:04:10.960019916 +0000 UTC m=+3202.447316590"
Jul 25 02:04:14.122359 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:14.122328    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:04:20.671471 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:20.671418    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:20.671471 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:20.671476    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:20.683820 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:20.683794    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:20.998543 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:20.998469    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:04:25.030084 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:25.030051    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:04:50.629690 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.629595    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"]
Jul 25 02:04:50.633015 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.632999    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.635884 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.635857    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1a3abd\"/\"kube-root-ca.crt\""
Jul 25 02:04:50.636011 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.635887    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1a3abd\"/\"openshift-service-ca.crt\""
Jul 25 02:04:50.636011 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.635926    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1a3abd\"/\"default-dockercfg-kzjxs\""
Jul 25 02:04:50.636011 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.635973    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1a3abd\"/\"test-model-1a3abd-kserve-self-signed-certs\""
Jul 25 02:04:50.641970 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.641947    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"]
Jul 25 02:04:50.731479 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.731450    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-model-cache\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.731648 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.731539    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-home\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.731648 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.731587    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-dshm\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.731648 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.731609    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrtfl\" (UniqueName: \"kubernetes.io/projected/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-kube-api-access-xrtfl\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.731842 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.731702    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-tmp-dir\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.731842 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.731735    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-tls-certs\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.832396 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.832363    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-dshm\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.832560 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.832404    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrtfl\" (UniqueName: \"kubernetes.io/projected/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-kube-api-access-xrtfl\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.832560 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.832426    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-tmp-dir\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.832560 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.832444    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-tls-certs\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.832560 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.832477    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-model-cache\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.832560 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.832521    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-home\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.832908 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.832888    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-tmp-dir\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.832971 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.832929    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-model-cache\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.832971 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.832942    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-home\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.834543 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.834520    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-dshm\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.834817 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.834802    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-tls-certs\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.839455 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.839435    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrtfl\" (UniqueName: \"kubernetes.io/projected/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-kube-api-access-xrtfl\") pod \"test-model-1a3abd-kserve-68998b8d77-p74wj\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") " pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:50.942860 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:50.942835    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:04:51.069917 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:51.069886    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"]
Jul 25 02:04:51.072640 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:04:51.072611    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0de1d7a0_c2df_4c55_8f6e_dcff7fc1ca5c.slice/crio-7af746062348a56dbd56eb47e16dc7a815a53d4b57ab075774d10d1795187609 WatchSource:0}: Error finding container 7af746062348a56dbd56eb47e16dc7a815a53d4b57ab075774d10d1795187609: Status 404 returned error can't find the container with id 7af746062348a56dbd56eb47e16dc7a815a53d4b57ab075774d10d1795187609
Jul 25 02:04:51.110959 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:51.110931    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj" event={"ID":"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c","Type":"ContainerStarted","Data":"7af746062348a56dbd56eb47e16dc7a815a53d4b57ab075774d10d1795187609"}
Jul 25 02:04:52.116946 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:52.116911    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj" event={"ID":"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c","Type":"ContainerStarted","Data":"1f5a03dcd836b655171b5dec8f27f951d07b90ca7f719bcc9b5eb45920f808cf"}
Jul 25 02:04:52.132822 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:52.132771    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj" podStartSLOduration=2.13275231 podStartE2EDuration="2.13275231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 02:04:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:04:52.132648864 +0000 UTC m=+3243.619945539" watchObservedRunningTime="2026-07-25 02:04:52.13275231 +0000 UTC m=+3243.620048988"
Jul 25 02:04:54.819463 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:04:54.819427    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:05:00.943236 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:00.943198    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:05:00.943236 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:00.943243    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:05:00.955581 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:00.955559    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:05:01.164349 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:01.164311    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:05:09.131202 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:09.131146    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:05:33.162031 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:33.161995    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"]
Jul 25 02:05:34.211134 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:34.211105    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"]
Jul 25 02:05:37.730621 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:37.730583    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:05:48.181070 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.180981    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v" podUID="dc906577-ba91-4d16-ba4b-071043ddbe43" containerName="main" containerID="cri-o://267e47ed65741fce4708ac6c11ae94b12c51d643b6350c494da682d7ec48dcc1" gracePeriod=45
Jul 25 02:05:48.362864 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.362829    2576 generic.go:356] "Generic (PLEG): container finished" podID="dc906577-ba91-4d16-ba4b-071043ddbe43" containerID="267e47ed65741fce4708ac6c11ae94b12c51d643b6350c494da682d7ec48dcc1" exitCode=0
Jul 25 02:05:48.363004 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.362875    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v" event={"ID":"dc906577-ba91-4d16-ba4b-071043ddbe43","Type":"ContainerDied","Data":"267e47ed65741fce4708ac6c11ae94b12c51d643b6350c494da682d7ec48dcc1"}
Jul 25 02:05:48.413878 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.413856    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:05:48.602591 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.602508    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-model-cache\") pod \"dc906577-ba91-4d16-ba4b-071043ddbe43\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") "
Jul 25 02:05:48.602591 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.602540    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc906577-ba91-4d16-ba4b-071043ddbe43-tls-certs\") pod \"dc906577-ba91-4d16-ba4b-071043ddbe43\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") "
Jul 25 02:05:48.602844 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.602616    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r89vs\" (UniqueName: \"kubernetes.io/projected/dc906577-ba91-4d16-ba4b-071043ddbe43-kube-api-access-r89vs\") pod \"dc906577-ba91-4d16-ba4b-071043ddbe43\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") "
Jul 25 02:05:48.602844 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.602638    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-tmp-dir\") pod \"dc906577-ba91-4d16-ba4b-071043ddbe43\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") "
Jul 25 02:05:48.602844 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.602694    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-dshm\") pod \"dc906577-ba91-4d16-ba4b-071043ddbe43\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") "
Jul 25 02:05:48.602844 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.602719    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-home\") pod \"dc906577-ba91-4d16-ba4b-071043ddbe43\" (UID: \"dc906577-ba91-4d16-ba4b-071043ddbe43\") "
Jul 25 02:05:48.603155 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.602848    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-model-cache" (OuterVolumeSpecName: "model-cache") pod "dc906577-ba91-4d16-ba4b-071043ddbe43" (UID: "dc906577-ba91-4d16-ba4b-071043ddbe43"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:05:48.603155 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.602949    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dc906577-ba91-4d16-ba4b-071043ddbe43" (UID: "dc906577-ba91-4d16-ba4b-071043ddbe43"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:05:48.603155 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.603071    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-tmp-dir\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:05:48.603155 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.603088    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-model-cache\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:05:48.603155 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.603083    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-home" (OuterVolumeSpecName: "home") pod "dc906577-ba91-4d16-ba4b-071043ddbe43" (UID: "dc906577-ba91-4d16-ba4b-071043ddbe43"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:05:48.604973 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.604937    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-dshm" (OuterVolumeSpecName: "dshm") pod "dc906577-ba91-4d16-ba4b-071043ddbe43" (UID: "dc906577-ba91-4d16-ba4b-071043ddbe43"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:05:48.605084 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.604985    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc906577-ba91-4d16-ba4b-071043ddbe43-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dc906577-ba91-4d16-ba4b-071043ddbe43" (UID: "dc906577-ba91-4d16-ba4b-071043ddbe43"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 02:05:48.605084 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.605019    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc906577-ba91-4d16-ba4b-071043ddbe43-kube-api-access-r89vs" (OuterVolumeSpecName: "kube-api-access-r89vs") pod "dc906577-ba91-4d16-ba4b-071043ddbe43" (UID: "dc906577-ba91-4d16-ba4b-071043ddbe43"). InnerVolumeSpecName "kube-api-access-r89vs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:05:48.703536 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.703496    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r89vs\" (UniqueName: \"kubernetes.io/projected/dc906577-ba91-4d16-ba4b-071043ddbe43-kube-api-access-r89vs\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:05:48.703536 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.703531    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-dshm\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:05:48.703536 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.703544    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc906577-ba91-4d16-ba4b-071043ddbe43-home\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:05:48.703800 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:48.703555    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc906577-ba91-4d16-ba4b-071043ddbe43-tls-certs\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:05:49.201142 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.201031    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 02:05:49.213891 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.204583    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 02:05:49.213891 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.209088    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 02:05:49.213891 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.212856    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 02:05:49.230234 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.230205    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj" podUID="0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c" containerName="main" containerID="cri-o://1f5a03dcd836b655171b5dec8f27f951d07b90ca7f719bcc9b5eb45920f808cf" gracePeriod=45
Jul 25 02:05:49.368986 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.368957    2576 generic.go:356] "Generic (PLEG): container finished" podID="0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c" containerID="1f5a03dcd836b655171b5dec8f27f951d07b90ca7f719bcc9b5eb45920f808cf" exitCode=0
Jul 25 02:05:49.369155 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.369045    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj" event={"ID":"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c","Type":"ContainerDied","Data":"1f5a03dcd836b655171b5dec8f27f951d07b90ca7f719bcc9b5eb45920f808cf"}
Jul 25 02:05:49.370322 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.370297    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"
Jul 25 02:05:49.370322 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.370308    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v" event={"ID":"dc906577-ba91-4d16-ba4b-071043ddbe43","Type":"ContainerDied","Data":"ac5cef90498f6ba3eba58c7e86d88ad246d55673ef9080e13676e6480f98cc3e"}
Jul 25 02:05:49.370485 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.370344    2576 scope.go:117] "RemoveContainer" containerID="267e47ed65741fce4708ac6c11ae94b12c51d643b6350c494da682d7ec48dcc1"
Jul 25 02:05:49.384145 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.384121    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"]
Jul 25 02:05:49.389847 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.389823    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-c13f56/test-model-c13f56-kserve-544cc89cd9-46c5v"]
Jul 25 02:05:49.472843 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.472821    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:05:49.610288 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.610211    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-home\") pod \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") "
Jul 25 02:05:49.610288 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.610240    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrtfl\" (UniqueName: \"kubernetes.io/projected/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-kube-api-access-xrtfl\") pod \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") "
Jul 25 02:05:49.610288 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.610276    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-model-cache\") pod \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") "
Jul 25 02:05:49.610573 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.610296    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-dshm\") pod \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") "
Jul 25 02:05:49.610573 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.610339    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-tmp-dir\") pod \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") "
Jul 25 02:05:49.610573 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.610396    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-tls-certs\") pod \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\" (UID: \"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c\") "
Jul 25 02:05:49.610573 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.610432    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-home" (OuterVolumeSpecName: "home") pod "0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c" (UID: "0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:05:49.610573 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.610551    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-model-cache" (OuterVolumeSpecName: "model-cache") pod "0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c" (UID: "0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:05:49.610859 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.610628    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c" (UID: "0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:05:49.610859 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.610691    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-home\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:05:49.610859 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.610708    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-model-cache\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:05:49.612530 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.612496    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-dshm" (OuterVolumeSpecName: "dshm") pod "0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c" (UID: "0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:05:49.612694 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.612556    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c" (UID: "0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 02:05:49.612694 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.612568    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-kube-api-access-xrtfl" (OuterVolumeSpecName: "kube-api-access-xrtfl") pod "0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c" (UID: "0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c"). InnerVolumeSpecName "kube-api-access-xrtfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:05:49.712057 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.712025    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrtfl\" (UniqueName: \"kubernetes.io/projected/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-kube-api-access-xrtfl\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:05:49.712057 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.712052    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-dshm\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:05:49.712057 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.712062    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-tmp-dir\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:05:49.712288 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:49.712071    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c-tls-certs\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:05:50.376969 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:50.376943    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"
Jul 25 02:05:50.376969 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:50.376962    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj" event={"ID":"0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c","Type":"ContainerDied","Data":"7af746062348a56dbd56eb47e16dc7a815a53d4b57ab075774d10d1795187609"}
Jul 25 02:05:50.377459 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:50.377013    2576 scope.go:117] "RemoveContainer" containerID="1f5a03dcd836b655171b5dec8f27f951d07b90ca7f719bcc9b5eb45920f808cf"
Jul 25 02:05:50.396283 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:50.396256    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"]
Jul 25 02:05:50.401936 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:50.401907    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-1a3abd/test-model-1a3abd-kserve-68998b8d77-p74wj"]
Jul 25 02:05:50.996205 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:50.996171    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c" path="/var/lib/kubelet/pods/0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c/volumes"
Jul 25 02:05:50.996556 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:50.996543    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc906577-ba91-4d16-ba4b-071043ddbe43" path="/var/lib/kubelet/pods/dc906577-ba91-4d16-ba4b-071043ddbe43/volumes"
Jul 25 02:05:53.427300 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:05:53.427269    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:06:09.924835 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:09.924791    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:06:28.930391 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:28.930314    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:06:41.773042 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:41.773009    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q"]
Jul 25 02:06:41.773523 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:41.773324    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c" containerName="main"
Jul 25 02:06:41.773523 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:41.773334    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c" containerName="main"
Jul 25 02:06:41.773523 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:41.773346    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc906577-ba91-4d16-ba4b-071043ddbe43" containerName="main"
Jul 25 02:06:41.773523 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:41.773351    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc906577-ba91-4d16-ba4b-071043ddbe43" containerName="main"
Jul 25 02:06:41.773523 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:41.773401    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc906577-ba91-4d16-ba4b-071043ddbe43" containerName="main"
Jul 25 02:06:41.773523 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:41.773410    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0de1d7a0-c2df-4c55-8f6e-dcff7fc1ca5c" containerName="main"
Jul 25 02:06:41.776172 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:41.776156    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q"
Jul 25 02:06:41.778216 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:41.778195    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 02:06:41.778306 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:41.778195    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cjsrv\""
Jul 25 02:06:41.782198 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:41.782178    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q"]
Jul 25 02:06:41.905613 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:41.905583    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56de6abf-2ac2-4443-a406-c010be0d781f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q\" (UID: \"56de6abf-2ac2-4443-a406-c010be0d781f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q"
Jul 25 02:06:42.006252 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:42.006212    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56de6abf-2ac2-4443-a406-c010be0d781f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q\" (UID: \"56de6abf-2ac2-4443-a406-c010be0d781f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q"
Jul 25 02:06:42.006849 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:42.006830    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56de6abf-2ac2-4443-a406-c010be0d781f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q\" (UID: \"56de6abf-2ac2-4443-a406-c010be0d781f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q"
Jul 25 02:06:42.087479 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:42.087398    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q"
Jul 25 02:06:42.214121 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:42.214088    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q"]
Jul 25 02:06:42.217361 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:06:42.217335    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56de6abf_2ac2_4443_a406_c010be0d781f.slice/crio-e5bdb04005b933c7f13c032a8e663ece15539b4792a6bfd2475cffdd217b6b8d WatchSource:0}: Error finding container e5bdb04005b933c7f13c032a8e663ece15539b4792a6bfd2475cffdd217b6b8d: Status 404 returned error can't find the container with id e5bdb04005b933c7f13c032a8e663ece15539b4792a6bfd2475cffdd217b6b8d
Jul 25 02:06:42.219215 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:42.219199    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 02:06:42.602558 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:42.602525    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q" event={"ID":"56de6abf-2ac2-4443-a406-c010be0d781f","Type":"ContainerStarted","Data":"e5bdb04005b933c7f13c032a8e663ece15539b4792a6bfd2475cffdd217b6b8d"}
Jul 25 02:06:44.726569 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:44.726539    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:06:45.620095 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:45.620014    2576 generic.go:356] "Generic (PLEG): container finished" podID="56de6abf-2ac2-4443-a406-c010be0d781f" containerID="f1133913d8a56ab0aa5eb88dde847412133bff4f998272b9bdb1d2a18c5ebe6f" exitCode=0
Jul 25 02:06:45.620233 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:45.620096    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q" event={"ID":"56de6abf-2ac2-4443-a406-c010be0d781f","Type":"ContainerDied","Data":"f1133913d8a56ab0aa5eb88dde847412133bff4f998272b9bdb1d2a18c5ebe6f"}
Jul 25 02:06:46.747824 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:46.747801    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q"
Jul 25 02:06:46.844570 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:46.844543    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56de6abf-2ac2-4443-a406-c010be0d781f-maas-api-service-ca\") pod \"56de6abf-2ac2-4443-a406-c010be0d781f\" (UID: \"56de6abf-2ac2-4443-a406-c010be0d781f\") "
Jul 25 02:06:46.844911 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:46.844886    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56de6abf-2ac2-4443-a406-c010be0d781f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "56de6abf-2ac2-4443-a406-c010be0d781f" (UID: "56de6abf-2ac2-4443-a406-c010be0d781f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 02:06:46.945457 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:46.945432    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56de6abf-2ac2-4443-a406-c010be0d781f-maas-api-service-ca\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:06:47.629949 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:47.629915    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q"
Jul 25 02:06:47.630124 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:47.629922    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q" event={"ID":"56de6abf-2ac2-4443-a406-c010be0d781f","Type":"ContainerDied","Data":"e5bdb04005b933c7f13c032a8e663ece15539b4792a6bfd2475cffdd217b6b8d"}
Jul 25 02:06:47.630124 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:06:47.630067    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e5bdb04005b933c7f13c032a8e663ece15539b4792a6bfd2475cffdd217b6b8d"
Jul 25 02:07:00.523726 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:00.523682    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:07:01.851139 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:01.851097    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q"]
Jul 25 02:07:01.853517 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:01.853493    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-a9b5fd-k945q"]
Jul 25 02:07:02.995989 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:02.995958    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56de6abf-2ac2-4443-a406-c010be0d781f" path="/var/lib/kubelet/pods/56de6abf-2ac2-4443-a406-c010be0d781f/volumes"
Jul 25 02:07:08.532788 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:08.532750    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96"]
Jul 25 02:07:08.533167 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:08.533084    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56de6abf-2ac2-4443-a406-c010be0d781f" containerName="revoke-keys"
Jul 25 02:07:08.533167 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:08.533094    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="56de6abf-2ac2-4443-a406-c010be0d781f" containerName="revoke-keys"
Jul 25 02:07:08.533167 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:08.533153    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="56de6abf-2ac2-4443-a406-c010be0d781f" containerName="revoke-keys"
Jul 25 02:07:08.536213 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:08.536196    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96"
Jul 25 02:07:08.537929 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:08.537904    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 02:07:08.538050 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:08.537907    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cjsrv\""
Jul 25 02:07:08.543285 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:08.543043    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96"]
Jul 25 02:07:08.723092 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:08.723054    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05d35129-0513-44d3-87d6-31fa953f4351-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96\" (UID: \"05d35129-0513-44d3-87d6-31fa953f4351\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96"
Jul 25 02:07:08.824037 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:08.823952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05d35129-0513-44d3-87d6-31fa953f4351-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96\" (UID: \"05d35129-0513-44d3-87d6-31fa953f4351\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96"
Jul 25 02:07:08.824548 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:08.824530    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05d35129-0513-44d3-87d6-31fa953f4351-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96\" (UID: \"05d35129-0513-44d3-87d6-31fa953f4351\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96"
Jul 25 02:07:08.848177 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:08.848152    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96"
Jul 25 02:07:08.971416 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:08.971387    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96"]
Jul 25 02:07:08.974104 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:07:08.974081    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05d35129_0513_44d3_87d6_31fa953f4351.slice/crio-5ae0fb2976035938aefd985d6b1bd5a3ae8e179bf23f666c59fc2921cb29feee WatchSource:0}: Error finding container 5ae0fb2976035938aefd985d6b1bd5a3ae8e179bf23f666c59fc2921cb29feee: Status 404 returned error can't find the container with id 5ae0fb2976035938aefd985d6b1bd5a3ae8e179bf23f666c59fc2921cb29feee
Jul 25 02:07:09.728907 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:09.728875    2576 generic.go:356] "Generic (PLEG): container finished" podID="05d35129-0513-44d3-87d6-31fa953f4351" containerID="35182f1591a568d7286a05c83d8e777c29705ae19fb5dd60b7e75c89ed137833" exitCode=0
Jul 25 02:07:09.729259 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:09.728965    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96" event={"ID":"05d35129-0513-44d3-87d6-31fa953f4351","Type":"ContainerDied","Data":"35182f1591a568d7286a05c83d8e777c29705ae19fb5dd60b7e75c89ed137833"}
Jul 25 02:07:09.729259 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:09.729028    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96" event={"ID":"05d35129-0513-44d3-87d6-31fa953f4351","Type":"ContainerStarted","Data":"5ae0fb2976035938aefd985d6b1bd5a3ae8e179bf23f666c59fc2921cb29feee"}
Jul 25 02:07:10.862938 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:10.862917    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96"
Jul 25 02:07:11.040160 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:11.040090    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05d35129-0513-44d3-87d6-31fa953f4351-maas-api-service-ca\") pod \"05d35129-0513-44d3-87d6-31fa953f4351\" (UID: \"05d35129-0513-44d3-87d6-31fa953f4351\") "
Jul 25 02:07:11.040434 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:11.040412    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05d35129-0513-44d3-87d6-31fa953f4351-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "05d35129-0513-44d3-87d6-31fa953f4351" (UID: "05d35129-0513-44d3-87d6-31fa953f4351"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 02:07:11.140988 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:11.140958    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/05d35129-0513-44d3-87d6-31fa953f4351-maas-api-service-ca\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:07:11.739376 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:11.739336    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96"
Jul 25 02:07:11.739543 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:11.739371    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96" event={"ID":"05d35129-0513-44d3-87d6-31fa953f4351","Type":"ContainerDied","Data":"5ae0fb2976035938aefd985d6b1bd5a3ae8e179bf23f666c59fc2921cb29feee"}
Jul 25 02:07:11.739543 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:11.739416    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ae0fb2976035938aefd985d6b1bd5a3ae8e179bf23f666c59fc2921cb29feee"
Jul 25 02:07:28.419820 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:28.419784    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:07:28.614100 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:28.614065    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96"]
Jul 25 02:07:28.616115 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:28.616089    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6fd1ef-k9z96"]
Jul 25 02:07:28.996857 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:28.996820    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05d35129-0513-44d3-87d6-31fa953f4351" path="/var/lib/kubelet/pods/05d35129-0513-44d3-87d6-31fa953f4351/volumes"
Jul 25 02:07:47.045482 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:47.045396    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:07:57.923607 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:07:57.923571    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:08:09.329860 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.329826    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"]
Jul 25 02:08:09.330442 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.330279    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05d35129-0513-44d3-87d6-31fa953f4351" containerName="revoke-keys"
Jul 25 02:08:09.330442 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.330298    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="05d35129-0513-44d3-87d6-31fa953f4351" containerName="revoke-keys"
Jul 25 02:08:09.330442 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.330424    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="05d35129-0513-44d3-87d6-31fa953f4351" containerName="revoke-keys"
Jul 25 02:08:09.333374 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.333355    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.335225 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.335201    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7e06ec\"/\"kube-root-ca.crt\""
Jul 25 02:08:09.335790 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.335772    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7e06ec\"/\"openshift-service-ca.crt\""
Jul 25 02:08:09.335869 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.335813    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7e06ec\"/\"default-dockercfg-5dxrp\""
Jul 25 02:08:09.335933 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.335817    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7e06ec\"/\"auth-test-model-7e06ec-kserve-self-signed-certs\""
Jul 25 02:08:09.342146 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.342120    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"]
Jul 25 02:08:09.518431 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.518385    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-tmp-dir\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.518601 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.518452    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs9kg\" (UniqueName: \"kubernetes.io/projected/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-kube-api-access-gs9kg\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.518601 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.518525    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-home\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.518601 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.518554    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-model-cache\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.518601 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.518569    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-tls-certs\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.518601 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.518584    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-dshm\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.619923 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.619834    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-tmp-dir\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.619923 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.619888    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs9kg\" (UniqueName: \"kubernetes.io/projected/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-kube-api-access-gs9kg\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.619923 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.619930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-home\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.620201 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.619947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-model-cache\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.620201 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.619963    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-tls-certs\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.620201 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.619978    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-dshm\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.620336 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.620280    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-tmp-dir\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.620336 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.620289    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-home\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.620403 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.620368    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-model-cache\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.622159 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.622131    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-dshm\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.622473 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.622456    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-tls-certs\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.627448 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.627430    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs9kg\" (UniqueName: \"kubernetes.io/projected/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-kube-api-access-gs9kg\") pod \"auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") " pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.643017 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.642995    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:09.771169 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.771142    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"]
Jul 25 02:08:09.773807 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:08:09.773777    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1bdc8be5_a2e1_47bd_9315_d834fe65f63a.slice/crio-9ab43a1d942f8d94d59bc8b8f851b45ddd2f77467ea00775e8890b55c3639657 WatchSource:0}: Error finding container 9ab43a1d942f8d94d59bc8b8f851b45ddd2f77467ea00775e8890b55c3639657: Status 404 returned error can't find the container with id 9ab43a1d942f8d94d59bc8b8f851b45ddd2f77467ea00775e8890b55c3639657
Jul 25 02:08:09.990646 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.990615    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9" event={"ID":"1bdc8be5-a2e1-47bd-9315-d834fe65f63a","Type":"ContainerStarted","Data":"a41a1534b35f4ffa257d19ca4e0c060c018079ce9d13e6202164b818de3c1160"}
Jul 25 02:08:09.990646 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:09.990667    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9" event={"ID":"1bdc8be5-a2e1-47bd-9315-d834fe65f63a","Type":"ContainerStarted","Data":"9ab43a1d942f8d94d59bc8b8f851b45ddd2f77467ea00775e8890b55c3639657"}
Jul 25 02:08:10.006580 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:10.006537    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9" podStartSLOduration=1.006522646 podStartE2EDuration="1.006522646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 02:08:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:08:10.005697532 +0000 UTC m=+3441.492994206" watchObservedRunningTime="2026-07-25 02:08:10.006522646 +0000 UTC m=+3441.493819344"
Jul 25 02:08:12.726094 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:12.726062    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:08:19.643905 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:19.643865    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:19.644393 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:19.643920    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:19.656581 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:19.656558    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:20.042242 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:20.042205    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:08:28.829265 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.829234    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"]
Jul 25 02:08:28.832902 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.832882    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.835492 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.835465    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-575aa7\"/\"openshift-service-ca.crt\""
Jul 25 02:08:28.835604 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.835535    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-575aa7\"/\"auth-test-model-575aa7-kserve-self-signed-certs\""
Jul 25 02:08:28.835604 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.835551    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-575aa7\"/\"default-dockercfg-tmdb9\""
Jul 25 02:08:28.835604 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.835595    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-575aa7\"/\"kube-root-ca.crt\""
Jul 25 02:08:28.840083 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.840059    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"]
Jul 25 02:08:28.869467 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.869444    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-dshm\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.869590 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.869479    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnzf6\" (UniqueName: \"kubernetes.io/projected/88cd568b-edaf-45c3-aa81-e98a6da94384-kube-api-access-rnzf6\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.869590 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.869499    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-tmp-dir\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.869590 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.869526    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-home\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.869590 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.869569    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-model-cache\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.869771 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.869604    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88cd568b-edaf-45c3-aa81-e98a6da94384-tls-certs\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.970956 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.970930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-dshm\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.971109 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.970970    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnzf6\" (UniqueName: \"kubernetes.io/projected/88cd568b-edaf-45c3-aa81-e98a6da94384-kube-api-access-rnzf6\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.971109 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.970994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-tmp-dir\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.971109 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.971028    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-home\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.971109 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.971052    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-model-cache\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.971109 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.971077    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88cd568b-edaf-45c3-aa81-e98a6da94384-tls-certs\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.971442 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.971418    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-home\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.971510 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.971427    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-model-cache\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.971510 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.971460    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-tmp-dir\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.973142 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.973124    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-dshm\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.973399 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.973382    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88cd568b-edaf-45c3-aa81-e98a6da94384-tls-certs\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:28.978197 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:28.978177    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnzf6\" (UniqueName: \"kubernetes.io/projected/88cd568b-edaf-45c3-aa81-e98a6da94384-kube-api-access-rnzf6\") pod \"auth-test-model-575aa7-kserve-75889cc9ff-zqzgn\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") " pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:29.144015 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:29.143936    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:29.271861 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:29.271838    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"]
Jul 25 02:08:29.274315 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:08:29.274288    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88cd568b_edaf_45c3_aa81_e98a6da94384.slice/crio-b7bb21b185e9c246d0299079a80e00361d80536cd9a264eadadba1cd1e6c70af WatchSource:0}: Error finding container b7bb21b185e9c246d0299079a80e00361d80536cd9a264eadadba1cd1e6c70af: Status 404 returned error can't find the container with id b7bb21b185e9c246d0299079a80e00361d80536cd9a264eadadba1cd1e6c70af
Jul 25 02:08:30.074539 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:30.074502    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn" event={"ID":"88cd568b-edaf-45c3-aa81-e98a6da94384","Type":"ContainerStarted","Data":"b905793a4f5793bdd6021d5d35e9e10a62d8e459982d9ed48ca084f57893c275"}
Jul 25 02:08:30.074539 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:30.074543    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn" event={"ID":"88cd568b-edaf-45c3-aa81-e98a6da94384","Type":"ContainerStarted","Data":"b7bb21b185e9c246d0299079a80e00361d80536cd9a264eadadba1cd1e6c70af"}
Jul 25 02:08:30.089695 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:30.089635    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn" podStartSLOduration=2.089620757 podStartE2EDuration="2.089620757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 02:08:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:08:30.088844427 +0000 UTC m=+3461.576141102" watchObservedRunningTime="2026-07-25 02:08:30.089620757 +0000 UTC m=+3461.576917432"
Jul 25 02:08:32.525305 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:32.525263    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:08:39.144440 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:39.144409    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:39.144955 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:39.144460    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:39.156910 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:39.156884    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:40.132676 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:40.132629    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:08:57.325401 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:08:57.325366    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:09:18.119826 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:09:18.119745    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:09:35.826777 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:09:35.826740    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:10:03.524753 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:10:03.524717    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:10:21.332682 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:10:21.331927    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:10:41.728873 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:10:41.728787    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:10:49.228981 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:10:49.228878    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 02:10:49.235783 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:10:49.232925    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 02:10:49.237692 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:10:49.237673    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 02:10:49.241098 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:10:49.241082    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 02:10:59.332687 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:10:59.332640    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:11:22.926051 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:11:22.926019    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:11:40.822783 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:11:40.822749    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:12:03.223989 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:03.223955    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:12:18.730708 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:18.730623    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:12:42.136560 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:42.136525    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:12:46.387980 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:46.387948    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"]
Jul 25 02:12:47.222170 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:47.222133    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"]
Jul 25 02:12:49.348360 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:49.348330    2576 scope.go:117] "RemoveContainer" containerID="f1133913d8a56ab0aa5eb88dde847412133bff4f998272b9bdb1d2a18c5ebe6f"
Jul 25 02:12:53.226553 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.226518    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"]
Jul 25 02:12:53.231008 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.230986    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.233288 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.233266    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7e06ec\"/\"sub-test-model-7e06ec-kserve-self-signed-certs\""
Jul 25 02:12:53.238950 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.238927    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"]
Jul 25 02:12:53.314822 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.314792    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-home\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.314938 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.314834    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/859fce15-eb9c-4655-9d5b-ab2b80434602-tls-certs\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.314938 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.314853    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnggf\" (UniqueName: \"kubernetes.io/projected/859fce15-eb9c-4655-9d5b-ab2b80434602-kube-api-access-dnggf\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.314938 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.314924    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-dshm\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.315115 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.314965    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-tmp-dir\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.315115 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.314995    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-model-cache\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.416143 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.416109    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-home\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.416266 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.416155    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/859fce15-eb9c-4655-9d5b-ab2b80434602-tls-certs\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.416266 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.416181    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnggf\" (UniqueName: \"kubernetes.io/projected/859fce15-eb9c-4655-9d5b-ab2b80434602-kube-api-access-dnggf\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.416266 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.416221    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-dshm\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.416436 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.416362    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-tmp-dir\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.416436 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.416415    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-model-cache\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.416544 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.416507    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-home\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.416759 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.416731    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-tmp-dir\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.416877 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.416739    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-model-cache\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.418493 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.418473    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-dshm\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.418596 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.418580    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/859fce15-eb9c-4655-9d5b-ab2b80434602-tls-certs\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.423588 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.423570    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnggf\" (UniqueName: \"kubernetes.io/projected/859fce15-eb9c-4655-9d5b-ab2b80434602-kube-api-access-dnggf\") pod \"sub-test-model-7e06ec-kserve-64dbc668c-vg7dr\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") " pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.541829 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.541770    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:12:53.668033 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:53.668001    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"]
Jul 25 02:12:53.671419 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:12:53.671392    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod859fce15_eb9c_4655_9d5b_ab2b80434602.slice/crio-10da56d98f5eeb9b4424c834c45bc5b0cdf53696246477531d77da56290e1cde WatchSource:0}: Error finding container 10da56d98f5eeb9b4424c834c45bc5b0cdf53696246477531d77da56290e1cde: Status 404 returned error can't find the container with id 10da56d98f5eeb9b4424c834c45bc5b0cdf53696246477531d77da56290e1cde
Jul 25 02:12:54.249282 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:54.249246    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr" event={"ID":"859fce15-eb9c-4655-9d5b-ab2b80434602","Type":"ContainerStarted","Data":"32cf32c7aef979af99f3f1121ce68f668e299c50447d63e40a9931c3c110fa7c"}
Jul 25 02:12:54.249710 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:54.249293    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr" event={"ID":"859fce15-eb9c-4655-9d5b-ab2b80434602","Type":"ContainerStarted","Data":"10da56d98f5eeb9b4424c834c45bc5b0cdf53696246477531d77da56290e1cde"}
Jul 25 02:12:54.265091 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:54.265037    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr" podStartSLOduration=1.265018585 podStartE2EDuration="1.265018585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 02:12:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:12:54.264296426 +0000 UTC m=+3725.751593100" watchObservedRunningTime="2026-07-25 02:12:54.265018585 +0000 UTC m=+3725.752315264"
Jul 25 02:12:58.222677 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:12:58.222623    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:13:01.406722 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.406680    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9" podUID="1bdc8be5-a2e1-47bd-9315-d834fe65f63a" containerName="main" containerID="cri-o://a41a1534b35f4ffa257d19ca4e0c060c018079ce9d13e6202164b818de3c1160" gracePeriod=45
Jul 25 02:13:01.652331 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.652309    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:13:01.687039 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.686976    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-tmp-dir\") pod \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") "
Jul 25 02:13:01.687039 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.687007    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-tls-certs\") pod \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") "
Jul 25 02:13:01.687219 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.687046    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-model-cache\") pod \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") "
Jul 25 02:13:01.687219 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.687076    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-home\") pod \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") "
Jul 25 02:13:01.687219 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.687117    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-dshm\") pod \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") "
Jul 25 02:13:01.687219 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.687139    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gs9kg\" (UniqueName: \"kubernetes.io/projected/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-kube-api-access-gs9kg\") pod \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\" (UID: \"1bdc8be5-a2e1-47bd-9315-d834fe65f63a\") "
Jul 25 02:13:01.687424 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.687270    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1bdc8be5-a2e1-47bd-9315-d834fe65f63a" (UID: "1bdc8be5-a2e1-47bd-9315-d834fe65f63a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:13:01.687424 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.687380    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-model-cache" (OuterVolumeSpecName: "model-cache") pod "1bdc8be5-a2e1-47bd-9315-d834fe65f63a" (UID: "1bdc8be5-a2e1-47bd-9315-d834fe65f63a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:13:01.687424 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.687390    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-home" (OuterVolumeSpecName: "home") pod "1bdc8be5-a2e1-47bd-9315-d834fe65f63a" (UID: "1bdc8be5-a2e1-47bd-9315-d834fe65f63a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:13:01.687586 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.687474    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-model-cache\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:13:01.687586 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.687491    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-home\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:13:01.687586 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.687503    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-tmp-dir\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:13:01.689352 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.689331    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-dshm" (OuterVolumeSpecName: "dshm") pod "1bdc8be5-a2e1-47bd-9315-d834fe65f63a" (UID: "1bdc8be5-a2e1-47bd-9315-d834fe65f63a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:13:01.689490 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.689471    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-kube-api-access-gs9kg" (OuterVolumeSpecName: "kube-api-access-gs9kg") pod "1bdc8be5-a2e1-47bd-9315-d834fe65f63a" (UID: "1bdc8be5-a2e1-47bd-9315-d834fe65f63a"). InnerVolumeSpecName "kube-api-access-gs9kg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:13:01.689565 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.689549    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1bdc8be5-a2e1-47bd-9315-d834fe65f63a" (UID: "1bdc8be5-a2e1-47bd-9315-d834fe65f63a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 02:13:01.788091 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.788065    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gs9kg\" (UniqueName: \"kubernetes.io/projected/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-kube-api-access-gs9kg\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:13:01.788091 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.788088    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-tls-certs\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:13:01.788246 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:01.788098    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bdc8be5-a2e1-47bd-9315-d834fe65f63a-dshm\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:13:02.243175 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.243112    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn" podUID="88cd568b-edaf-45c3-aa81-e98a6da94384" containerName="main" containerID="cri-o://b905793a4f5793bdd6021d5d35e9e10a62d8e459982d9ed48ca084f57893c275" gracePeriod=45
Jul 25 02:13:02.293017 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.292987    2576 generic.go:356] "Generic (PLEG): container finished" podID="1bdc8be5-a2e1-47bd-9315-d834fe65f63a" containerID="a41a1534b35f4ffa257d19ca4e0c060c018079ce9d13e6202164b818de3c1160" exitCode=0
Jul 25 02:13:02.293146 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.293027    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"
Jul 25 02:13:02.293146 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.293052    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9" event={"ID":"1bdc8be5-a2e1-47bd-9315-d834fe65f63a","Type":"ContainerDied","Data":"a41a1534b35f4ffa257d19ca4e0c060c018079ce9d13e6202164b818de3c1160"}
Jul 25 02:13:02.293146 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.293086    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9" event={"ID":"1bdc8be5-a2e1-47bd-9315-d834fe65f63a","Type":"ContainerDied","Data":"9ab43a1d942f8d94d59bc8b8f851b45ddd2f77467ea00775e8890b55c3639657"}
Jul 25 02:13:02.293146 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.293103    2576 scope.go:117] "RemoveContainer" containerID="a41a1534b35f4ffa257d19ca4e0c060c018079ce9d13e6202164b818de3c1160"
Jul 25 02:13:02.314682 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.314638    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"]
Jul 25 02:13:02.316462 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.316440    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/auth-test-model-7e06ec-kserve-86dc7b7f5f-q9jl9"]
Jul 25 02:13:02.353990 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.353968    2576 scope.go:117] "RemoveContainer" containerID="a41a1534b35f4ffa257d19ca4e0c060c018079ce9d13e6202164b818de3c1160"
Jul 25 02:13:02.354249 ip-10-0-134-238 kubenswrapper[2576]: E0725 02:13:02.354230    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a41a1534b35f4ffa257d19ca4e0c060c018079ce9d13e6202164b818de3c1160\": container with ID starting with a41a1534b35f4ffa257d19ca4e0c060c018079ce9d13e6202164b818de3c1160 not found: ID does not exist" containerID="a41a1534b35f4ffa257d19ca4e0c060c018079ce9d13e6202164b818de3c1160"
Jul 25 02:13:02.354306 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.354259    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a41a1534b35f4ffa257d19ca4e0c060c018079ce9d13e6202164b818de3c1160"} err="failed to get container status \"a41a1534b35f4ffa257d19ca4e0c060c018079ce9d13e6202164b818de3c1160\": rpc error: code = NotFound desc = could not find container \"a41a1534b35f4ffa257d19ca4e0c060c018079ce9d13e6202164b818de3c1160\": container with ID starting with a41a1534b35f4ffa257d19ca4e0c060c018079ce9d13e6202164b818de3c1160 not found: ID does not exist"
Jul 25 02:13:02.497618 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.497560    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:13:02.594835 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.594805    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-tmp-dir\") pod \"88cd568b-edaf-45c3-aa81-e98a6da94384\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") "
Jul 25 02:13:02.594997 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.594849    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-dshm\") pod \"88cd568b-edaf-45c3-aa81-e98a6da94384\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") "
Jul 25 02:13:02.594997 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.594866    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-home\") pod \"88cd568b-edaf-45c3-aa81-e98a6da94384\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") "
Jul 25 02:13:02.594997 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.594896    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnzf6\" (UniqueName: \"kubernetes.io/projected/88cd568b-edaf-45c3-aa81-e98a6da94384-kube-api-access-rnzf6\") pod \"88cd568b-edaf-45c3-aa81-e98a6da94384\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") "
Jul 25 02:13:02.595143 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.595003    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88cd568b-edaf-45c3-aa81-e98a6da94384-tls-certs\") pod \"88cd568b-edaf-45c3-aa81-e98a6da94384\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") "
Jul 25 02:13:02.595143 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.595061    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-model-cache\") pod \"88cd568b-edaf-45c3-aa81-e98a6da94384\" (UID: \"88cd568b-edaf-45c3-aa81-e98a6da94384\") "
Jul 25 02:13:02.595143 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.595075    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "88cd568b-edaf-45c3-aa81-e98a6da94384" (UID: "88cd568b-edaf-45c3-aa81-e98a6da94384"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:13:02.595143 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.595128    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-home" (OuterVolumeSpecName: "home") pod "88cd568b-edaf-45c3-aa81-e98a6da94384" (UID: "88cd568b-edaf-45c3-aa81-e98a6da94384"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:13:02.595360 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.595320    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-model-cache" (OuterVolumeSpecName: "model-cache") pod "88cd568b-edaf-45c3-aa81-e98a6da94384" (UID: "88cd568b-edaf-45c3-aa81-e98a6da94384"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:13:02.595411 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.595364    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-tmp-dir\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:13:02.595411 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.595381    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-home\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:13:02.597174 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.597152    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88cd568b-edaf-45c3-aa81-e98a6da94384-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "88cd568b-edaf-45c3-aa81-e98a6da94384" (UID: "88cd568b-edaf-45c3-aa81-e98a6da94384"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 02:13:02.597371 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.597355    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-dshm" (OuterVolumeSpecName: "dshm") pod "88cd568b-edaf-45c3-aa81-e98a6da94384" (UID: "88cd568b-edaf-45c3-aa81-e98a6da94384"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:13:02.597441 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.597391    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88cd568b-edaf-45c3-aa81-e98a6da94384-kube-api-access-rnzf6" (OuterVolumeSpecName: "kube-api-access-rnzf6") pod "88cd568b-edaf-45c3-aa81-e98a6da94384" (UID: "88cd568b-edaf-45c3-aa81-e98a6da94384"). InnerVolumeSpecName "kube-api-access-rnzf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:13:02.695905 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.695878    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-model-cache\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:13:02.695905 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.695902    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88cd568b-edaf-45c3-aa81-e98a6da94384-dshm\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:13:02.695905 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.695911    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnzf6\" (UniqueName: \"kubernetes.io/projected/88cd568b-edaf-45c3-aa81-e98a6da94384-kube-api-access-rnzf6\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:13:02.696102 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.695920    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88cd568b-edaf-45c3-aa81-e98a6da94384-tls-certs\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:13:02.996472 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:02.996431    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bdc8be5-a2e1-47bd-9315-d834fe65f63a" path="/var/lib/kubelet/pods/1bdc8be5-a2e1-47bd-9315-d834fe65f63a/volumes"
Jul 25 02:13:03.300289 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:03.300196    2576 generic.go:356] "Generic (PLEG): container finished" podID="88cd568b-edaf-45c3-aa81-e98a6da94384" containerID="b905793a4f5793bdd6021d5d35e9e10a62d8e459982d9ed48ca084f57893c275" exitCode=0
Jul 25 02:13:03.300289 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:03.300242    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"
Jul 25 02:13:03.300481 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:03.300241    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn" event={"ID":"88cd568b-edaf-45c3-aa81-e98a6da94384","Type":"ContainerDied","Data":"b905793a4f5793bdd6021d5d35e9e10a62d8e459982d9ed48ca084f57893c275"}
Jul 25 02:13:03.300481 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:03.300369    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn" event={"ID":"88cd568b-edaf-45c3-aa81-e98a6da94384","Type":"ContainerDied","Data":"b7bb21b185e9c246d0299079a80e00361d80536cd9a264eadadba1cd1e6c70af"}
Jul 25 02:13:03.300481 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:03.300392    2576 scope.go:117] "RemoveContainer" containerID="b905793a4f5793bdd6021d5d35e9e10a62d8e459982d9ed48ca084f57893c275"
Jul 25 02:13:03.308601 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:03.308575    2576 scope.go:117] "RemoveContainer" containerID="b905793a4f5793bdd6021d5d35e9e10a62d8e459982d9ed48ca084f57893c275"
Jul 25 02:13:03.308900 ip-10-0-134-238 kubenswrapper[2576]: E0725 02:13:03.308881    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b905793a4f5793bdd6021d5d35e9e10a62d8e459982d9ed48ca084f57893c275\": container with ID starting with b905793a4f5793bdd6021d5d35e9e10a62d8e459982d9ed48ca084f57893c275 not found: ID does not exist" containerID="b905793a4f5793bdd6021d5d35e9e10a62d8e459982d9ed48ca084f57893c275"
Jul 25 02:13:03.309000 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:03.308907    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b905793a4f5793bdd6021d5d35e9e10a62d8e459982d9ed48ca084f57893c275"} err="failed to get container status \"b905793a4f5793bdd6021d5d35e9e10a62d8e459982d9ed48ca084f57893c275\": rpc error: code = NotFound desc = could not find container \"b905793a4f5793bdd6021d5d35e9e10a62d8e459982d9ed48ca084f57893c275\": container with ID starting with b905793a4f5793bdd6021d5d35e9e10a62d8e459982d9ed48ca084f57893c275 not found: ID does not exist"
Jul 25 02:13:03.313483 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:03.313464    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"]
Jul 25 02:13:03.317195 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:03.317175    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/auth-test-model-575aa7-kserve-75889cc9ff-zqzgn"]
Jul 25 02:13:03.541940 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:03.541911    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:13:03.542346 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:03.541966    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:13:03.553967 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:03.553904    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:13:04.317671 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:04.317634    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:13:04.996673 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:04.996630    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88cd568b-edaf-45c3-aa81-e98a6da94384" path="/var/lib/kubelet/pods/88cd568b-edaf-45c3-aa81-e98a6da94384/volumes"
Jul 25 02:13:18.732517 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.732475    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"]
Jul 25 02:13:18.733147 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.732950    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bdc8be5-a2e1-47bd-9315-d834fe65f63a" containerName="main"
Jul 25 02:13:18.733147 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.732970    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bdc8be5-a2e1-47bd-9315-d834fe65f63a" containerName="main"
Jul 25 02:13:18.733147 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.733007    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88cd568b-edaf-45c3-aa81-e98a6da94384" containerName="main"
Jul 25 02:13:18.733147 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.733017    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="88cd568b-edaf-45c3-aa81-e98a6da94384" containerName="main"
Jul 25 02:13:18.733147 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.733119    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="88cd568b-edaf-45c3-aa81-e98a6da94384" containerName="main"
Jul 25 02:13:18.733147 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.733132    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bdc8be5-a2e1-47bd-9315-d834fe65f63a" containerName="main"
Jul 25 02:13:18.736161 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.736141    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.738795 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.738771    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-575aa7\"/\"kube-root-ca.crt\""
Jul 25 02:13:18.738795 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.738786    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-575aa7\"/\"openshift-service-ca.crt\""
Jul 25 02:13:18.738959 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.738830    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-575aa7\"/\"sub-test-model-575aa7-kserve-self-signed-certs\""
Jul 25 02:13:18.739037 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.739020    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-575aa7\"/\"default-dockercfg-tmdb9\""
Jul 25 02:13:18.746810 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.746792    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"]
Jul 25 02:13:18.828754 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.828719    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-tmp-dir\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.828754 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.828759    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8aa470f4-44d8-4f73-958a-206ada19b482-tls-certs\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.828947 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.828785    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl5mx\" (UniqueName: \"kubernetes.io/projected/8aa470f4-44d8-4f73-958a-206ada19b482-kube-api-access-bl5mx\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.828947 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.828863    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-home\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.828947 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.828896    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-dshm\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.828947 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.828913    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-model-cache\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.929958 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.929920    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-tmp-dir\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.929958 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.929965    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8aa470f4-44d8-4f73-958a-206ada19b482-tls-certs\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.930170 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.929985    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl5mx\" (UniqueName: \"kubernetes.io/projected/8aa470f4-44d8-4f73-958a-206ada19b482-kube-api-access-bl5mx\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.930170 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.930026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-home\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.930170 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.930047    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-dshm\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.930170 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.930064    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-model-cache\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.930428 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.930400    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-home\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.930492 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.930432    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-model-cache\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.930605 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.930582    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-tmp-dir\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.932280 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.932258    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-dshm\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.932483 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.932464    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8aa470f4-44d8-4f73-958a-206ada19b482-tls-certs\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:18.939822 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:18.939797    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl5mx\" (UniqueName: \"kubernetes.io/projected/8aa470f4-44d8-4f73-958a-206ada19b482-kube-api-access-bl5mx\") pod \"sub-test-model-575aa7-kserve-975cf868c-btwb4\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") " pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:19.045330 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:19.045239    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:19.175450 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:19.175410    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"]
Jul 25 02:13:19.178315 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:13:19.178286    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8aa470f4_44d8_4f73_958a_206ada19b482.slice/crio-b5cc76c513175bd0e6e2dddec7f724bdea37f1e29f041191d39faf7b8f8cb829 WatchSource:0}: Error finding container b5cc76c513175bd0e6e2dddec7f724bdea37f1e29f041191d39faf7b8f8cb829: Status 404 returned error can't find the container with id b5cc76c513175bd0e6e2dddec7f724bdea37f1e29f041191d39faf7b8f8cb829
Jul 25 02:13:19.374032 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:19.373933    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4" event={"ID":"8aa470f4-44d8-4f73-958a-206ada19b482","Type":"ContainerStarted","Data":"d468695fbcbf71456d012a70b348678f5c31e1411f030ae16c032825d96693b0"}
Jul 25 02:13:19.374032 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:19.373976    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4" event={"ID":"8aa470f4-44d8-4f73-958a-206ada19b482","Type":"ContainerStarted","Data":"b5cc76c513175bd0e6e2dddec7f724bdea37f1e29f041191d39faf7b8f8cb829"}
Jul 25 02:13:19.390766 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:19.390716    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4" podStartSLOduration=1.390698943 podStartE2EDuration="1.390698943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 02:13:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:13:19.38960462 +0000 UTC m=+3750.876901296" watchObservedRunningTime="2026-07-25 02:13:19.390698943 +0000 UTC m=+3750.877995647"
Jul 25 02:13:22.533687 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:22.533639    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:13:29.045947 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:29.045911    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:29.046331 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:29.045963    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:29.058567 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:29.058542    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:29.427903 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:29.427870    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:13:49.365714 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:49.365628    2576 scope.go:117] "RemoveContainer" containerID="35182f1591a568d7286a05c83d8e777c29705ae19fb5dd60b7e75c89ed137833"
Jul 25 02:13:50.520435 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:13:50.520399    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:14:08.629169 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:08.629132    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:14:20.926434 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:20.926403    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"]
Jul 25 02:14:21.844694 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:21.844644    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"]
Jul 25 02:14:23.220304 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:23.220261    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:14:35.945176 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:35.945120    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr" podUID="859fce15-eb9c-4655-9d5b-ab2b80434602" containerName="main" containerID="cri-o://32cf32c7aef979af99f3f1121ce68f668e299c50447d63e40a9931c3c110fa7c" gracePeriod=45
Jul 25 02:14:36.204713 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.204642    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:14:36.287790 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.287761    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-tmp-dir\") pod \"859fce15-eb9c-4655-9d5b-ab2b80434602\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") "
Jul 25 02:14:36.287968 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.287841    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dnggf\" (UniqueName: \"kubernetes.io/projected/859fce15-eb9c-4655-9d5b-ab2b80434602-kube-api-access-dnggf\") pod \"859fce15-eb9c-4655-9d5b-ab2b80434602\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") "
Jul 25 02:14:36.287968 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.287865    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-dshm\") pod \"859fce15-eb9c-4655-9d5b-ab2b80434602\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") "
Jul 25 02:14:36.287968 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.287880    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-model-cache\") pod \"859fce15-eb9c-4655-9d5b-ab2b80434602\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") "
Jul 25 02:14:36.287968 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.287897    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/859fce15-eb9c-4655-9d5b-ab2b80434602-tls-certs\") pod \"859fce15-eb9c-4655-9d5b-ab2b80434602\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") "
Jul 25 02:14:36.287968 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.287927    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-home\") pod \"859fce15-eb9c-4655-9d5b-ab2b80434602\" (UID: \"859fce15-eb9c-4655-9d5b-ab2b80434602\") "
Jul 25 02:14:36.288251 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.288055    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "859fce15-eb9c-4655-9d5b-ab2b80434602" (UID: "859fce15-eb9c-4655-9d5b-ab2b80434602"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:14:36.288251 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.288137    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-tmp-dir\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:14:36.288251 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.288172    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-model-cache" (OuterVolumeSpecName: "model-cache") pod "859fce15-eb9c-4655-9d5b-ab2b80434602" (UID: "859fce15-eb9c-4655-9d5b-ab2b80434602"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:14:36.288404 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.288251    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-home" (OuterVolumeSpecName: "home") pod "859fce15-eb9c-4655-9d5b-ab2b80434602" (UID: "859fce15-eb9c-4655-9d5b-ab2b80434602"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:14:36.290027 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.290001    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-dshm" (OuterVolumeSpecName: "dshm") pod "859fce15-eb9c-4655-9d5b-ab2b80434602" (UID: "859fce15-eb9c-4655-9d5b-ab2b80434602"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:14:36.290137 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.290070    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/859fce15-eb9c-4655-9d5b-ab2b80434602-kube-api-access-dnggf" (OuterVolumeSpecName: "kube-api-access-dnggf") pod "859fce15-eb9c-4655-9d5b-ab2b80434602" (UID: "859fce15-eb9c-4655-9d5b-ab2b80434602"). InnerVolumeSpecName "kube-api-access-dnggf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:14:36.290137 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.290075    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/859fce15-eb9c-4655-9d5b-ab2b80434602-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "859fce15-eb9c-4655-9d5b-ab2b80434602" (UID: "859fce15-eb9c-4655-9d5b-ab2b80434602"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 02:14:36.388760 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.388729    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-home\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:14:36.388760 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.388759    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnggf\" (UniqueName: \"kubernetes.io/projected/859fce15-eb9c-4655-9d5b-ab2b80434602-kube-api-access-dnggf\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:14:36.388760 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.388769    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-dshm\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:14:36.389019 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.388779    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/859fce15-eb9c-4655-9d5b-ab2b80434602-model-cache\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:14:36.389019 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.388788    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/859fce15-eb9c-4655-9d5b-ab2b80434602-tls-certs\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:14:36.715120 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.715089    2576 generic.go:356] "Generic (PLEG): container finished" podID="859fce15-eb9c-4655-9d5b-ab2b80434602" containerID="32cf32c7aef979af99f3f1121ce68f668e299c50447d63e40a9931c3c110fa7c" exitCode=0
Jul 25 02:14:36.715284 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.715127    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"
Jul 25 02:14:36.715284 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.715152    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr" event={"ID":"859fce15-eb9c-4655-9d5b-ab2b80434602","Type":"ContainerDied","Data":"32cf32c7aef979af99f3f1121ce68f668e299c50447d63e40a9931c3c110fa7c"}
Jul 25 02:14:36.715284 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.715182    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr" event={"ID":"859fce15-eb9c-4655-9d5b-ab2b80434602","Type":"ContainerDied","Data":"10da56d98f5eeb9b4424c834c45bc5b0cdf53696246477531d77da56290e1cde"}
Jul 25 02:14:36.715284 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.715198    2576 scope.go:117] "RemoveContainer" containerID="32cf32c7aef979af99f3f1121ce68f668e299c50447d63e40a9931c3c110fa7c"
Jul 25 02:14:36.724372 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.724352    2576 scope.go:117] "RemoveContainer" containerID="32cf32c7aef979af99f3f1121ce68f668e299c50447d63e40a9931c3c110fa7c"
Jul 25 02:14:36.724621 ip-10-0-134-238 kubenswrapper[2576]: E0725 02:14:36.724599    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32cf32c7aef979af99f3f1121ce68f668e299c50447d63e40a9931c3c110fa7c\": container with ID starting with 32cf32c7aef979af99f3f1121ce68f668e299c50447d63e40a9931c3c110fa7c not found: ID does not exist" containerID="32cf32c7aef979af99f3f1121ce68f668e299c50447d63e40a9931c3c110fa7c"
Jul 25 02:14:36.724711 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.724633    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32cf32c7aef979af99f3f1121ce68f668e299c50447d63e40a9931c3c110fa7c"} err="failed to get container status \"32cf32c7aef979af99f3f1121ce68f668e299c50447d63e40a9931c3c110fa7c\": rpc error: code = NotFound desc = could not find container \"32cf32c7aef979af99f3f1121ce68f668e299c50447d63e40a9931c3c110fa7c\": container with ID starting with 32cf32c7aef979af99f3f1121ce68f668e299c50447d63e40a9931c3c110fa7c not found: ID does not exist"
Jul 25 02:14:36.733572 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.733548    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"]
Jul 25 02:14:36.736796 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.736778    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/sub-test-model-7e06ec-kserve-64dbc668c-vg7dr"]
Jul 25 02:14:36.864647 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:36.864580    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4" podUID="8aa470f4-44d8-4f73-958a-206ada19b482" containerName="main" containerID="cri-o://d468695fbcbf71456d012a70b348678f5c31e1411f030ae16c032825d96693b0" gracePeriod=45
Jul 25 02:14:37.000667 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.000619    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="859fce15-eb9c-4655-9d5b-ab2b80434602" path="/var/lib/kubelet/pods/859fce15-eb9c-4655-9d5b-ab2b80434602/volumes"
Jul 25 02:14:37.112547 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.112517    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:14:37.194229 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.194201    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl5mx\" (UniqueName: \"kubernetes.io/projected/8aa470f4-44d8-4f73-958a-206ada19b482-kube-api-access-bl5mx\") pod \"8aa470f4-44d8-4f73-958a-206ada19b482\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") "
Jul 25 02:14:37.194229 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.194232    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-tmp-dir\") pod \"8aa470f4-44d8-4f73-958a-206ada19b482\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") "
Jul 25 02:14:37.194416 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.194281    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8aa470f4-44d8-4f73-958a-206ada19b482-tls-certs\") pod \"8aa470f4-44d8-4f73-958a-206ada19b482\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") "
Jul 25 02:14:37.194416 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.194372    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-home\") pod \"8aa470f4-44d8-4f73-958a-206ada19b482\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") "
Jul 25 02:14:37.194483 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.194426    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-model-cache\") pod \"8aa470f4-44d8-4f73-958a-206ada19b482\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") "
Jul 25 02:14:37.194483 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.194447    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-dshm\") pod \"8aa470f4-44d8-4f73-958a-206ada19b482\" (UID: \"8aa470f4-44d8-4f73-958a-206ada19b482\") "
Jul 25 02:14:37.194579 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.194539    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8aa470f4-44d8-4f73-958a-206ada19b482" (UID: "8aa470f4-44d8-4f73-958a-206ada19b482"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:14:37.194631 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.194593    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-home" (OuterVolumeSpecName: "home") pod "8aa470f4-44d8-4f73-958a-206ada19b482" (UID: "8aa470f4-44d8-4f73-958a-206ada19b482"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:14:37.194720 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.194700    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-tmp-dir\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:14:37.194720 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.194704    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-model-cache" (OuterVolumeSpecName: "model-cache") pod "8aa470f4-44d8-4f73-958a-206ada19b482" (UID: "8aa470f4-44d8-4f73-958a-206ada19b482"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:14:37.194821 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.194714    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-home\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:14:37.196324 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.196303    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-dshm" (OuterVolumeSpecName: "dshm") pod "8aa470f4-44d8-4f73-958a-206ada19b482" (UID: "8aa470f4-44d8-4f73-958a-206ada19b482"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:14:37.196385 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.196310    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aa470f4-44d8-4f73-958a-206ada19b482-kube-api-access-bl5mx" (OuterVolumeSpecName: "kube-api-access-bl5mx") pod "8aa470f4-44d8-4f73-958a-206ada19b482" (UID: "8aa470f4-44d8-4f73-958a-206ada19b482"). InnerVolumeSpecName "kube-api-access-bl5mx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:14:37.196385 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.196363    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8aa470f4-44d8-4f73-958a-206ada19b482-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8aa470f4-44d8-4f73-958a-206ada19b482" (UID: "8aa470f4-44d8-4f73-958a-206ada19b482"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 02:14:37.295898 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.295814    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bl5mx\" (UniqueName: \"kubernetes.io/projected/8aa470f4-44d8-4f73-958a-206ada19b482-kube-api-access-bl5mx\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:14:37.295898 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.295841    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8aa470f4-44d8-4f73-958a-206ada19b482-tls-certs\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:14:37.295898 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.295854    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-model-cache\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:14:37.295898 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.295865    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8aa470f4-44d8-4f73-958a-206ada19b482-dshm\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:14:37.519082 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.519048    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:14:37.721706 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.721675    2576 generic.go:356] "Generic (PLEG): container finished" podID="8aa470f4-44d8-4f73-958a-206ada19b482" containerID="d468695fbcbf71456d012a70b348678f5c31e1411f030ae16c032825d96693b0" exitCode=0
Jul 25 02:14:37.721706 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.721715    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"
Jul 25 02:14:37.721973 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.721750    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4" event={"ID":"8aa470f4-44d8-4f73-958a-206ada19b482","Type":"ContainerDied","Data":"d468695fbcbf71456d012a70b348678f5c31e1411f030ae16c032825d96693b0"}
Jul 25 02:14:37.721973 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.721792    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4" event={"ID":"8aa470f4-44d8-4f73-958a-206ada19b482","Type":"ContainerDied","Data":"b5cc76c513175bd0e6e2dddec7f724bdea37f1e29f041191d39faf7b8f8cb829"}
Jul 25 02:14:37.721973 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.721810    2576 scope.go:117] "RemoveContainer" containerID="d468695fbcbf71456d012a70b348678f5c31e1411f030ae16c032825d96693b0"
Jul 25 02:14:37.730308 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.730293    2576 scope.go:117] "RemoveContainer" containerID="d468695fbcbf71456d012a70b348678f5c31e1411f030ae16c032825d96693b0"
Jul 25 02:14:37.730545 ip-10-0-134-238 kubenswrapper[2576]: E0725 02:14:37.730530    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d468695fbcbf71456d012a70b348678f5c31e1411f030ae16c032825d96693b0\": container with ID starting with d468695fbcbf71456d012a70b348678f5c31e1411f030ae16c032825d96693b0 not found: ID does not exist" containerID="d468695fbcbf71456d012a70b348678f5c31e1411f030ae16c032825d96693b0"
Jul 25 02:14:37.730590 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.730552    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d468695fbcbf71456d012a70b348678f5c31e1411f030ae16c032825d96693b0"} err="failed to get container status \"d468695fbcbf71456d012a70b348678f5c31e1411f030ae16c032825d96693b0\": rpc error: code = NotFound desc = could not find container \"d468695fbcbf71456d012a70b348678f5c31e1411f030ae16c032825d96693b0\": container with ID starting with d468695fbcbf71456d012a70b348678f5c31e1411f030ae16c032825d96693b0 not found: ID does not exist"
Jul 25 02:14:37.741174 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.741154    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"]
Jul 25 02:14:37.744218 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:37.744194    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/sub-test-model-575aa7-kserve-975cf868c-btwb4"]
Jul 25 02:14:38.996127 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:38.996096    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8aa470f4-44d8-4f73-958a-206ada19b482" path="/var/lib/kubelet/pods/8aa470f4-44d8-4f73-958a-206ada19b482/volumes"
Jul 25 02:14:39.134813 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.134776    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"]
Jul 25 02:14:39.135145 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.135128    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8aa470f4-44d8-4f73-958a-206ada19b482" containerName="main"
Jul 25 02:14:39.135226 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.135147    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8aa470f4-44d8-4f73-958a-206ada19b482" containerName="main"
Jul 25 02:14:39.135226 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.135161    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="859fce15-eb9c-4655-9d5b-ab2b80434602" containerName="main"
Jul 25 02:14:39.135226 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.135169    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="859fce15-eb9c-4655-9d5b-ab2b80434602" containerName="main"
Jul 25 02:14:39.135374 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.135273    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="859fce15-eb9c-4655-9d5b-ab2b80434602" containerName="main"
Jul 25 02:14:39.135374 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.135288    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8aa470f4-44d8-4f73-958a-206ada19b482" containerName="main"
Jul 25 02:14:39.139530 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.139511    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.141464 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.141442    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7e06ec\"/\"kube-root-ca.crt\""
Jul 25 02:14:39.141568 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.141445    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7e06ec\"/\"openshift-service-ca.crt\""
Jul 25 02:14:39.142000 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.141982    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7e06ec\"/\"rate-test-model-7e06ec-kserve-self-signed-certs\""
Jul 25 02:14:39.142119 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.142029    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7e06ec\"/\"default-dockercfg-5dxrp\""
Jul 25 02:14:39.147124 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.146721    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"]
Jul 25 02:14:39.210941 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.210915    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-dshm\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.211086 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.210953    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3683ed32-398f-4c24-93f8-a38de386e56d-tls-certs\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.211086 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.211026    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-home\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.211086 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.211075    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rmsr\" (UniqueName: \"kubernetes.io/projected/3683ed32-398f-4c24-93f8-a38de386e56d-kube-api-access-6rmsr\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.211235 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.211102    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-tmp-dir\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.211235 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.211214    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-model-cache\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.311636 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.311559    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-model-cache\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.311636 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.311598    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-dshm\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.311636 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.311619    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3683ed32-398f-4c24-93f8-a38de386e56d-tls-certs\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.311928 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.311644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-home\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.311928 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.311691    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rmsr\" (UniqueName: \"kubernetes.io/projected/3683ed32-398f-4c24-93f8-a38de386e56d-kube-api-access-6rmsr\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.311928 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.311710    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-tmp-dir\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.312102 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.312005    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-model-cache\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.312102 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.312085    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-home\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.312170 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.312094    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-tmp-dir\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.313853 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.313827    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-dshm\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.314103 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.314087    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3683ed32-398f-4c24-93f8-a38de386e56d-tls-certs\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.318951 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.318925    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rmsr\" (UniqueName: \"kubernetes.io/projected/3683ed32-398f-4c24-93f8-a38de386e56d-kube-api-access-6rmsr\") pod \"rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") " pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.450175 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.450147    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:39.582353 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.582314    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"]
Jul 25 02:14:39.584574 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:14:39.584548    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3683ed32_398f_4c24_93f8_a38de386e56d.slice/crio-e1e148188ffb70bb3bdc9033e3bae8d623864ee7bce709928efc65d89a59720f WatchSource:0}: Error finding container e1e148188ffb70bb3bdc9033e3bae8d623864ee7bce709928efc65d89a59720f: Status 404 returned error can't find the container with id e1e148188ffb70bb3bdc9033e3bae8d623864ee7bce709928efc65d89a59720f
Jul 25 02:14:39.732756 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.732721    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s" event={"ID":"3683ed32-398f-4c24-93f8-a38de386e56d","Type":"ContainerStarted","Data":"a94b1c6f9363b54915fbcee993c4b69d0e2d706970433c3ab94ff30980377401"}
Jul 25 02:14:39.732912 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.732767    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s" event={"ID":"3683ed32-398f-4c24-93f8-a38de386e56d","Type":"ContainerStarted","Data":"e1e148188ffb70bb3bdc9033e3bae8d623864ee7bce709928efc65d89a59720f"}
Jul 25 02:14:39.748021 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:39.747974    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s" podStartSLOduration=0.747960957 podStartE2EDuration="747.960957ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 02:14:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:14:39.74648386 +0000 UTC m=+3831.233780533" watchObservedRunningTime="2026-07-25 02:14:39.747960957 +0000 UTC m=+3831.235257631"
Jul 25 02:14:49.450394 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:49.450361    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:49.450689 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:49.450410    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:49.463173 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:49.463152    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:14:49.631893 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:49.626686    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:14:49.788673 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:14:49.788636    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:15:00.136849 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.136806    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29749095-98sq7"]
Jul 25 02:15:00.141242 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.141223    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e06ec-29749095-2czkm"]
Jul 25 02:15:00.141359 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.141338    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749095-98sq7"
Jul 25 02:15:00.143358 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.143339    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cjsrv\""
Jul 25 02:15:00.144453 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.144436    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-575aa7-29749095-r7psl"]
Jul 25 02:15:00.144641 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.144601    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e06ec-29749095-2czkm"
Jul 25 02:15:00.148123 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.148102    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29749095-98sq7"]
Jul 25 02:15:00.148388 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.148193    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-575aa7-29749095-r7psl"
Jul 25 02:15:00.149178 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.149159    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e06ec-29749095-2czkm"]
Jul 25 02:15:00.151261 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.151242    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-575aa7-29749095-r7psl"]
Jul 25 02:15:00.155488 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.155470    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749095-98sq7"
Jul 25 02:15:00.163058 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.163036    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e06ec-29749095-2czkm"
Jul 25 02:15:00.194002 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.193907    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-575aa7-29749095-r7psl"
Jul 25 02:15:00.312013 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.311987    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29749095-98sq7"]
Jul 25 02:15:00.358566 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.358544    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-575aa7-29749095-r7psl"]
Jul 25 02:15:00.360720 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:15:00.360694    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35baa7d1_311c_4794_b637_90704ebedae1.slice/crio-786057797fba795af8aacd8a4fcc1c18d25936280eea4b7f0a62c482c0243737 WatchSource:0}: Error finding container 786057797fba795af8aacd8a4fcc1c18d25936280eea4b7f0a62c482c0243737: Status 404 returned error can't find the container with id 786057797fba795af8aacd8a4fcc1c18d25936280eea4b7f0a62c482c0243737
Jul 25 02:15:00.546086 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.545940    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e06ec-29749095-2czkm"]
Jul 25 02:15:00.553701 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:15:00.553679    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8229208_c671_4dd7_9201_2185d0c90d22.slice/crio-814b07098d6010be72da7df5dfc2f8d55cfe3cce577e174115b5fcfa77d24dc0 WatchSource:0}: Error finding container 814b07098d6010be72da7df5dfc2f8d55cfe3cce577e174115b5fcfa77d24dc0: Status 404 returned error can't find the container with id 814b07098d6010be72da7df5dfc2f8d55cfe3cce577e174115b5fcfa77d24dc0
Jul 25 02:15:00.828622 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.828578    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-575aa7-29749095-r7psl" event={"ID":"35baa7d1-311c-4794-b637-90704ebedae1","Type":"ContainerStarted","Data":"ce16c814eff7b460dc0897b4145910f05c70726303ddb5a5282c6daa7648ca69"}
Jul 25 02:15:00.828791 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.828697    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-575aa7-29749095-r7psl" event={"ID":"35baa7d1-311c-4794-b637-90704ebedae1","Type":"ContainerStarted","Data":"786057797fba795af8aacd8a4fcc1c18d25936280eea4b7f0a62c482c0243737"}
Jul 25 02:15:00.830069 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.830044    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749095-98sq7" event={"ID":"75ed5571-22d6-43f8-9cd8-97424a713dea","Type":"ContainerStarted","Data":"720a41af443eecafdb4e5aa1b52119632ff4f885f3dbad2778b1ef61630773c9"}
Jul 25 02:15:00.830184 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.830082    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749095-98sq7" event={"ID":"75ed5571-22d6-43f8-9cd8-97424a713dea","Type":"ContainerStarted","Data":"679e36ef571bcc99b7e1d7aae4706f13ae108a52770d0f682000c70b5d42e02e"}
Jul 25 02:15:00.831166 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.831146    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e06ec-29749095-2czkm" event={"ID":"e8229208-c671-4dd7-9201-2185d0c90d22","Type":"ContainerStarted","Data":"814b07098d6010be72da7df5dfc2f8d55cfe3cce577e174115b5fcfa77d24dc0"}
Jul 25 02:15:00.844640 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.844599    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-575aa7-29749095-r7psl" podStartSLOduration=0.844578829 podStartE2EDuration="844.578829ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 02:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:15:00.841969406 +0000 UTC m=+3852.329266082" watchObservedRunningTime="2026-07-25 02:15:00.844578829 +0000 UTC m=+3852.331875503"
Jul 25 02:15:00.854873 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:00.854836    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749095-98sq7" podStartSLOduration=0.854823812 podStartE2EDuration="854.823812ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 02:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:15:00.854212208 +0000 UTC m=+3852.341508886" watchObservedRunningTime="2026-07-25 02:15:00.854823812 +0000 UTC m=+3852.342120486"
Jul 25 02:15:01.837443 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:01.837351    2576 generic.go:356] "Generic (PLEG): container finished" podID="35baa7d1-311c-4794-b637-90704ebedae1" containerID="ce16c814eff7b460dc0897b4145910f05c70726303ddb5a5282c6daa7648ca69" exitCode=0
Jul 25 02:15:01.837443 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:01.837405    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-575aa7-29749095-r7psl" event={"ID":"35baa7d1-311c-4794-b637-90704ebedae1","Type":"ContainerDied","Data":"ce16c814eff7b460dc0897b4145910f05c70726303ddb5a5282c6daa7648ca69"}
Jul 25 02:15:01.838847 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:01.838826    2576 generic.go:356] "Generic (PLEG): container finished" podID="75ed5571-22d6-43f8-9cd8-97424a713dea" containerID="720a41af443eecafdb4e5aa1b52119632ff4f885f3dbad2778b1ef61630773c9" exitCode=0
Jul 25 02:15:01.838935 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:01.838899    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749095-98sq7" event={"ID":"75ed5571-22d6-43f8-9cd8-97424a713dea","Type":"ContainerDied","Data":"720a41af443eecafdb4e5aa1b52119632ff4f885f3dbad2778b1ef61630773c9"}
Jul 25 02:15:01.840251 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:01.840233    2576 generic.go:356] "Generic (PLEG): container finished" podID="e8229208-c671-4dd7-9201-2185d0c90d22" containerID="0702bdcc7c37d90846793ca6979b47715c58a9800ee3c7bad879098ab4d43c2b" exitCode=0
Jul 25 02:15:01.840350 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:01.840258    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e06ec-29749095-2czkm" event={"ID":"e8229208-c671-4dd7-9201-2185d0c90d22","Type":"ContainerDied","Data":"0702bdcc7c37d90846793ca6979b47715c58a9800ee3c7bad879098ab4d43c2b"}
Jul 25 02:15:02.979392 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:02.979363    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-575aa7-29749095-r7psl"
Jul 25 02:15:03.021438 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:03.021417    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e06ec-29749095-2czkm"
Jul 25 02:15:03.024078 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:03.024054    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749095-98sq7"
Jul 25 02:15:03.850714 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:03.850689    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749095-98sq7"
Jul 25 02:15:03.850714 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:03.850704    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29749095-98sq7" event={"ID":"75ed5571-22d6-43f8-9cd8-97424a713dea","Type":"ContainerDied","Data":"679e36ef571bcc99b7e1d7aae4706f13ae108a52770d0f682000c70b5d42e02e"}
Jul 25 02:15:03.850976 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:03.850749    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="679e36ef571bcc99b7e1d7aae4706f13ae108a52770d0f682000c70b5d42e02e"
Jul 25 02:15:03.851989 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:03.851972    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e06ec-29749095-2czkm"
Jul 25 02:15:03.852100 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:03.852019    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e06ec-29749095-2czkm" event={"ID":"e8229208-c671-4dd7-9201-2185d0c90d22","Type":"ContainerDied","Data":"814b07098d6010be72da7df5dfc2f8d55cfe3cce577e174115b5fcfa77d24dc0"}
Jul 25 02:15:03.852100 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:03.852057    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="814b07098d6010be72da7df5dfc2f8d55cfe3cce577e174115b5fcfa77d24dc0"
Jul 25 02:15:03.853301 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:03.853280    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-575aa7-29749095-r7psl" event={"ID":"35baa7d1-311c-4794-b637-90704ebedae1","Type":"ContainerDied","Data":"786057797fba795af8aacd8a4fcc1c18d25936280eea4b7f0a62c482c0243737"}
Jul 25 02:15:03.853393 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:03.853318    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="786057797fba795af8aacd8a4fcc1c18d25936280eea4b7f0a62c482c0243737"
Jul 25 02:15:03.853393 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:03.853283    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-575aa7-29749095-r7psl"
Jul 25 02:15:06.428706 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.428647    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"]
Jul 25 02:15:06.429169 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.429139    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8229208-c671-4dd7-9201-2185d0c90d22" containerName="cleanup"
Jul 25 02:15:06.429169 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.429157    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8229208-c671-4dd7-9201-2185d0c90d22" containerName="cleanup"
Jul 25 02:15:06.429292 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.429200    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75ed5571-22d6-43f8-9cd8-97424a713dea" containerName="cleanup"
Jul 25 02:15:06.429292 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.429210    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="75ed5571-22d6-43f8-9cd8-97424a713dea" containerName="cleanup"
Jul 25 02:15:06.429292 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.429224    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35baa7d1-311c-4794-b637-90704ebedae1" containerName="cleanup"
Jul 25 02:15:06.429292 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.429235    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="35baa7d1-311c-4794-b637-90704ebedae1" containerName="cleanup"
Jul 25 02:15:06.429477 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.429311    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8229208-c671-4dd7-9201-2185d0c90d22" containerName="cleanup"
Jul 25 02:15:06.429477 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.429325    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="35baa7d1-311c-4794-b637-90704ebedae1" containerName="cleanup"
Jul 25 02:15:06.429477 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.429335    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="75ed5571-22d6-43f8-9cd8-97424a713dea" containerName="cleanup"
Jul 25 02:15:06.432388 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.432366    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.434934 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.434916    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-575aa7\"/\"default-dockercfg-tmdb9\""
Jul 25 02:15:06.435043 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.434937    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-575aa7\"/\"openshift-service-ca.crt\""
Jul 25 02:15:06.435043 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.434966    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-575aa7\"/\"kube-root-ca.crt\""
Jul 25 02:15:06.435176 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.435162    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-575aa7\"/\"rate-test-model-575aa7-kserve-self-signed-certs\""
Jul 25 02:15:06.443707 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.443687    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"]
Jul 25 02:15:06.527969 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.527942    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqzhd\" (UniqueName: \"kubernetes.io/projected/ec085dc4-cd84-4087-8c92-79304107f59f-kube-api-access-lqzhd\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.528090 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.527984    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-home\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.528090 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.528001    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-dshm\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.528090 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.528072    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-tmp-dir\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.528218 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.528095    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec085dc4-cd84-4087-8c92-79304107f59f-tls-certs\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.528218 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.528114    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-model-cache\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.628964 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.628939    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec085dc4-cd84-4087-8c92-79304107f59f-tls-certs\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.629098 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.628972    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-model-cache\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.629098 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.628997    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqzhd\" (UniqueName: \"kubernetes.io/projected/ec085dc4-cd84-4087-8c92-79304107f59f-kube-api-access-lqzhd\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.629098 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.629027    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-home\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.629264 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.629101    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-dshm\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.629264 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.629160    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-tmp-dir\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.629420 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.629401    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-model-cache\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.629480 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.629409    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-home\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.629480 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.629469    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-tmp-dir\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.631346 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.631324    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-dshm\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.631567 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.631549    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec085dc4-cd84-4087-8c92-79304107f59f-tls-certs\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.636443 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.636422    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqzhd\" (UniqueName: \"kubernetes.io/projected/ec085dc4-cd84-4087-8c92-79304107f59f-kube-api-access-lqzhd\") pod \"rate-test-model-575aa7-kserve-5b54f94947-ztvrf\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") " pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.741332 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.741259    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:06.870927 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:06.870733    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"]
Jul 25 02:15:06.873539 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:15:06.873512    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec085dc4_cd84_4087_8c92_79304107f59f.slice/crio-0ff74c55050c9069a2a32de6cbd599240a3cb421da91fbd1ec9d863cd117ed29 WatchSource:0}: Error finding container 0ff74c55050c9069a2a32de6cbd599240a3cb421da91fbd1ec9d863cd117ed29: Status 404 returned error can't find the container with id 0ff74c55050c9069a2a32de6cbd599240a3cb421da91fbd1ec9d863cd117ed29
Jul 25 02:15:07.874027 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:07.873996    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf" event={"ID":"ec085dc4-cd84-4087-8c92-79304107f59f","Type":"ContainerStarted","Data":"5c361e968a248f24d9360f098590ebe9515465d887cd03fd017f710d22cd4474"}
Jul 25 02:15:07.874425 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:07.874035    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf" event={"ID":"ec085dc4-cd84-4087-8c92-79304107f59f","Type":"ContainerStarted","Data":"0ff74c55050c9069a2a32de6cbd599240a3cb421da91fbd1ec9d863cd117ed29"}
Jul 25 02:15:07.891112 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:07.891068    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf" podStartSLOduration=1.891054352 podStartE2EDuration="1.891054352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 02:15:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:15:07.888812044 +0000 UTC m=+3859.376108720" watchObservedRunningTime="2026-07-25 02:15:07.891054352 +0000 UTC m=+3859.378351027"
Jul 25 02:15:10.420776 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:10.420740    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:15:16.742082 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:16.741985    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:16.742082 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:16.742050    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:16.754695 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:16.754668    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:16.929276 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:16.929247    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:15:45.229064 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:45.229028    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:15:49.256280 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:49.256183    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 02:15:49.262352 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:49.262331    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 02:15:49.264753 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:49.264733    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 02:15:49.269730 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:15:49.269696    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 02:16:00.329155 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:00.329123    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:16:14.223187 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:14.223153    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:16:20.144344 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:20.144312    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"]
Jul 25 02:16:21.014142 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:21.014107    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"]
Jul 25 02:16:22.014778 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:22.014745    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7962-139975443771952"]
Jul 25 02:16:22.018209 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:22.018189    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443771952"
Jul 25 02:16:22.020215 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:22.020195    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-pwd6j\""
Jul 25 02:16:22.022999 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:22.022979    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7962-139975443771952"]
Jul 25 02:16:22.069222 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:22.069190    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zznz\" (UniqueName: \"kubernetes.io/projected/1c128aa7-6516-419f-bc5c-55268cdcdcc1-kube-api-access-7zznz\") pod \"test-curl-7962-139975443771952\" (UID: \"1c128aa7-6516-419f-bc5c-55268cdcdcc1\") " pod="opendatahub/test-curl-7962-139975443771952"
Jul 25 02:16:22.170105 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:22.170072    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zznz\" (UniqueName: \"kubernetes.io/projected/1c128aa7-6516-419f-bc5c-55268cdcdcc1-kube-api-access-7zznz\") pod \"test-curl-7962-139975443771952\" (UID: \"1c128aa7-6516-419f-bc5c-55268cdcdcc1\") " pod="opendatahub/test-curl-7962-139975443771952"
Jul 25 02:16:22.178394 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:22.178364    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zznz\" (UniqueName: \"kubernetes.io/projected/1c128aa7-6516-419f-bc5c-55268cdcdcc1-kube-api-access-7zznz\") pod \"test-curl-7962-139975443771952\" (UID: \"1c128aa7-6516-419f-bc5c-55268cdcdcc1\") " pod="opendatahub/test-curl-7962-139975443771952"
Jul 25 02:16:22.327859 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:22.327776    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443771952"
Jul 25 02:16:22.457156 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:22.457129    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7962-139975443771952"]
Jul 25 02:16:22.459565 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:16:22.459538    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c128aa7_6516_419f_bc5c_55268cdcdcc1.slice/crio-1377bd649c4261de07219e1e67bb726d80525b496fb1edf33392d7cbebbcf373 WatchSource:0}: Error finding container 1377bd649c4261de07219e1e67bb726d80525b496fb1edf33392d7cbebbcf373: Status 404 returned error can't find the container with id 1377bd649c4261de07219e1e67bb726d80525b496fb1edf33392d7cbebbcf373
Jul 25 02:16:22.460965 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:22.460949    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 02:16:23.210370 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:23.210344    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7962-139975443771952" event={"ID":"1c128aa7-6516-419f-bc5c-55268cdcdcc1","Type":"ContainerStarted","Data":"1377bd649c4261de07219e1e67bb726d80525b496fb1edf33392d7cbebbcf373"}
Jul 25 02:16:24.216604 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:24.216564    2576 generic.go:356] "Generic (PLEG): container finished" podID="1c128aa7-6516-419f-bc5c-55268cdcdcc1" containerID="1f5db047bfc682ea7428d8d4ad90e3dda63c6bba336816cec1eca8d58e8c8338" exitCode=0
Jul 25 02:16:24.216995 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:24.216647    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7962-139975443771952" event={"ID":"1c128aa7-6516-419f-bc5c-55268cdcdcc1","Type":"ContainerDied","Data":"1f5db047bfc682ea7428d8d4ad90e3dda63c6bba336816cec1eca8d58e8c8338"}
Jul 25 02:16:25.354247 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.354220    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443771952"
Jul 25 02:16:25.388472 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.388446    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7962-139975443771952_1c128aa7-6516-419f-bc5c-55268cdcdcc1/test-curl-7962-139975443771952/0.log"
Jul 25 02:16:25.395682 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.395649    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zznz\" (UniqueName: \"kubernetes.io/projected/1c128aa7-6516-419f-bc5c-55268cdcdcc1-kube-api-access-7zznz\") pod \"1c128aa7-6516-419f-bc5c-55268cdcdcc1\" (UID: \"1c128aa7-6516-419f-bc5c-55268cdcdcc1\") "
Jul 25 02:16:25.397785 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.397758    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c128aa7-6516-419f-bc5c-55268cdcdcc1-kube-api-access-7zznz" (OuterVolumeSpecName: "kube-api-access-7zznz") pod "1c128aa7-6516-419f-bc5c-55268cdcdcc1" (UID: "1c128aa7-6516-419f-bc5c-55268cdcdcc1"). InnerVolumeSpecName "kube-api-access-7zznz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:16:25.412320 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.412299    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7962-139975443771952"]
Jul 25 02:16:25.415840 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.415820    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7962-139975443771952"]
Jul 25 02:16:25.496389 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.496326    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zznz\" (UniqueName: \"kubernetes.io/projected/1c128aa7-6516-419f-bc5c-55268cdcdcc1-kube-api-access-7zznz\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:25.538840 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.538809    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7962-139975443677232"]
Jul 25 02:16:25.539137 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.539124    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c128aa7-6516-419f-bc5c-55268cdcdcc1" containerName="test-curl-7962-139975443771952"
Jul 25 02:16:25.539189 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.539139    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c128aa7-6516-419f-bc5c-55268cdcdcc1" containerName="test-curl-7962-139975443771952"
Jul 25 02:16:25.539232 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.539217    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c128aa7-6516-419f-bc5c-55268cdcdcc1" containerName="test-curl-7962-139975443771952"
Jul 25 02:16:25.542117 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.542102    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443677232"
Jul 25 02:16:25.548682 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.548643    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7962-139975443677232"]
Jul 25 02:16:25.596854 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.596828    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m6wv\" (UniqueName: \"kubernetes.io/projected/51adaaf0-22f8-4932-9a0d-c1b8b019e43a-kube-api-access-2m6wv\") pod \"test-curl-7962-139975443677232\" (UID: \"51adaaf0-22f8-4932-9a0d-c1b8b019e43a\") " pod="opendatahub/test-curl-7962-139975443677232"
Jul 25 02:16:25.697535 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.697501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m6wv\" (UniqueName: \"kubernetes.io/projected/51adaaf0-22f8-4932-9a0d-c1b8b019e43a-kube-api-access-2m6wv\") pod \"test-curl-7962-139975443677232\" (UID: \"51adaaf0-22f8-4932-9a0d-c1b8b019e43a\") " pod="opendatahub/test-curl-7962-139975443677232"
Jul 25 02:16:25.704478 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.704451    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m6wv\" (UniqueName: \"kubernetes.io/projected/51adaaf0-22f8-4932-9a0d-c1b8b019e43a-kube-api-access-2m6wv\") pod \"test-curl-7962-139975443677232\" (UID: \"51adaaf0-22f8-4932-9a0d-c1b8b019e43a\") " pod="opendatahub/test-curl-7962-139975443677232"
Jul 25 02:16:25.852060 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.851984    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443677232"
Jul 25 02:16:25.975379 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:25.975357    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7962-139975443677232"]
Jul 25 02:16:25.977866 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:16:25.977846    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51adaaf0_22f8_4932_9a0d_c1b8b019e43a.slice/crio-7cc715422dcb98c5ec3a2893e79f1d1c23f54966af6442de9e6f53473d0126cb WatchSource:0}: Error finding container 7cc715422dcb98c5ec3a2893e79f1d1c23f54966af6442de9e6f53473d0126cb: Status 404 returned error can't find the container with id 7cc715422dcb98c5ec3a2893e79f1d1c23f54966af6442de9e6f53473d0126cb
Jul 25 02:16:26.226643 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:26.226617    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1377bd649c4261de07219e1e67bb726d80525b496fb1edf33392d7cbebbcf373"
Jul 25 02:16:26.226820 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:26.226646    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443771952"
Jul 25 02:16:26.227896 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:26.227869    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7962-139975443677232" event={"ID":"51adaaf0-22f8-4932-9a0d-c1b8b019e43a","Type":"ContainerStarted","Data":"7cc715422dcb98c5ec3a2893e79f1d1c23f54966af6442de9e6f53473d0126cb"}
Jul 25 02:16:26.237705 ip-10-0-134-238 kubenswrapper[2576]: E0725 02:16:26.237679    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7962-139975443771952\" is forbidden: User \"system:node:ip-10-0-134-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-238.ec2.internal' and this object" podUID="1c128aa7-6516-419f-bc5c-55268cdcdcc1" pod="opendatahub/test-curl-7962-139975443771952"
Jul 25 02:16:26.996555 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:26.996525    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c128aa7-6516-419f-bc5c-55268cdcdcc1" path="/var/lib/kubelet/pods/1c128aa7-6516-419f-bc5c-55268cdcdcc1/volumes"
Jul 25 02:16:27.234897 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:27.234868    2576 generic.go:356] "Generic (PLEG): container finished" podID="51adaaf0-22f8-4932-9a0d-c1b8b019e43a" containerID="90430537a3dbcf70d0781781468d425ee9d085133a6b27497f4846b94b299caf" exitCode=0
Jul 25 02:16:27.235071 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:27.234927    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7962-139975443677232" event={"ID":"51adaaf0-22f8-4932-9a0d-c1b8b019e43a","Type":"ContainerDied","Data":"90430537a3dbcf70d0781781468d425ee9d085133a6b27497f4846b94b299caf"}
Jul 25 02:16:27.928282 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:27.928246    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:16:28.371905 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.371874    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443677232"
Jul 25 02:16:28.390358 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.390321    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7962-139975443677232_51adaaf0-22f8-4932-9a0d-c1b8b019e43a/test-curl-7962-139975443677232/0.log"
Jul 25 02:16:28.411726 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.411628    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7962-139975443677232"]
Jul 25 02:16:28.413638 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.413610    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7962-139975443677232"]
Jul 25 02:16:28.419554 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.419539    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2m6wv\" (UniqueName: \"kubernetes.io/projected/51adaaf0-22f8-4932-9a0d-c1b8b019e43a-kube-api-access-2m6wv\") pod \"51adaaf0-22f8-4932-9a0d-c1b8b019e43a\" (UID: \"51adaaf0-22f8-4932-9a0d-c1b8b019e43a\") "
Jul 25 02:16:28.421543 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.421520    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51adaaf0-22f8-4932-9a0d-c1b8b019e43a-kube-api-access-2m6wv" (OuterVolumeSpecName: "kube-api-access-2m6wv") pod "51adaaf0-22f8-4932-9a0d-c1b8b019e43a" (UID: "51adaaf0-22f8-4932-9a0d-c1b8b019e43a"). InnerVolumeSpecName "kube-api-access-2m6wv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:16:28.520237 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.520168    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2m6wv\" (UniqueName: \"kubernetes.io/projected/51adaaf0-22f8-4932-9a0d-c1b8b019e43a-kube-api-access-2m6wv\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:28.572887 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.572860    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7962-139975443817648"]
Jul 25 02:16:28.573243 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.573229    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51adaaf0-22f8-4932-9a0d-c1b8b019e43a" containerName="test-curl-7962-139975443677232"
Jul 25 02:16:28.573283 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.573247    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="51adaaf0-22f8-4932-9a0d-c1b8b019e43a" containerName="test-curl-7962-139975443677232"
Jul 25 02:16:28.573315 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.573309    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="51adaaf0-22f8-4932-9a0d-c1b8b019e43a" containerName="test-curl-7962-139975443677232"
Jul 25 02:16:28.576184 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.576170    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443817648"
Jul 25 02:16:28.584121 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.584099    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7962-139975443817648"]
Jul 25 02:16:28.621303 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.621273    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxvjq\" (UniqueName: \"kubernetes.io/projected/d060d50f-7c6e-4252-86ca-6e4c93ae7c4d-kube-api-access-wxvjq\") pod \"test-curl-7962-139975443817648\" (UID: \"d060d50f-7c6e-4252-86ca-6e4c93ae7c4d\") " pod="opendatahub/test-curl-7962-139975443817648"
Jul 25 02:16:28.722707 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.722669    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxvjq\" (UniqueName: \"kubernetes.io/projected/d060d50f-7c6e-4252-86ca-6e4c93ae7c4d-kube-api-access-wxvjq\") pod \"test-curl-7962-139975443817648\" (UID: \"d060d50f-7c6e-4252-86ca-6e4c93ae7c4d\") " pod="opendatahub/test-curl-7962-139975443817648"
Jul 25 02:16:28.730639 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.730605    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxvjq\" (UniqueName: \"kubernetes.io/projected/d060d50f-7c6e-4252-86ca-6e4c93ae7c4d-kube-api-access-wxvjq\") pod \"test-curl-7962-139975443817648\" (UID: \"d060d50f-7c6e-4252-86ca-6e4c93ae7c4d\") " pod="opendatahub/test-curl-7962-139975443817648"
Jul 25 02:16:28.886833 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.886756    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443817648"
Jul 25 02:16:28.998223 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:28.998193    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51adaaf0-22f8-4932-9a0d-c1b8b019e43a" path="/var/lib/kubelet/pods/51adaaf0-22f8-4932-9a0d-c1b8b019e43a/volumes"
Jul 25 02:16:29.010061 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:29.009926    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7962-139975443817648"]
Jul 25 02:16:29.012499 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:16:29.012476    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd060d50f_7c6e_4252_86ca_6e4c93ae7c4d.slice/crio-a4e23531e348dc8359fc845d4a3da8989c5f70519a45282297d973bca12579ee WatchSource:0}: Error finding container a4e23531e348dc8359fc845d4a3da8989c5f70519a45282297d973bca12579ee: Status 404 returned error can't find the container with id a4e23531e348dc8359fc845d4a3da8989c5f70519a45282297d973bca12579ee
Jul 25 02:16:29.245809 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:29.245782    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443677232"
Jul 25 02:16:29.246031 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:29.245782    2576 scope.go:117] "RemoveContainer" containerID="90430537a3dbcf70d0781781468d425ee9d085133a6b27497f4846b94b299caf"
Jul 25 02:16:29.247179 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:29.246934    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7962-139975443817648" event={"ID":"d060d50f-7c6e-4252-86ca-6e4c93ae7c4d","Type":"ContainerStarted","Data":"a4e23531e348dc8359fc845d4a3da8989c5f70519a45282297d973bca12579ee"}
Jul 25 02:16:30.253768 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:30.253731    2576 generic.go:356] "Generic (PLEG): container finished" podID="d060d50f-7c6e-4252-86ca-6e4c93ae7c4d" containerID="5d9c3430146550f3cb24296c2435b0987bedd586f9f014cf6e9ff600554ecd4b" exitCode=0
Jul 25 02:16:30.254148 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:30.253796    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7962-139975443817648" event={"ID":"d060d50f-7c6e-4252-86ca-6e4c93ae7c4d","Type":"ContainerDied","Data":"5d9c3430146550f3cb24296c2435b0987bedd586f9f014cf6e9ff600554ecd4b"}
Jul 25 02:16:31.386715 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.386691    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443817648"
Jul 25 02:16:31.404355 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.404334    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7962-139975443817648_d060d50f-7c6e-4252-86ca-6e4c93ae7c4d/test-curl-7962-139975443817648/0.log"
Jul 25 02:16:31.429673 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.429634    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7962-139975443817648"]
Jul 25 02:16:31.432751 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.432729    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7962-139975443817648"]
Jul 25 02:16:31.443279 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.443258    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxvjq\" (UniqueName: \"kubernetes.io/projected/d060d50f-7c6e-4252-86ca-6e4c93ae7c4d-kube-api-access-wxvjq\") pod \"d060d50f-7c6e-4252-86ca-6e4c93ae7c4d\" (UID: \"d060d50f-7c6e-4252-86ca-6e4c93ae7c4d\") "
Jul 25 02:16:31.445337 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.445306    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d060d50f-7c6e-4252-86ca-6e4c93ae7c4d-kube-api-access-wxvjq" (OuterVolumeSpecName: "kube-api-access-wxvjq") pod "d060d50f-7c6e-4252-86ca-6e4c93ae7c4d" (UID: "d060d50f-7c6e-4252-86ca-6e4c93ae7c4d"). InnerVolumeSpecName "kube-api-access-wxvjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:16:31.544625 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.544560    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxvjq\" (UniqueName: \"kubernetes.io/projected/d060d50f-7c6e-4252-86ca-6e4c93ae7c4d-kube-api-access-wxvjq\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:31.553845 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.553818    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7962-139975443817648"]
Jul 25 02:16:31.554162 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.554146    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d060d50f-7c6e-4252-86ca-6e4c93ae7c4d" containerName="test-curl-7962-139975443817648"
Jul 25 02:16:31.554237 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.554165    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d060d50f-7c6e-4252-86ca-6e4c93ae7c4d" containerName="test-curl-7962-139975443817648"
Jul 25 02:16:31.554307 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.554294    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d060d50f-7c6e-4252-86ca-6e4c93ae7c4d" containerName="test-curl-7962-139975443817648"
Jul 25 02:16:31.557087 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.557071    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443817648"
Jul 25 02:16:31.562048 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.562027    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7962-139975443817648"]
Jul 25 02:16:31.645413 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.645382    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdg7g\" (UniqueName: \"kubernetes.io/projected/6bbf3bba-2bfe-4e3d-99ea-ad163c802e80-kube-api-access-gdg7g\") pod \"test-curl-7962-139975443817648\" (UID: \"6bbf3bba-2bfe-4e3d-99ea-ad163c802e80\") " pod="opendatahub/test-curl-7962-139975443817648"
Jul 25 02:16:31.745985 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.745955    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdg7g\" (UniqueName: \"kubernetes.io/projected/6bbf3bba-2bfe-4e3d-99ea-ad163c802e80-kube-api-access-gdg7g\") pod \"test-curl-7962-139975443817648\" (UID: \"6bbf3bba-2bfe-4e3d-99ea-ad163c802e80\") " pod="opendatahub/test-curl-7962-139975443817648"
Jul 25 02:16:31.753459 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.753437    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdg7g\" (UniqueName: \"kubernetes.io/projected/6bbf3bba-2bfe-4e3d-99ea-ad163c802e80-kube-api-access-gdg7g\") pod \"test-curl-7962-139975443817648\" (UID: \"6bbf3bba-2bfe-4e3d-99ea-ad163c802e80\") " pod="opendatahub/test-curl-7962-139975443817648"
Jul 25 02:16:31.867118 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.867033    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443817648"
Jul 25 02:16:31.993417 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:31.993391    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7962-139975443817648"]
Jul 25 02:16:31.995416 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:16:31.995388    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6bbf3bba_2bfe_4e3d_99ea_ad163c802e80.slice/crio-5d8bb0411a3c02db7faf9dd1382c44596687b47feb58eb2e7639d6adf658f592 WatchSource:0}: Error finding container 5d8bb0411a3c02db7faf9dd1382c44596687b47feb58eb2e7639d6adf658f592: Status 404 returned error can't find the container with id 5d8bb0411a3c02db7faf9dd1382c44596687b47feb58eb2e7639d6adf658f592
Jul 25 02:16:32.263857 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:32.263819    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7962-139975443817648" event={"ID":"6bbf3bba-2bfe-4e3d-99ea-ad163c802e80","Type":"ContainerStarted","Data":"5d8bb0411a3c02db7faf9dd1382c44596687b47feb58eb2e7639d6adf658f592"}
Jul 25 02:16:32.264852 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:32.264832    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a4e23531e348dc8359fc845d4a3da8989c5f70519a45282297d973bca12579ee"
Jul 25 02:16:32.264978 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:32.264852    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443817648"
Jul 25 02:16:32.275678 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:32.275640    2576 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7962-139975443817648" oldPodUID="d060d50f-7c6e-4252-86ca-6e4c93ae7c4d" podUID="6bbf3bba-2bfe-4e3d-99ea-ad163c802e80"
Jul 25 02:16:32.996997 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:32.996962    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d060d50f-7c6e-4252-86ca-6e4c93ae7c4d" path="/var/lib/kubelet/pods/d060d50f-7c6e-4252-86ca-6e4c93ae7c4d/volumes"
Jul 25 02:16:33.271341 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:33.271257    2576 generic.go:356] "Generic (PLEG): container finished" podID="6bbf3bba-2bfe-4e3d-99ea-ad163c802e80" containerID="7d3dc63e4406fd861aa9b989d0c37020aeabf65b3aacb2c2c46760099224284d" exitCode=0
Jul 25 02:16:33.271476 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:33.271343    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7962-139975443817648" event={"ID":"6bbf3bba-2bfe-4e3d-99ea-ad163c802e80","Type":"ContainerDied","Data":"7d3dc63e4406fd861aa9b989d0c37020aeabf65b3aacb2c2c46760099224284d"}
Jul 25 02:16:34.405360 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.405336    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443817648"
Jul 25 02:16:34.421918 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.421895    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7962-139975443817648_6bbf3bba-2bfe-4e3d-99ea-ad163c802e80/test-curl-7962-139975443817648/0.log"
Jul 25 02:16:34.442592 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.442567    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7962-139975443817648"]
Jul 25 02:16:34.446262 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.446244    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7962-139975443817648"]
Jul 25 02:16:34.468816 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.468796    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdg7g\" (UniqueName: \"kubernetes.io/projected/6bbf3bba-2bfe-4e3d-99ea-ad163c802e80-kube-api-access-gdg7g\") pod \"6bbf3bba-2bfe-4e3d-99ea-ad163c802e80\" (UID: \"6bbf3bba-2bfe-4e3d-99ea-ad163c802e80\") "
Jul 25 02:16:34.470779 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.470751    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bbf3bba-2bfe-4e3d-99ea-ad163c802e80-kube-api-access-gdg7g" (OuterVolumeSpecName: "kube-api-access-gdg7g") pod "6bbf3bba-2bfe-4e3d-99ea-ad163c802e80" (UID: "6bbf3bba-2bfe-4e3d-99ea-ad163c802e80"). InnerVolumeSpecName "kube-api-access-gdg7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:16:34.562217 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.562156    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7962-139975443830656"]
Jul 25 02:16:34.562499 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.562485    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bbf3bba-2bfe-4e3d-99ea-ad163c802e80" containerName="test-curl-7962-139975443817648"
Jul 25 02:16:34.562546 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.562502    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bbf3bba-2bfe-4e3d-99ea-ad163c802e80" containerName="test-curl-7962-139975443817648"
Jul 25 02:16:34.562588 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.562571    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bbf3bba-2bfe-4e3d-99ea-ad163c802e80" containerName="test-curl-7962-139975443817648"
Jul 25 02:16:34.565385 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.565370    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443830656"
Jul 25 02:16:34.569593 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.569570    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdg7g\" (UniqueName: \"kubernetes.io/projected/6bbf3bba-2bfe-4e3d-99ea-ad163c802e80-kube-api-access-gdg7g\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:34.571511 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.571489    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7962-139975443830656"]
Jul 25 02:16:34.670417 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.670382    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8msj\" (UniqueName: \"kubernetes.io/projected/784af2e7-cb7a-4143-af04-0532440c4dca-kube-api-access-n8msj\") pod \"test-curl-7962-139975443830656\" (UID: \"784af2e7-cb7a-4143-af04-0532440c4dca\") " pod="opendatahub/test-curl-7962-139975443830656"
Jul 25 02:16:34.771068 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.771039    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8msj\" (UniqueName: \"kubernetes.io/projected/784af2e7-cb7a-4143-af04-0532440c4dca-kube-api-access-n8msj\") pod \"test-curl-7962-139975443830656\" (UID: \"784af2e7-cb7a-4143-af04-0532440c4dca\") " pod="opendatahub/test-curl-7962-139975443830656"
Jul 25 02:16:34.777990 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.777966    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8msj\" (UniqueName: \"kubernetes.io/projected/784af2e7-cb7a-4143-af04-0532440c4dca-kube-api-access-n8msj\") pod \"test-curl-7962-139975443830656\" (UID: \"784af2e7-cb7a-4143-af04-0532440c4dca\") " pod="opendatahub/test-curl-7962-139975443830656"
Jul 25 02:16:34.874800 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.874719    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443830656"
Jul 25 02:16:34.996410 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.996372    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bbf3bba-2bfe-4e3d-99ea-ad163c802e80" path="/var/lib/kubelet/pods/6bbf3bba-2bfe-4e3d-99ea-ad163c802e80/volumes"
Jul 25 02:16:34.998568 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:34.998542    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7962-139975443830656"]
Jul 25 02:16:35.001496 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:16:35.001473    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod784af2e7_cb7a_4143_af04_0532440c4dca.slice/crio-f10393e3dc3f3962962e99ddb8129c4e6be2d566e0c4c21f32cb04c95c72e14f WatchSource:0}: Error finding container f10393e3dc3f3962962e99ddb8129c4e6be2d566e0c4c21f32cb04c95c72e14f: Status 404 returned error can't find the container with id f10393e3dc3f3962962e99ddb8129c4e6be2d566e0c4c21f32cb04c95c72e14f
Jul 25 02:16:35.162783 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.162756    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s" podUID="3683ed32-398f-4c24-93f8-a38de386e56d" containerName="main" containerID="cri-o://a94b1c6f9363b54915fbcee993c4b69d0e2d706970433c3ab94ff30980377401" gracePeriod=45
Jul 25 02:16:35.282089 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.282068    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443817648"
Jul 25 02:16:35.282213 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.282105    2576 scope.go:117] "RemoveContainer" containerID="7d3dc63e4406fd861aa9b989d0c37020aeabf65b3aacb2c2c46760099224284d"
Jul 25 02:16:35.284151 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.284114    2576 generic.go:356] "Generic (PLEG): container finished" podID="3683ed32-398f-4c24-93f8-a38de386e56d" containerID="a94b1c6f9363b54915fbcee993c4b69d0e2d706970433c3ab94ff30980377401" exitCode=0
Jul 25 02:16:35.284259 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.284215    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s" event={"ID":"3683ed32-398f-4c24-93f8-a38de386e56d","Type":"ContainerDied","Data":"a94b1c6f9363b54915fbcee993c4b69d0e2d706970433c3ab94ff30980377401"}
Jul 25 02:16:35.285558 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.285535    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7962-139975443830656" event={"ID":"784af2e7-cb7a-4143-af04-0532440c4dca","Type":"ContainerStarted","Data":"f10393e3dc3f3962962e99ddb8129c4e6be2d566e0c4c21f32cb04c95c72e14f"}
Jul 25 02:16:35.468694 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.468628    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:16:35.577566 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.577538    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-tmp-dir\") pod \"3683ed32-398f-4c24-93f8-a38de386e56d\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") "
Jul 25 02:16:35.577739 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.577607    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rmsr\" (UniqueName: \"kubernetes.io/projected/3683ed32-398f-4c24-93f8-a38de386e56d-kube-api-access-6rmsr\") pod \"3683ed32-398f-4c24-93f8-a38de386e56d\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") "
Jul 25 02:16:35.577739 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.577673    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-model-cache\") pod \"3683ed32-398f-4c24-93f8-a38de386e56d\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") "
Jul 25 02:16:35.577739 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.577722    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-dshm\") pod \"3683ed32-398f-4c24-93f8-a38de386e56d\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") "
Jul 25 02:16:35.577882 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.577746    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3683ed32-398f-4c24-93f8-a38de386e56d-tls-certs\") pod \"3683ed32-398f-4c24-93f8-a38de386e56d\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") "
Jul 25 02:16:35.577882 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.577785    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-home\") pod \"3683ed32-398f-4c24-93f8-a38de386e56d\" (UID: \"3683ed32-398f-4c24-93f8-a38de386e56d\") "
Jul 25 02:16:35.577882 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.577828    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3683ed32-398f-4c24-93f8-a38de386e56d" (UID: "3683ed32-398f-4c24-93f8-a38de386e56d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:16:35.578035 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.577992    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-model-cache" (OuterVolumeSpecName: "model-cache") pod "3683ed32-398f-4c24-93f8-a38de386e56d" (UID: "3683ed32-398f-4c24-93f8-a38de386e56d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:16:35.578035 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.578009    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-tmp-dir\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:35.578139 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.578050    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-home" (OuterVolumeSpecName: "home") pod "3683ed32-398f-4c24-93f8-a38de386e56d" (UID: "3683ed32-398f-4c24-93f8-a38de386e56d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:16:35.579848 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.579825    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3683ed32-398f-4c24-93f8-a38de386e56d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3683ed32-398f-4c24-93f8-a38de386e56d" (UID: "3683ed32-398f-4c24-93f8-a38de386e56d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 02:16:35.579930 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.579901    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3683ed32-398f-4c24-93f8-a38de386e56d-kube-api-access-6rmsr" (OuterVolumeSpecName: "kube-api-access-6rmsr") pod "3683ed32-398f-4c24-93f8-a38de386e56d" (UID: "3683ed32-398f-4c24-93f8-a38de386e56d"). InnerVolumeSpecName "kube-api-access-6rmsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:16:35.579982 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.579963    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-dshm" (OuterVolumeSpecName: "dshm") pod "3683ed32-398f-4c24-93f8-a38de386e56d" (UID: "3683ed32-398f-4c24-93f8-a38de386e56d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:16:35.679224 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.679187    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-dshm\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:35.679224 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.679219    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3683ed32-398f-4c24-93f8-a38de386e56d-tls-certs\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:35.679224 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.679230    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-home\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:35.679224 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.679239    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rmsr\" (UniqueName: \"kubernetes.io/projected/3683ed32-398f-4c24-93f8-a38de386e56d-kube-api-access-6rmsr\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:35.679452 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:35.679247    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3683ed32-398f-4c24-93f8-a38de386e56d-model-cache\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:36.034389 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.034352    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf" podUID="ec085dc4-cd84-4087-8c92-79304107f59f" containerName="main" containerID="cri-o://5c361e968a248f24d9360f098590ebe9515465d887cd03fd017f710d22cd4474" gracePeriod=45
Jul 25 02:16:36.273644 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.273623    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:16:36.293987 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.293921    2576 generic.go:356] "Generic (PLEG): container finished" podID="ec085dc4-cd84-4087-8c92-79304107f59f" containerID="5c361e968a248f24d9360f098590ebe9515465d887cd03fd017f710d22cd4474" exitCode=0
Jul 25 02:16:36.293987 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.293957    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"
Jul 25 02:16:36.294153 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.293995    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf" event={"ID":"ec085dc4-cd84-4087-8c92-79304107f59f","Type":"ContainerDied","Data":"5c361e968a248f24d9360f098590ebe9515465d887cd03fd017f710d22cd4474"}
Jul 25 02:16:36.294153 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.294030    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf" event={"ID":"ec085dc4-cd84-4087-8c92-79304107f59f","Type":"ContainerDied","Data":"0ff74c55050c9069a2a32de6cbd599240a3cb421da91fbd1ec9d863cd117ed29"}
Jul 25 02:16:36.294153 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.294052    2576 scope.go:117] "RemoveContainer" containerID="5c361e968a248f24d9360f098590ebe9515465d887cd03fd017f710d22cd4474"
Jul 25 02:16:36.296462 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.296442    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"
Jul 25 02:16:36.296566 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.296475    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s" event={"ID":"3683ed32-398f-4c24-93f8-a38de386e56d","Type":"ContainerDied","Data":"e1e148188ffb70bb3bdc9033e3bae8d623864ee7bce709928efc65d89a59720f"}
Jul 25 02:16:36.298427 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.298402    2576 generic.go:356] "Generic (PLEG): container finished" podID="784af2e7-cb7a-4143-af04-0532440c4dca" containerID="587d379bf2497f40b72889abbfe78d6ee35126b3cbd1755a3a52a6a763bbaf35" exitCode=0
Jul 25 02:16:36.298548 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.298456    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7962-139975443830656" event={"ID":"784af2e7-cb7a-4143-af04-0532440c4dca","Type":"ContainerDied","Data":"587d379bf2497f40b72889abbfe78d6ee35126b3cbd1755a3a52a6a763bbaf35"}
Jul 25 02:16:36.303858 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.303772    2576 scope.go:117] "RemoveContainer" containerID="5c361e968a248f24d9360f098590ebe9515465d887cd03fd017f710d22cd4474"
Jul 25 02:16:36.304075 ip-10-0-134-238 kubenswrapper[2576]: E0725 02:16:36.304046    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c361e968a248f24d9360f098590ebe9515465d887cd03fd017f710d22cd4474\": container with ID starting with 5c361e968a248f24d9360f098590ebe9515465d887cd03fd017f710d22cd4474 not found: ID does not exist" containerID="5c361e968a248f24d9360f098590ebe9515465d887cd03fd017f710d22cd4474"
Jul 25 02:16:36.304168 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.304081    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c361e968a248f24d9360f098590ebe9515465d887cd03fd017f710d22cd4474"} err="failed to get container status \"5c361e968a248f24d9360f098590ebe9515465d887cd03fd017f710d22cd4474\": rpc error: code = NotFound desc = could not find container \"5c361e968a248f24d9360f098590ebe9515465d887cd03fd017f710d22cd4474\": container with ID starting with 5c361e968a248f24d9360f098590ebe9515465d887cd03fd017f710d22cd4474 not found: ID does not exist"
Jul 25 02:16:36.304168 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.304101    2576 scope.go:117] "RemoveContainer" containerID="a94b1c6f9363b54915fbcee993c4b69d0e2d706970433c3ab94ff30980377401"
Jul 25 02:16:36.321939 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.321914    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"]
Jul 25 02:16:36.323316 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.323298    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7e06ec/rate-test-model-7e06ec-kserve-6c94f598f9-gnz9s"]
Jul 25 02:16:36.385076 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.385050    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec085dc4-cd84-4087-8c92-79304107f59f-tls-certs\") pod \"ec085dc4-cd84-4087-8c92-79304107f59f\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") "
Jul 25 02:16:36.385182 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.385098    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-dshm\") pod \"ec085dc4-cd84-4087-8c92-79304107f59f\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") "
Jul 25 02:16:36.385182 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.385118    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-model-cache\") pod \"ec085dc4-cd84-4087-8c92-79304107f59f\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") "
Jul 25 02:16:36.385182 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.385147    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqzhd\" (UniqueName: \"kubernetes.io/projected/ec085dc4-cd84-4087-8c92-79304107f59f-kube-api-access-lqzhd\") pod \"ec085dc4-cd84-4087-8c92-79304107f59f\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") "
Jul 25 02:16:36.385358 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.385184    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-home\") pod \"ec085dc4-cd84-4087-8c92-79304107f59f\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") "
Jul 25 02:16:36.385358 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.385215    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-tmp-dir\") pod \"ec085dc4-cd84-4087-8c92-79304107f59f\" (UID: \"ec085dc4-cd84-4087-8c92-79304107f59f\") "
Jul 25 02:16:36.385460 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.385395    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-model-cache" (OuterVolumeSpecName: "model-cache") pod "ec085dc4-cd84-4087-8c92-79304107f59f" (UID: "ec085dc4-cd84-4087-8c92-79304107f59f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:16:36.385558 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.385541    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-model-cache\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:36.385611 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.385591    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-home" (OuterVolumeSpecName: "home") pod "ec085dc4-cd84-4087-8c92-79304107f59f" (UID: "ec085dc4-cd84-4087-8c92-79304107f59f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:16:36.385647 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.385604    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ec085dc4-cd84-4087-8c92-79304107f59f" (UID: "ec085dc4-cd84-4087-8c92-79304107f59f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:16:36.387316 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.387292    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-dshm" (OuterVolumeSpecName: "dshm") pod "ec085dc4-cd84-4087-8c92-79304107f59f" (UID: "ec085dc4-cd84-4087-8c92-79304107f59f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:16:36.387409 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.387363    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec085dc4-cd84-4087-8c92-79304107f59f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ec085dc4-cd84-4087-8c92-79304107f59f" (UID: "ec085dc4-cd84-4087-8c92-79304107f59f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 02:16:36.387484 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.387462    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec085dc4-cd84-4087-8c92-79304107f59f-kube-api-access-lqzhd" (OuterVolumeSpecName: "kube-api-access-lqzhd") pod "ec085dc4-cd84-4087-8c92-79304107f59f" (UID: "ec085dc4-cd84-4087-8c92-79304107f59f"). InnerVolumeSpecName "kube-api-access-lqzhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:16:36.486159 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.486131    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqzhd\" (UniqueName: \"kubernetes.io/projected/ec085dc4-cd84-4087-8c92-79304107f59f-kube-api-access-lqzhd\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:36.486159 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.486155    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-home\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:36.486503 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.486167    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-tmp-dir\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:36.486503 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.486179    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec085dc4-cd84-4087-8c92-79304107f59f-tls-certs\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:36.486503 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.486190    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec085dc4-cd84-4087-8c92-79304107f59f-dshm\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:36.614644 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.614613    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"]
Jul 25 02:16:36.618424 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.618396    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-575aa7/rate-test-model-575aa7-kserve-5b54f94947-ztvrf"]
Jul 25 02:16:36.999531 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.999493    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3683ed32-398f-4c24-93f8-a38de386e56d" path="/var/lib/kubelet/pods/3683ed32-398f-4c24-93f8-a38de386e56d/volumes"
Jul 25 02:16:36.999916 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:36.999901    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec085dc4-cd84-4087-8c92-79304107f59f" path="/var/lib/kubelet/pods/ec085dc4-cd84-4087-8c92-79304107f59f/volumes"
Jul 25 02:16:37.439409 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.439387    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443830656"
Jul 25 02:16:37.461398 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.461371    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7962-139975443830656_784af2e7-cb7a-4143-af04-0532440c4dca/test-curl-7962-139975443830656/0.log"
Jul 25 02:16:37.483854 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.483827    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7962-139975443830656"]
Jul 25 02:16:37.485587 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.485557    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7962-139975443830656"]
Jul 25 02:16:37.493065 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.493048    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8msj\" (UniqueName: \"kubernetes.io/projected/784af2e7-cb7a-4143-af04-0532440c4dca-kube-api-access-n8msj\") pod \"784af2e7-cb7a-4143-af04-0532440c4dca\" (UID: \"784af2e7-cb7a-4143-af04-0532440c4dca\") "
Jul 25 02:16:37.495110 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.495080    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/784af2e7-cb7a-4143-af04-0532440c4dca-kube-api-access-n8msj" (OuterVolumeSpecName: "kube-api-access-n8msj") pod "784af2e7-cb7a-4143-af04-0532440c4dca" (UID: "784af2e7-cb7a-4143-af04-0532440c4dca"). InnerVolumeSpecName "kube-api-access-n8msj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:16:37.594042 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.593971    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8msj\" (UniqueName: \"kubernetes.io/projected/784af2e7-cb7a-4143-af04-0532440c4dca-kube-api-access-n8msj\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:37.606898 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.606869    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7962-139975443830656"]
Jul 25 02:16:37.607201 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.607188    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="784af2e7-cb7a-4143-af04-0532440c4dca" containerName="test-curl-7962-139975443830656"
Jul 25 02:16:37.607239 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.607202    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="784af2e7-cb7a-4143-af04-0532440c4dca" containerName="test-curl-7962-139975443830656"
Jul 25 02:16:37.607239 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.607211    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec085dc4-cd84-4087-8c92-79304107f59f" containerName="main"
Jul 25 02:16:37.607239 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.607216    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec085dc4-cd84-4087-8c92-79304107f59f" containerName="main"
Jul 25 02:16:37.607239 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.607230    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3683ed32-398f-4c24-93f8-a38de386e56d" containerName="main"
Jul 25 02:16:37.607239 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.607236    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3683ed32-398f-4c24-93f8-a38de386e56d" containerName="main"
Jul 25 02:16:37.607396 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.607288    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="784af2e7-cb7a-4143-af04-0532440c4dca" containerName="test-curl-7962-139975443830656"
Jul 25 02:16:37.607396 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.607297    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3683ed32-398f-4c24-93f8-a38de386e56d" containerName="main"
Jul 25 02:16:37.607396 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.607303    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec085dc4-cd84-4087-8c92-79304107f59f" containerName="main"
Jul 25 02:16:37.610143 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.610127    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443830656"
Jul 25 02:16:37.615871 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.615835    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7962-139975443830656"]
Jul 25 02:16:37.694601 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.694568    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv258\" (UniqueName: \"kubernetes.io/projected/d810f0dd-6294-406f-b4af-88ff65e1c4f6-kube-api-access-fv258\") pod \"test-curl-7962-139975443830656\" (UID: \"d810f0dd-6294-406f-b4af-88ff65e1c4f6\") " pod="opendatahub/test-curl-7962-139975443830656"
Jul 25 02:16:37.795472 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.795447    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv258\" (UniqueName: \"kubernetes.io/projected/d810f0dd-6294-406f-b4af-88ff65e1c4f6-kube-api-access-fv258\") pod \"test-curl-7962-139975443830656\" (UID: \"d810f0dd-6294-406f-b4af-88ff65e1c4f6\") " pod="opendatahub/test-curl-7962-139975443830656"
Jul 25 02:16:37.802474 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.802451    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv258\" (UniqueName: \"kubernetes.io/projected/d810f0dd-6294-406f-b4af-88ff65e1c4f6-kube-api-access-fv258\") pod \"test-curl-7962-139975443830656\" (UID: \"d810f0dd-6294-406f-b4af-88ff65e1c4f6\") " pod="opendatahub/test-curl-7962-139975443830656"
Jul 25 02:16:37.920726 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:37.920703    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443830656"
Jul 25 02:16:38.044120 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:38.044094    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7962-139975443830656"]
Jul 25 02:16:38.046621 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:16:38.046582    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd810f0dd_6294_406f_b4af_88ff65e1c4f6.slice/crio-21ef1743d53b287d8834e4b7ab65ae42f4e956810f3dc7d173c5b5cbcc8dbcdf WatchSource:0}: Error finding container 21ef1743d53b287d8834e4b7ab65ae42f4e956810f3dc7d173c5b5cbcc8dbcdf: Status 404 returned error can't find the container with id 21ef1743d53b287d8834e4b7ab65ae42f4e956810f3dc7d173c5b5cbcc8dbcdf
Jul 25 02:16:38.310805 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:38.310725    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f10393e3dc3f3962962e99ddb8129c4e6be2d566e0c4c21f32cb04c95c72e14f"
Jul 25 02:16:38.310805 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:38.310736    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443830656"
Jul 25 02:16:38.311965 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:38.311939    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7962-139975443830656" event={"ID":"d810f0dd-6294-406f-b4af-88ff65e1c4f6","Type":"ContainerStarted","Data":"21ef1743d53b287d8834e4b7ab65ae42f4e956810f3dc7d173c5b5cbcc8dbcdf"}
Jul 25 02:16:38.322008 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:38.321979    2576 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7962-139975443830656" oldPodUID="784af2e7-cb7a-4143-af04-0532440c4dca" podUID="d810f0dd-6294-406f-b4af-88ff65e1c4f6"
Jul 25 02:16:38.996405 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:38.996375    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="784af2e7-cb7a-4143-af04-0532440c4dca" path="/var/lib/kubelet/pods/784af2e7-cb7a-4143-af04-0532440c4dca/volumes"
Jul 25 02:16:39.318553 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:39.318472    2576 generic.go:356] "Generic (PLEG): container finished" podID="d810f0dd-6294-406f-b4af-88ff65e1c4f6" containerID="7dd4feb9209dc63e39e53cff48c5b29d679adeb301ed2610fe4b68018da4c4d6" exitCode=0
Jul 25 02:16:39.318715 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:39.318561    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7962-139975443830656" event={"ID":"d810f0dd-6294-406f-b4af-88ff65e1c4f6","Type":"ContainerDied","Data":"7dd4feb9209dc63e39e53cff48c5b29d679adeb301ed2610fe4b68018da4c4d6"}
Jul 25 02:16:40.452151 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:40.452123    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443830656"
Jul 25 02:16:40.468336 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:40.468309    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7962-139975443830656_d810f0dd-6294-406f-b4af-88ff65e1c4f6/test-curl-7962-139975443830656/0.log"
Jul 25 02:16:40.493622 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:40.493590    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7962-139975443830656"]
Jul 25 02:16:40.496084 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:40.496062    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7962-139975443830656"]
Jul 25 02:16:40.516178 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:40.516157    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fv258\" (UniqueName: \"kubernetes.io/projected/d810f0dd-6294-406f-b4af-88ff65e1c4f6-kube-api-access-fv258\") pod \"d810f0dd-6294-406f-b4af-88ff65e1c4f6\" (UID: \"d810f0dd-6294-406f-b4af-88ff65e1c4f6\") "
Jul 25 02:16:40.518135 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:40.518112    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d810f0dd-6294-406f-b4af-88ff65e1c4f6-kube-api-access-fv258" (OuterVolumeSpecName: "kube-api-access-fv258") pod "d810f0dd-6294-406f-b4af-88ff65e1c4f6" (UID: "d810f0dd-6294-406f-b4af-88ff65e1c4f6"). InnerVolumeSpecName "kube-api-access-fv258". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:16:40.616835 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:40.616772    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fv258\" (UniqueName: \"kubernetes.io/projected/d810f0dd-6294-406f-b4af-88ff65e1c4f6-kube-api-access-fv258\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:16:40.996061 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:40.996029    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d810f0dd-6294-406f-b4af-88ff65e1c4f6" path="/var/lib/kubelet/pods/d810f0dd-6294-406f-b4af-88ff65e1c4f6/volumes"
Jul 25 02:16:41.328702 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:41.328607    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7962-139975443830656"
Jul 25 02:16:41.328702 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:41.328670    2576 scope.go:117] "RemoveContainer" containerID="7dd4feb9209dc63e39e53cff48c5b29d679adeb301ed2610fe4b68018da4c4d6"
Jul 25 02:16:53.235496 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:16:53.235460    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:17:00.426842 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.426805    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"]
Jul 25 02:17:00.427209 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.427117    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d810f0dd-6294-406f-b4af-88ff65e1c4f6" containerName="test-curl-7962-139975443830656"
Jul 25 02:17:00.427209 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.427127    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d810f0dd-6294-406f-b4af-88ff65e1c4f6" containerName="test-curl-7962-139975443830656"
Jul 25 02:17:00.427209 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.427183    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d810f0dd-6294-406f-b4af-88ff65e1c4f6" containerName="test-curl-7962-139975443830656"
Jul 25 02:17:00.432777 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.432754    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.434848 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.434823    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-dc8abe-openshift-default-dockercfg-79sqn\""
Jul 25 02:17:00.434958 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.434866    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 25 02:17:00.442125 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.442101    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"]
Jul 25 02:17:00.580153 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.580117    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-envoy\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.580348 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.580168    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-credential-socket\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.580348 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.580198    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-token\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.580348 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.580237    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-workload-socket\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.580348 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.580257    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-podinfo\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.580348 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.580297    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-data\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.580571 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.580352    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-workload-certs\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.580571 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.580373    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istiod-ca-cert\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.580571 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.580421    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h69cj\" (UniqueName: \"kubernetes.io/projected/2b9098b5-2e5d-4db3-a7f8-075369e47b59-kube-api-access-h69cj\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.681570 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.681477    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-workload-socket\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.681570 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.681513    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-podinfo\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.681570 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.681532    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-data\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.681570 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.681572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-workload-certs\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.681939 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.681589    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istiod-ca-cert\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.681939 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.681609    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h69cj\" (UniqueName: \"kubernetes.io/projected/2b9098b5-2e5d-4db3-a7f8-075369e47b59-kube-api-access-h69cj\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.681939 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.681675    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-envoy\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.681939 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.681713    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-credential-socket\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.681939 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.681741    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-token\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.682166 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.681962    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-workload-socket\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.682166 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.682069    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-data\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.682166 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.682095    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-workload-certs\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.682166 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.682159    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-credential-socket\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.682401 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.682372    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istiod-ca-cert\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.684135 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.684111    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-envoy\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.684405 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.684389    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-podinfo\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.695262 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.695238    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-token\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.695548 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.695531    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h69cj\" (UniqueName: \"kubernetes.io/projected/2b9098b5-2e5d-4db3-a7f8-075369e47b59-kube-api-access-h69cj\") pod \"e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") " pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.742981 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.742957    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:00.885538 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:00.885473    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"]
Jul 25 02:17:00.887766 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:17:00.887742    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b9098b5_2e5d_4db3_a7f8_075369e47b59.slice/crio-b0317fb8673f96b40cb1231b901be27773f4f59c994acc1ea7f89c554a9734e3 WatchSource:0}: Error finding container b0317fb8673f96b40cb1231b901be27773f4f59c994acc1ea7f89c554a9734e3: Status 404 returned error can't find the container with id b0317fb8673f96b40cb1231b901be27773f4f59c994acc1ea7f89c554a9734e3
Jul 25 02:17:01.424795 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:01.424762    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh" event={"ID":"2b9098b5-2e5d-4db3-a7f8-075369e47b59","Type":"ContainerStarted","Data":"b0317fb8673f96b40cb1231b901be27773f4f59c994acc1ea7f89c554a9734e3"}
Jul 25 02:17:03.677465 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:03.677420    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 02:17:03.677761 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:03.677503    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 02:17:03.677761 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:03.677537    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 02:17:04.443474 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:04.443429    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh" event={"ID":"2b9098b5-2e5d-4db3-a7f8-075369e47b59","Type":"ContainerStarted","Data":"ea6a8c41b0faae6b33c5e58e1fb9b0e92be191823153c4ce0c645805737f75d5"}
Jul 25 02:17:04.461115 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:04.461068    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh" podStartSLOduration=1.673539847 podStartE2EDuration="4.461049476s" totalImagesPullingTime="2.787509629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 02:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:17:04.460819276 +0000 UTC m=+3975.948115951" watchObservedRunningTime="2026-07-25 02:17:04.461049476 +0000 UTC m=+3975.948346152"
Jul 25 02:17:04.743774 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:04.743689    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:04.748610 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:04.748581    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:05.448320 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:05.448284    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:05.449323 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:05.449305    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:12.132479 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:12.132444    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:17:20.449826 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:20.449785    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 25 02:17:20.450314 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:20.449858    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh" podUID="2b9098b5-2e5d-4db3-a7f8-075369e47b59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 25 02:17:20.520925 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:20.520891    2576 generic.go:356] "Generic (PLEG): container finished" podID="2b9098b5-2e5d-4db3-a7f8-075369e47b59" containerID="ea6a8c41b0faae6b33c5e58e1fb9b0e92be191823153c4ce0c645805737f75d5" exitCode=0
Jul 25 02:17:20.520925 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:20.520927    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh" event={"ID":"2b9098b5-2e5d-4db3-a7f8-075369e47b59","Type":"ContainerDied","Data":"ea6a8c41b0faae6b33c5e58e1fb9b0e92be191823153c4ce0c645805737f75d5"}
Jul 25 02:17:20.521295 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:20.521281    2576 scope.go:117] "RemoveContainer" containerID="ea6a8c41b0faae6b33c5e58e1fb9b0e92be191823153c4ce0c645805737f75d5"
Jul 25 02:17:20.522339 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:20.522309    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 02:17:20.522435 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:20.522387    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 02:17:20.522500 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:20.522438    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 02:17:21.527220 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:21.527189    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh" event={"ID":"2b9098b5-2e5d-4db3-a7f8-075369e47b59","Type":"ContainerStarted","Data":"d3477a501757232fd568a5ef656306787a77352b369b1df2cd9be771451748bd"}
Jul 25 02:17:21.743858 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:21.743506    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:21.744062 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:21.743878    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 25 02:17:21.744062 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:21.743932    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh" podUID="2b9098b5-2e5d-4db3-a7f8-075369e47b59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 25 02:17:22.743889 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:22.743848    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 25 02:17:22.744387 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:22.743921    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh" podUID="2b9098b5-2e5d-4db3-a7f8-075369e47b59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 25 02:17:23.744005 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:23.743968    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 25 02:17:23.744624 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:23.744587    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh" podUID="2b9098b5-2e5d-4db3-a7f8-075369e47b59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 25 02:17:24.744264 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:24.744224    2576 patch_prober.go:28] interesting pod/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 25 02:17:24.745010 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:24.744973    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh" podUID="2b9098b5-2e5d-4db3-a7f8-075369e47b59" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 25 02:17:25.748070 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:25.748040    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:25.748495 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:25.748327    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:25.748979 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:25.748958    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:17:45.833619 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:45.833582    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"]
Jul 25 02:17:45.836809 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:45.836791    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:45.840648 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:45.840623    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-67eb46\"/\"default-dockercfg-l9ws5\""
Jul 25 02:17:45.841214 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:45.841195    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-67eb46\"/\"kube-root-ca.crt\""
Jul 25 02:17:45.841331 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:45.841255    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-67eb46\"/\"openshift-service-ca.crt\""
Jul 25 02:17:45.841331 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:45.841312    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-67eb46\"/\"ipp-route-67eb46-kserve-self-signed-certs\""
Jul 25 02:17:45.851200 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:45.851175    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"]
Jul 25 02:17:45.954959 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:45.954921    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-model-cache\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:45.954959 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:45.954964    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b79567c3-b601-48a8-95ca-e2c9488634a0-tls-certs\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:45.955174 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:45.954997    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-tmp-dir\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:45.955174 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:45.955069    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7kvf\" (UniqueName: \"kubernetes.io/projected/b79567c3-b601-48a8-95ca-e2c9488634a0-kube-api-access-d7kvf\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:45.955174 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:45.955104    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-home\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:45.955174 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:45.955123    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-dshm\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:46.056410 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:46.056373    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7kvf\" (UniqueName: \"kubernetes.io/projected/b79567c3-b601-48a8-95ca-e2c9488634a0-kube-api-access-d7kvf\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:46.056591 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:46.056417    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-home\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:46.056591 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:46.056439    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-dshm\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:46.056591 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:46.056509    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-model-cache\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:46.056591 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:46.056557    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b79567c3-b601-48a8-95ca-e2c9488634a0-tls-certs\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:46.056851 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:46.056619    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-tmp-dir\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:46.056912 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:46.056898    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-home\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:46.057006 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:46.056982    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-model-cache\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:46.057070 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:46.057016    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-tmp-dir\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:46.058815 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:46.058792    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-dshm\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:46.059075 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:46.059058    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b79567c3-b601-48a8-95ca-e2c9488634a0-tls-certs\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:46.064685 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:46.064648    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7kvf\" (UniqueName: \"kubernetes.io/projected/b79567c3-b601-48a8-95ca-e2c9488634a0-kube-api-access-d7kvf\") pod \"ipp-route-67eb46-kserve-77c4f78467-6r7qm\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") " pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:46.147042 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:46.146956    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:46.279933 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:46.279910    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"]
Jul 25 02:17:46.282725 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:17:46.282698    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb79567c3_b601_48a8_95ca_e2c9488634a0.slice/crio-bdbd4a44018270b72f785ddcc70d0bbda74ce488e68c1087b0caa922addc771d WatchSource:0}: Error finding container bdbd4a44018270b72f785ddcc70d0bbda74ce488e68c1087b0caa922addc771d: Status 404 returned error can't find the container with id bdbd4a44018270b72f785ddcc70d0bbda74ce488e68c1087b0caa922addc771d
Jul 25 02:17:46.649446 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:46.649409    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm" event={"ID":"b79567c3-b601-48a8-95ca-e2c9488634a0","Type":"ContainerStarted","Data":"6f6e383ba727c3de23e96b33f7b98a7ef0e968be1a0b0e55e7181d49f60859a0"}
Jul 25 02:17:46.649446 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:46.649452    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm" event={"ID":"b79567c3-b601-48a8-95ca-e2c9488634a0","Type":"ContainerStarted","Data":"bdbd4a44018270b72f785ddcc70d0bbda74ce488e68c1087b0caa922addc771d"}
Jul 25 02:17:46.664320 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:46.664274    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm" podStartSLOduration=1.664259889 podStartE2EDuration="1.664259889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 02:17:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:17:46.662871509 +0000 UTC m=+4018.150168183" watchObservedRunningTime="2026-07-25 02:17:46.664259889 +0000 UTC m=+4018.151556564"
Jul 25 02:17:49.627546 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:49.627506    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:17:56.147849 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:56.147803    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:56.147849 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:56.147858    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:56.160241 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:56.160217    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:17:56.705890 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:17:56.705862    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:18:18.023052 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:18.022957    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:18:29.125210 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.125176    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"]
Jul 25 02:18:29.128701 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.128682    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.130722 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.130700    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-389518-openshift-default-dockercfg-69t97\""
Jul 25 02:18:29.138684 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.138640    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"]
Jul 25 02:18:29.206559 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.206526    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-podinfo\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.206559 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.206561    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-data\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.206782 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.206607    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istiod-ca-cert\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.206782 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.206625    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-credential-socket\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.206782 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.206641    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-workload-socket\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.206782 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.206681    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf46r\" (UniqueName: \"kubernetes.io/projected/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-kube-api-access-sf46r\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.206782 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.206711    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-token\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.206782 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.206730    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-workload-certs\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.206782 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.206751    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-envoy\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.307711 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.307682    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istiod-ca-cert\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.307711 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.307714    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-credential-socket\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.307946 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.307732    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-workload-socket\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.307946 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.307748    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf46r\" (UniqueName: \"kubernetes.io/projected/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-kube-api-access-sf46r\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.307946 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.307769    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-token\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.307946 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.307787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-workload-certs\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.307946 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.307825    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-envoy\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.307946 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.307871    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-podinfo\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.307946 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.307900    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-data\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.308274 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.308160    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-credential-socket\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.308274 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.308204    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-workload-certs\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.308274 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.308263    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-workload-socket\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.308417 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.308287    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-data\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.308575 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.308550    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istiod-ca-cert\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.310266 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.310240    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-envoy\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.310476 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.310459    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-podinfo\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.314755 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.314731    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-token\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.314879 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.314861    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf46r\" (UniqueName: \"kubernetes.io/projected/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-kube-api-access-sf46r\") pod \"e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") " pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.440046 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.440019    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:29.573799 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.573776    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"]
Jul 25 02:18:29.576140 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:18:29.576105    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f9be38c_c6a6_4849_b8e9_e74d86f4d9a7.slice/crio-7ab799e95b691e8dcdb15fe57700d8560f9b793ebc0747bdf3186181040e2821 WatchSource:0}: Error finding container 7ab799e95b691e8dcdb15fe57700d8560f9b793ebc0747bdf3186181040e2821: Status 404 returned error can't find the container with id 7ab799e95b691e8dcdb15fe57700d8560f9b793ebc0747bdf3186181040e2821
Jul 25 02:18:29.578256 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.578222    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 02:18:29.578357 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.578298    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 02:18:29.578357 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.578341    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 02:18:29.854276 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.854197    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7" event={"ID":"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7","Type":"ContainerStarted","Data":"9c2f6b3299ff677b4ed1953556195822c2c51709aeac79dc4967988d0e6babd7"}
Jul 25 02:18:29.854276 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.854255    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7" event={"ID":"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7","Type":"ContainerStarted","Data":"7ab799e95b691e8dcdb15fe57700d8560f9b793ebc0747bdf3186181040e2821"}
Jul 25 02:18:29.871824 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:29.871780    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7" podStartSLOduration=0.871764794 podStartE2EDuration="871.764794ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 02:18:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:18:29.870125508 +0000 UTC m=+4061.357422183" watchObservedRunningTime="2026-07-25 02:18:29.871764794 +0000 UTC m=+4061.359061469"
Jul 25 02:18:30.440780 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:30.440750    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:30.445842 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:30.445819    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:30.859536 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:30.859451    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:30.860358 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:30.860337    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:36.938605 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:36.938570    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:18:45.860558 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:45.860520    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 25 02:18:45.861007 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:45.860586    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7" podUID="1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 25 02:18:45.938854 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:45.938823    2576 generic.go:356] "Generic (PLEG): container finished" podID="1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" containerID="9c2f6b3299ff677b4ed1953556195822c2c51709aeac79dc4967988d0e6babd7" exitCode=0
Jul 25 02:18:45.939037 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:45.938898    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7" event={"ID":"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7","Type":"ContainerDied","Data":"9c2f6b3299ff677b4ed1953556195822c2c51709aeac79dc4967988d0e6babd7"}
Jul 25 02:18:45.939344 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:45.939328    2576 scope.go:117] "RemoveContainer" containerID="9c2f6b3299ff677b4ed1953556195822c2c51709aeac79dc4967988d0e6babd7"
Jul 25 02:18:45.940479 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:45.940449    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 02:18:45.940536 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:45.940516    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 02:18:45.940574 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:45.940547    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 25 02:18:46.949248 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:46.949200    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7" event={"ID":"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7","Type":"ContainerStarted","Data":"736b32198096cebdb627dc4398d02edd0bb43360d350d85af1425e2b04fe38eb"}
Jul 25 02:18:47.440771 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:47.440740    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:47.441174 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:47.441148    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 25 02:18:47.441356 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:47.441327    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7" podUID="1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 25 02:18:48.440946 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:48.440907    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 25 02:18:48.441429 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:48.440971    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7" podUID="1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 25 02:18:49.440902 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:49.440856    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 25 02:18:49.441212 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:49.440943    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7" podUID="1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 25 02:18:50.440877 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:50.440841    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 25 02:18:50.441171 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:50.441142    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7" podUID="1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 25 02:18:51.444935 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:51.444896    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:51.445386 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:51.445165    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:51.445936 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:51.445919    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:18:52.413178 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:52.413142    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2"]
Jul 25 02:18:52.416758 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:52.416740    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2"
Jul 25 02:18:52.418624 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:52.418597    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 02:18:52.418624 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:52.418602    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cjsrv\""
Jul 25 02:18:52.421524 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:52.421505    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2"]
Jul 25 02:18:52.517283 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:52.517235    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aaf0124-4f25-4417-8e85-373832772ec3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2\" (UID: \"4aaf0124-4f25-4417-8e85-373832772ec3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2"
Jul 25 02:18:52.618597 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:52.618563    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aaf0124-4f25-4417-8e85-373832772ec3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2\" (UID: \"4aaf0124-4f25-4417-8e85-373832772ec3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2"
Jul 25 02:18:52.619230 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:52.619208    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aaf0124-4f25-4417-8e85-373832772ec3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2\" (UID: \"4aaf0124-4f25-4417-8e85-373832772ec3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2"
Jul 25 02:18:52.728569 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:52.728487    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2"
Jul 25 02:18:52.857602 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:52.857513    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2"]
Jul 25 02:18:52.859971 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:18:52.859944    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aaf0124_4f25_4417_8e85_373832772ec3.slice/crio-78ada3baaacc10023b7f439cf9ed0eba1263dda1485d9ad802262bccf93f70a6 WatchSource:0}: Error finding container 78ada3baaacc10023b7f439cf9ed0eba1263dda1485d9ad802262bccf93f70a6: Status 404 returned error can't find the container with id 78ada3baaacc10023b7f439cf9ed0eba1263dda1485d9ad802262bccf93f70a6
Jul 25 02:18:52.984899 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:52.984808    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2" event={"ID":"4aaf0124-4f25-4417-8e85-373832772ec3","Type":"ContainerStarted","Data":"78ada3baaacc10023b7f439cf9ed0eba1263dda1485d9ad802262bccf93f70a6"}
Jul 25 02:18:53.991030 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:53.990991    2576 generic.go:356] "Generic (PLEG): container finished" podID="4aaf0124-4f25-4417-8e85-373832772ec3" containerID="a6bba2778436648402f353b945c1d5dccd0457dac2f63140217f64f991fc7b1b" exitCode=0
Jul 25 02:18:53.991480 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:53.991043    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2" event={"ID":"4aaf0124-4f25-4417-8e85-373832772ec3","Type":"ContainerDied","Data":"a6bba2778436648402f353b945c1d5dccd0457dac2f63140217f64f991fc7b1b"}
Jul 25 02:18:55.132066 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:55.132042    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2"
Jul 25 02:18:55.242982 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:55.242947    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aaf0124-4f25-4417-8e85-373832772ec3-maas-api-service-ca\") pod \"4aaf0124-4f25-4417-8e85-373832772ec3\" (UID: \"4aaf0124-4f25-4417-8e85-373832772ec3\") "
Jul 25 02:18:55.243339 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:55.243318    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aaf0124-4f25-4417-8e85-373832772ec3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4aaf0124-4f25-4417-8e85-373832772ec3" (UID: "4aaf0124-4f25-4417-8e85-373832772ec3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 02:18:55.343992 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:55.343909    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aaf0124-4f25-4417-8e85-373832772ec3-maas-api-service-ca\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:18:56.002302 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:56.002269    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2"
Jul 25 02:18:56.002503 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:56.002310    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2" event={"ID":"4aaf0124-4f25-4417-8e85-373832772ec3","Type":"ContainerDied","Data":"78ada3baaacc10023b7f439cf9ed0eba1263dda1485d9ad802262bccf93f70a6"}
Jul 25 02:18:56.002503 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:18:56.002354    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78ada3baaacc10023b7f439cf9ed0eba1263dda1485d9ad802262bccf93f70a6"
Jul 25 02:19:07.554982 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:07.554948    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2"]
Jul 25 02:19:07.559051 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:07.559026    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-389518-xhpj2"]
Jul 25 02:19:08.322546 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:08.322512    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:19:08.996621 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:08.996582    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4aaf0124-4f25-4417-8e85-373832772ec3" path="/var/lib/kubelet/pods/4aaf0124-4f25-4417-8e85-373832772ec3/volumes"
Jul 25 02:19:14.079940 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:14.079904    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"]
Jul 25 02:19:14.080418 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:14.080160    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7" podUID="1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" containerName="istio-proxy" containerID="cri-o://736b32198096cebdb627dc4398d02edd0bb43360d350d85af1425e2b04fe38eb" gracePeriod=30
Jul 25 02:19:17.722543 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:17.722506    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj"]
Jul 25 02:19:17.722950 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:17.722847    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4aaf0124-4f25-4417-8e85-373832772ec3" containerName="revoke-keys"
Jul 25 02:19:17.722950 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:17.722859    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4aaf0124-4f25-4417-8e85-373832772ec3" containerName="revoke-keys"
Jul 25 02:19:17.722950 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:17.722924    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4aaf0124-4f25-4417-8e85-373832772ec3" containerName="revoke-keys"
Jul 25 02:19:17.726057 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:17.726040    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj"
Jul 25 02:19:17.727916 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:17.727890    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cjsrv\""
Jul 25 02:19:17.728060 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:17.727890    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 02:19:17.732757 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:17.732732    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj"]
Jul 25 02:19:17.836599 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:17.836571    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48ee41a1-a0f0-462a-80bd-fe9449bfeec1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj\" (UID: \"48ee41a1-a0f0-462a-80bd-fe9449bfeec1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj"
Jul 25 02:19:17.937136 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:17.937105    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48ee41a1-a0f0-462a-80bd-fe9449bfeec1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj\" (UID: \"48ee41a1-a0f0-462a-80bd-fe9449bfeec1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj"
Jul 25 02:19:17.937755 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:17.937736    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48ee41a1-a0f0-462a-80bd-fe9449bfeec1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj\" (UID: \"48ee41a1-a0f0-462a-80bd-fe9449bfeec1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj"
Jul 25 02:19:18.037449 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:18.037380    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj"
Jul 25 02:19:18.164135 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:18.164105    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj"]
Jul 25 02:19:18.167436 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:19:18.167410    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48ee41a1_a0f0_462a_80bd_fe9449bfeec1.slice/crio-9904c1ee0c60c24d09a3e56635508d4a6c84badcd1c6ee9032fcdb797519c830 WatchSource:0}: Error finding container 9904c1ee0c60c24d09a3e56635508d4a6c84badcd1c6ee9032fcdb797519c830: Status 404 returned error can't find the container with id 9904c1ee0c60c24d09a3e56635508d4a6c84badcd1c6ee9032fcdb797519c830
Jul 25 02:19:19.112483 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.112457    2576 generic.go:356] "Generic (PLEG): container finished" podID="48ee41a1-a0f0-462a-80bd-fe9449bfeec1" containerID="8962fce7b1c131800d5a9c14571c1b073433366daa8d79c2d6fdb7a368320582" exitCode=0
Jul 25 02:19:19.112818 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.112540    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj" event={"ID":"48ee41a1-a0f0-462a-80bd-fe9449bfeec1","Type":"ContainerDied","Data":"8962fce7b1c131800d5a9c14571c1b073433366daa8d79c2d6fdb7a368320582"}
Jul 25 02:19:19.112818 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.112592    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj" event={"ID":"48ee41a1-a0f0-462a-80bd-fe9449bfeec1","Type":"ContainerStarted","Data":"9904c1ee0c60c24d09a3e56635508d4a6c84badcd1c6ee9032fcdb797519c830"}
Jul 25 02:19:19.333819 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.333795    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:19:19.350109 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.350056    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-credential-socket\") pod \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") "
Jul 25 02:19:19.350109 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.350101    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-token\") pod \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") "
Jul 25 02:19:19.350290 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.350118    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-workload-certs\") pod \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") "
Jul 25 02:19:19.350290 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.350138    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-envoy\") pod \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") "
Jul 25 02:19:19.350290 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.350174    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sf46r\" (UniqueName: \"kubernetes.io/projected/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-kube-api-access-sf46r\") pod \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") "
Jul 25 02:19:19.350290 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.350206    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-data\") pod \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") "
Jul 25 02:19:19.350290 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.350226    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istiod-ca-cert\") pod \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") "
Jul 25 02:19:19.350290 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.350251    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-podinfo\") pod \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") "
Jul 25 02:19:19.350290 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.350287    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-workload-socket\") pod \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\" (UID: \"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7\") "
Jul 25 02:19:19.350675 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.350358    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" (UID: "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:19:19.350675 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.350394    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" (UID: "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:19:19.350675 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.350600    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" (UID: "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:19:19.350675 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.350617    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-credential-socket\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:19:19.350675 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.350641    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-workload-certs\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:19:19.350929 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.350618    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" (UID: "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 02:19:19.351475 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.351447    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-data" (OuterVolumeSpecName: "istio-data") pod "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" (UID: "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:19:19.352529 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.352500    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" (UID: "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 25 02:19:19.352637 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.352504    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-token" (OuterVolumeSpecName: "istio-token") pod "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" (UID: "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:19:19.352763 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.352745    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" (UID: "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:19:19.352826 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.352806    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-kube-api-access-sf46r" (OuterVolumeSpecName: "kube-api-access-sf46r") pod "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" (UID: "1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7"). InnerVolumeSpecName "kube-api-access-sf46r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:19:19.451584 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.451551    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sf46r\" (UniqueName: \"kubernetes.io/projected/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-kube-api-access-sf46r\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:19:19.451584 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.451580    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-data\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:19:19.451818 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.451595    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istiod-ca-cert\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:19:19.451818 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.451606    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-podinfo\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:19:19.451818 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.451618    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-workload-socket\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:19:19.451818 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.451630    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-token\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:19:19.451818 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:19.451641    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7-istio-envoy\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:19:20.120424 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:20.120390    2576 generic.go:356] "Generic (PLEG): container finished" podID="1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" containerID="736b32198096cebdb627dc4398d02edd0bb43360d350d85af1425e2b04fe38eb" exitCode=0
Jul 25 02:19:20.121018 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:20.120453    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7" event={"ID":"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7","Type":"ContainerDied","Data":"736b32198096cebdb627dc4398d02edd0bb43360d350d85af1425e2b04fe38eb"}
Jul 25 02:19:20.121018 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:20.120477    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"
Jul 25 02:19:20.121018 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:20.120522    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7" event={"ID":"1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7","Type":"ContainerDied","Data":"7ab799e95b691e8dcdb15fe57700d8560f9b793ebc0747bdf3186181040e2821"}
Jul 25 02:19:20.121018 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:20.120543    2576 scope.go:117] "RemoveContainer" containerID="736b32198096cebdb627dc4398d02edd0bb43360d350d85af1425e2b04fe38eb"
Jul 25 02:19:20.135021 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:20.134997    2576 scope.go:117] "RemoveContainer" containerID="9c2f6b3299ff677b4ed1953556195822c2c51709aeac79dc4967988d0e6babd7"
Jul 25 02:19:20.140877 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:20.140844    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"]
Jul 25 02:19:20.144465 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:20.144437    2576 scope.go:117] "RemoveContainer" containerID="736b32198096cebdb627dc4398d02edd0bb43360d350d85af1425e2b04fe38eb"
Jul 25 02:19:20.144856 ip-10-0-134-238 kubenswrapper[2576]: E0725 02:19:20.144829    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"736b32198096cebdb627dc4398d02edd0bb43360d350d85af1425e2b04fe38eb\": container with ID starting with 736b32198096cebdb627dc4398d02edd0bb43360d350d85af1425e2b04fe38eb not found: ID does not exist" containerID="736b32198096cebdb627dc4398d02edd0bb43360d350d85af1425e2b04fe38eb"
Jul 25 02:19:20.144964 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:20.144866    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"736b32198096cebdb627dc4398d02edd0bb43360d350d85af1425e2b04fe38eb"} err="failed to get container status \"736b32198096cebdb627dc4398d02edd0bb43360d350d85af1425e2b04fe38eb\": rpc error: code = NotFound desc = could not find container \"736b32198096cebdb627dc4398d02edd0bb43360d350d85af1425e2b04fe38eb\": container with ID starting with 736b32198096cebdb627dc4398d02edd0bb43360d350d85af1425e2b04fe38eb not found: ID does not exist"
Jul 25 02:19:20.144964 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:20.144889    2576 scope.go:117] "RemoveContainer" containerID="9c2f6b3299ff677b4ed1953556195822c2c51709aeac79dc4967988d0e6babd7"
Jul 25 02:19:20.145181 ip-10-0-134-238 kubenswrapper[2576]: E0725 02:19:20.145162    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c2f6b3299ff677b4ed1953556195822c2c51709aeac79dc4967988d0e6babd7\": container with ID starting with 9c2f6b3299ff677b4ed1953556195822c2c51709aeac79dc4967988d0e6babd7 not found: ID does not exist" containerID="9c2f6b3299ff677b4ed1953556195822c2c51709aeac79dc4967988d0e6babd7"
Jul 25 02:19:20.145245 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:20.145186    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c2f6b3299ff677b4ed1953556195822c2c51709aeac79dc4967988d0e6babd7"} err="failed to get container status \"9c2f6b3299ff677b4ed1953556195822c2c51709aeac79dc4967988d0e6babd7\": rpc error: code = NotFound desc = could not find container \"9c2f6b3299ff677b4ed1953556195822c2c51709aeac79dc4967988d0e6babd7\": container with ID starting with 9c2f6b3299ff677b4ed1953556195822c2c51709aeac79dc4967988d0e6babd7 not found: ID does not exist"
Jul 25 02:19:20.145499 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:20.145478    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-389518-openshift-default-66cbf9464d-gdkk7"]
Jul 25 02:19:20.254626 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:20.254605    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj"
Jul 25 02:19:20.358987 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:20.358952    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48ee41a1-a0f0-462a-80bd-fe9449bfeec1-maas-api-service-ca\") pod \"48ee41a1-a0f0-462a-80bd-fe9449bfeec1\" (UID: \"48ee41a1-a0f0-462a-80bd-fe9449bfeec1\") "
Jul 25 02:19:20.359302 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:20.359278    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48ee41a1-a0f0-462a-80bd-fe9449bfeec1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "48ee41a1-a0f0-462a-80bd-fe9449bfeec1" (UID: "48ee41a1-a0f0-462a-80bd-fe9449bfeec1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 02:19:20.460051 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:20.460013    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48ee41a1-a0f0-462a-80bd-fe9449bfeec1-maas-api-service-ca\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:19:20.996002 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:20.995967    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" path="/var/lib/kubelet/pods/1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7/volumes"
Jul 25 02:19:21.126430 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:21.126397    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj"
Jul 25 02:19:21.126908 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:21.126450    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj" event={"ID":"48ee41a1-a0f0-462a-80bd-fe9449bfeec1","Type":"ContainerDied","Data":"9904c1ee0c60c24d09a3e56635508d4a6c84badcd1c6ee9032fcdb797519c830"}
Jul 25 02:19:21.126908 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:21.126510    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9904c1ee0c60c24d09a3e56635508d4a6c84badcd1c6ee9032fcdb797519c830"
Jul 25 02:19:32.925072 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:32.925034    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:19:37.798463 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:37.798427    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj"]
Jul 25 02:19:37.800234 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:37.800211    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-67eb46-crddj"]
Jul 25 02:19:38.997592 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:38.997561    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48ee41a1-a0f0-462a-80bd-fe9449bfeec1" path="/var/lib/kubelet/pods/48ee41a1-a0f0-462a-80bd-fe9449bfeec1/volumes"
Jul 25 02:19:43.172098 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:43.172012    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"]
Jul 25 02:19:45.727240 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:45.727201    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:19:58.191387 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.191323    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm" podUID="b79567c3-b601-48a8-95ca-e2c9488634a0" containerName="main" containerID="cri-o://6f6e383ba727c3de23e96b33f7b98a7ef0e968be1a0b0e55e7181d49f60859a0" gracePeriod=45
Jul 25 02:19:58.310913 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.310887    2576 generic.go:356] "Generic (PLEG): container finished" podID="b79567c3-b601-48a8-95ca-e2c9488634a0" containerID="6f6e383ba727c3de23e96b33f7b98a7ef0e968be1a0b0e55e7181d49f60859a0" exitCode=0
Jul 25 02:19:58.311038 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.310943    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm" event={"ID":"b79567c3-b601-48a8-95ca-e2c9488634a0","Type":"ContainerDied","Data":"6f6e383ba727c3de23e96b33f7b98a7ef0e968be1a0b0e55e7181d49f60859a0"}
Jul 25 02:19:58.443314 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.443261    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:19:58.574924 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.574891    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b79567c3-b601-48a8-95ca-e2c9488634a0-tls-certs\") pod \"b79567c3-b601-48a8-95ca-e2c9488634a0\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") "
Jul 25 02:19:58.575090 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.574965    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7kvf\" (UniqueName: \"kubernetes.io/projected/b79567c3-b601-48a8-95ca-e2c9488634a0-kube-api-access-d7kvf\") pod \"b79567c3-b601-48a8-95ca-e2c9488634a0\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") "
Jul 25 02:19:58.575140 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.575121    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-model-cache\") pod \"b79567c3-b601-48a8-95ca-e2c9488634a0\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") "
Jul 25 02:19:58.575190 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.575156    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-home\") pod \"b79567c3-b601-48a8-95ca-e2c9488634a0\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") "
Jul 25 02:19:58.575190 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.575181    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-tmp-dir\") pod \"b79567c3-b601-48a8-95ca-e2c9488634a0\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") "
Jul 25 02:19:58.575303 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.575196    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-dshm\") pod \"b79567c3-b601-48a8-95ca-e2c9488634a0\" (UID: \"b79567c3-b601-48a8-95ca-e2c9488634a0\") "
Jul 25 02:19:58.575415 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.575388    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-model-cache" (OuterVolumeSpecName: "model-cache") pod "b79567c3-b601-48a8-95ca-e2c9488634a0" (UID: "b79567c3-b601-48a8-95ca-e2c9488634a0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:19:58.575566 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.575548    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-model-cache\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:19:58.575639 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.575593    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b79567c3-b601-48a8-95ca-e2c9488634a0" (UID: "b79567c3-b601-48a8-95ca-e2c9488634a0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:19:58.575721 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.575675    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-home" (OuterVolumeSpecName: "home") pod "b79567c3-b601-48a8-95ca-e2c9488634a0" (UID: "b79567c3-b601-48a8-95ca-e2c9488634a0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:19:58.577182 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.577158    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b79567c3-b601-48a8-95ca-e2c9488634a0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b79567c3-b601-48a8-95ca-e2c9488634a0" (UID: "b79567c3-b601-48a8-95ca-e2c9488634a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 02:19:58.577424 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.577400    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-dshm" (OuterVolumeSpecName: "dshm") pod "b79567c3-b601-48a8-95ca-e2c9488634a0" (UID: "b79567c3-b601-48a8-95ca-e2c9488634a0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:19:58.577494 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.577421    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b79567c3-b601-48a8-95ca-e2c9488634a0-kube-api-access-d7kvf" (OuterVolumeSpecName: "kube-api-access-d7kvf") pod "b79567c3-b601-48a8-95ca-e2c9488634a0" (UID: "b79567c3-b601-48a8-95ca-e2c9488634a0"). InnerVolumeSpecName "kube-api-access-d7kvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:19:58.676712 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.676676    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b79567c3-b601-48a8-95ca-e2c9488634a0-tls-certs\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:19:58.676712 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.676708    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7kvf\" (UniqueName: \"kubernetes.io/projected/b79567c3-b601-48a8-95ca-e2c9488634a0-kube-api-access-d7kvf\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:19:58.676712 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.676720    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-home\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:19:58.676959 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.676733    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-tmp-dir\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:19:58.676959 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:58.676743    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b79567c3-b601-48a8-95ca-e2c9488634a0-dshm\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:19:59.317433 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:59.317348    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"
Jul 25 02:19:59.317433 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:59.317354    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm" event={"ID":"b79567c3-b601-48a8-95ca-e2c9488634a0","Type":"ContainerDied","Data":"bdbd4a44018270b72f785ddcc70d0bbda74ce488e68c1087b0caa922addc771d"}
Jul 25 02:19:59.317984 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:59.317465    2576 scope.go:117] "RemoveContainer" containerID="6f6e383ba727c3de23e96b33f7b98a7ef0e968be1a0b0e55e7181d49f60859a0"
Jul 25 02:19:59.331612 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:59.331592    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"]
Jul 25 02:19:59.335275 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:19:59.335248    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-67eb46/ipp-route-67eb46-kserve-77c4f78467-6r7qm"]
Jul 25 02:20:00.995897 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:00.995865    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b79567c3-b601-48a8-95ca-e2c9488634a0" path="/var/lib/kubelet/pods/b79567c3-b601-48a8-95ca-e2c9488634a0/volumes"
Jul 25 02:20:49.287303 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:49.287274    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 02:20:49.293220 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:49.293199    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 02:20:49.296052 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:49.296032    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 02:20:49.302047 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:49.302027    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 02:20:50.985902 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:50.985846    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv"]
Jul 25 02:20:50.986322 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:50.986198    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48ee41a1-a0f0-462a-80bd-fe9449bfeec1" containerName="revoke-keys"
Jul 25 02:20:50.986322 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:50.986210    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="48ee41a1-a0f0-462a-80bd-fe9449bfeec1" containerName="revoke-keys"
Jul 25 02:20:50.986322 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:50.986224    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b79567c3-b601-48a8-95ca-e2c9488634a0" containerName="main"
Jul 25 02:20:50.986322 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:50.986229    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b79567c3-b601-48a8-95ca-e2c9488634a0" containerName="main"
Jul 25 02:20:50.986322 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:50.986239    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" containerName="istio-proxy"
Jul 25 02:20:50.986322 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:50.986244    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" containerName="istio-proxy"
Jul 25 02:20:50.986322 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:50.986254    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" containerName="istio-proxy"
Jul 25 02:20:50.986322 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:50.986260    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" containerName="istio-proxy"
Jul 25 02:20:50.986576 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:50.986335    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b79567c3-b601-48a8-95ca-e2c9488634a0" containerName="main"
Jul 25 02:20:50.986576 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:50.986347    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="48ee41a1-a0f0-462a-80bd-fe9449bfeec1" containerName="revoke-keys"
Jul 25 02:20:50.986576 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:50.986357    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" containerName="istio-proxy"
Jul 25 02:20:50.986576 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:50.986363    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f9be38c-c6a6-4849-b8e9-e74d86f4d9a7" containerName="istio-proxy"
Jul 25 02:20:50.990371 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:50.990355    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv"
Jul 25 02:20:50.992406 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:50.992387    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 02:20:50.992541 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:50.992423    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cjsrv\""
Jul 25 02:20:50.997574 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:50.997548    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv"]
Jul 25 02:20:51.167740 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:51.167704    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1daa5f64-6c80-4485-be62-ff3187d22c11-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv\" (UID: \"1daa5f64-6c80-4485-be62-ff3187d22c11\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv"
Jul 25 02:20:51.268583 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:51.268494    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1daa5f64-6c80-4485-be62-ff3187d22c11-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv\" (UID: \"1daa5f64-6c80-4485-be62-ff3187d22c11\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv"
Jul 25 02:20:51.269145 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:51.269124    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1daa5f64-6c80-4485-be62-ff3187d22c11-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv\" (UID: \"1daa5f64-6c80-4485-be62-ff3187d22c11\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv"
Jul 25 02:20:51.303548 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:51.303517    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv"
Jul 25 02:20:51.439313 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:51.439280    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv"]
Jul 25 02:20:51.442518 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:20:51.442477    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1daa5f64_6c80_4485_be62_ff3187d22c11.slice/crio-500d317e0e663b07005d929edbe77c5e28cb4e52c62ce2bbd63e6cc287089672 WatchSource:0}: Error finding container 500d317e0e663b07005d929edbe77c5e28cb4e52c62ce2bbd63e6cc287089672: Status 404 returned error can't find the container with id 500d317e0e663b07005d929edbe77c5e28cb4e52c62ce2bbd63e6cc287089672
Jul 25 02:20:51.576624 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:51.576541    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv" event={"ID":"1daa5f64-6c80-4485-be62-ff3187d22c11","Type":"ContainerStarted","Data":"500d317e0e663b07005d929edbe77c5e28cb4e52c62ce2bbd63e6cc287089672"}
Jul 25 02:20:52.583370 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:52.583339    2576 generic.go:356] "Generic (PLEG): container finished" podID="1daa5f64-6c80-4485-be62-ff3187d22c11" containerID="5c13c5d83b8f89ea26787894320b58c4ec86148d4e95037a603b7692846d1536" exitCode=0
Jul 25 02:20:52.583776 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:52.583383    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv" event={"ID":"1daa5f64-6c80-4485-be62-ff3187d22c11","Type":"ContainerDied","Data":"5c13c5d83b8f89ea26787894320b58c4ec86148d4e95037a603b7692846d1536"}
Jul 25 02:20:53.725498 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:53.725472    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv"
Jul 25 02:20:53.787721 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:53.787697    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1daa5f64-6c80-4485-be62-ff3187d22c11-maas-api-service-ca\") pod \"1daa5f64-6c80-4485-be62-ff3187d22c11\" (UID: \"1daa5f64-6c80-4485-be62-ff3187d22c11\") "
Jul 25 02:20:53.788028 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:53.788006    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1daa5f64-6c80-4485-be62-ff3187d22c11-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1daa5f64-6c80-4485-be62-ff3187d22c11" (UID: "1daa5f64-6c80-4485-be62-ff3187d22c11"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 02:20:53.889095 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:53.889023    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1daa5f64-6c80-4485-be62-ff3187d22c11-maas-api-service-ca\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:20:54.594873 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:54.594844    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv"
Jul 25 02:20:54.595047 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:54.594839    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv" event={"ID":"1daa5f64-6c80-4485-be62-ff3187d22c11","Type":"ContainerDied","Data":"500d317e0e663b07005d929edbe77c5e28cb4e52c62ce2bbd63e6cc287089672"}
Jul 25 02:20:54.595047 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:20:54.594966    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="500d317e0e663b07005d929edbe77c5e28cb4e52c62ce2bbd63e6cc287089672"
Jul 25 02:21:04.824805 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:04.824770    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:21:06.045913 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:06.045874    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv"]
Jul 25 02:21:06.052843 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:06.052811    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-dc8abe-n2tnv"]
Jul 25 02:21:06.996387 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:06.996351    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1daa5f64-6c80-4485-be62-ff3187d22c11" path="/var/lib/kubelet/pods/1daa5f64-6c80-4485-be62-ff3187d22c11/volumes"
Jul 25 02:21:12.886707 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:12.886604    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"]
Jul 25 02:21:12.887149 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:12.886872    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh" podUID="2b9098b5-2e5d-4db3-a7f8-075369e47b59" containerName="istio-proxy" containerID="cri-o://d3477a501757232fd568a5ef656306787a77352b369b1df2cd9be771451748bd" gracePeriod=30
Jul 25 02:21:13.275233 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:13.275203    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2"]
Jul 25 02:21:13.275552 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:13.275538    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1daa5f64-6c80-4485-be62-ff3187d22c11" containerName="revoke-keys"
Jul 25 02:21:13.275619 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:13.275554    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1daa5f64-6c80-4485-be62-ff3187d22c11" containerName="revoke-keys"
Jul 25 02:21:13.275678 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:13.275629    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1daa5f64-6c80-4485-be62-ff3187d22c11" containerName="revoke-keys"
Jul 25 02:21:13.278687 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:13.278668    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2"
Jul 25 02:21:13.280421 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:13.280403    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cjsrv\""
Jul 25 02:21:13.280543 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:13.280406    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 02:21:13.284461 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:13.284437    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2"]
Jul 25 02:21:13.342931 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:13.342902    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/277c8d82-3cd3-4689-8d20-4b0160c166e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2\" (UID: \"277c8d82-3cd3-4689-8d20-4b0160c166e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2"
Jul 25 02:21:13.443699 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:13.443647    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/277c8d82-3cd3-4689-8d20-4b0160c166e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2\" (UID: \"277c8d82-3cd3-4689-8d20-4b0160c166e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2"
Jul 25 02:21:13.444272 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:13.444251    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/277c8d82-3cd3-4689-8d20-4b0160c166e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2\" (UID: \"277c8d82-3cd3-4689-8d20-4b0160c166e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2"
Jul 25 02:21:13.590440 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:13.590362    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2"
Jul 25 02:21:13.723033 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:13.723003    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2"]
Jul 25 02:21:13.726685 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:21:13.726644    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod277c8d82_3cd3_4689_8d20_4b0160c166e9.slice/crio-2e73f2ec1c80e7768e54ca2c15da28cb70ec0d6ade2062fedd98f5850fe0be47 WatchSource:0}: Error finding container 2e73f2ec1c80e7768e54ca2c15da28cb70ec0d6ade2062fedd98f5850fe0be47: Status 404 returned error can't find the container with id 2e73f2ec1c80e7768e54ca2c15da28cb70ec0d6ade2062fedd98f5850fe0be47
Jul 25 02:21:14.690336 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:14.690239    2576 generic.go:356] "Generic (PLEG): container finished" podID="277c8d82-3cd3-4689-8d20-4b0160c166e9" containerID="ef1787c96876749268ddfb91d70e76e91092bb9a9134154aadb02404c49f5e58" exitCode=0
Jul 25 02:21:14.690336 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:14.690289    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2" event={"ID":"277c8d82-3cd3-4689-8d20-4b0160c166e9","Type":"ContainerDied","Data":"ef1787c96876749268ddfb91d70e76e91092bb9a9134154aadb02404c49f5e58"}
Jul 25 02:21:14.690764 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:14.690343    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2" event={"ID":"277c8d82-3cd3-4689-8d20-4b0160c166e9","Type":"ContainerStarted","Data":"2e73f2ec1c80e7768e54ca2c15da28cb70ec0d6ade2062fedd98f5850fe0be47"}
Jul 25 02:21:15.819052 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:15.819029    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2"
Jul 25 02:21:15.864486 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:15.864461    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/277c8d82-3cd3-4689-8d20-4b0160c166e9-maas-api-service-ca\") pod \"277c8d82-3cd3-4689-8d20-4b0160c166e9\" (UID: \"277c8d82-3cd3-4689-8d20-4b0160c166e9\") "
Jul 25 02:21:15.864821 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:15.864801    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/277c8d82-3cd3-4689-8d20-4b0160c166e9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "277c8d82-3cd3-4689-8d20-4b0160c166e9" (UID: "277c8d82-3cd3-4689-8d20-4b0160c166e9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 02:21:15.966459 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:15.966376    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/277c8d82-3cd3-4689-8d20-4b0160c166e9-maas-api-service-ca\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:21:16.700247 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:16.700216    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2"
Jul 25 02:21:16.700419 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:16.700245    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2" event={"ID":"277c8d82-3cd3-4689-8d20-4b0160c166e9","Type":"ContainerDied","Data":"2e73f2ec1c80e7768e54ca2c15da28cb70ec0d6ade2062fedd98f5850fe0be47"}
Jul 25 02:21:16.700419 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:16.700294    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e73f2ec1c80e7768e54ca2c15da28cb70ec0d6ade2062fedd98f5850fe0be47"
Jul 25 02:21:18.134941 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.134917    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:21:18.182955 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.182927    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-podinfo\") pod \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") "
Jul 25 02:21:18.183101 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.182972    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-workload-certs\") pod \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") "
Jul 25 02:21:18.183101 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.183010    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h69cj\" (UniqueName: \"kubernetes.io/projected/2b9098b5-2e5d-4db3-a7f8-075369e47b59-kube-api-access-h69cj\") pod \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") "
Jul 25 02:21:18.183101 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.183029    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-token\") pod \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") "
Jul 25 02:21:18.183101 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.183063    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-workload-socket\") pod \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") "
Jul 25 02:21:18.183466 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.183117    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-data\") pod \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") "
Jul 25 02:21:18.183466 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.183165    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-credential-socket\") pod \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") "
Jul 25 02:21:18.183466 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.183193    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istiod-ca-cert\") pod \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") "
Jul 25 02:21:18.183466 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.183220    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-envoy\") pod \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\" (UID: \"2b9098b5-2e5d-4db3-a7f8-075369e47b59\") "
Jul 25 02:21:18.183466 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.183294    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2b9098b5-2e5d-4db3-a7f8-075369e47b59" (UID: "2b9098b5-2e5d-4db3-a7f8-075369e47b59"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:21:18.183466 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.183434    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2b9098b5-2e5d-4db3-a7f8-075369e47b59" (UID: "2b9098b5-2e5d-4db3-a7f8-075369e47b59"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:21:18.183774 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.183581    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-workload-socket\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:21:18.183774 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.183601    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-workload-certs\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:21:18.183774 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.183692    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2b9098b5-2e5d-4db3-a7f8-075369e47b59" (UID: "2b9098b5-2e5d-4db3-a7f8-075369e47b59"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 02:21:18.183919 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.183895    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2b9098b5-2e5d-4db3-a7f8-075369e47b59" (UID: "2b9098b5-2e5d-4db3-a7f8-075369e47b59"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:21:18.184442 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.184414    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-data" (OuterVolumeSpecName: "istio-data") pod "2b9098b5-2e5d-4db3-a7f8-075369e47b59" (UID: "2b9098b5-2e5d-4db3-a7f8-075369e47b59"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:21:18.185311 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.185288    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-token" (OuterVolumeSpecName: "istio-token") pod "2b9098b5-2e5d-4db3-a7f8-075369e47b59" (UID: "2b9098b5-2e5d-4db3-a7f8-075369e47b59"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:21:18.185407 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.185304    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2b9098b5-2e5d-4db3-a7f8-075369e47b59" (UID: "2b9098b5-2e5d-4db3-a7f8-075369e47b59"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 25 02:21:18.185491 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.185475    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b9098b5-2e5d-4db3-a7f8-075369e47b59-kube-api-access-h69cj" (OuterVolumeSpecName: "kube-api-access-h69cj") pod "2b9098b5-2e5d-4db3-a7f8-075369e47b59" (UID: "2b9098b5-2e5d-4db3-a7f8-075369e47b59"). InnerVolumeSpecName "kube-api-access-h69cj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:21:18.185571 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.185554    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2b9098b5-2e5d-4db3-a7f8-075369e47b59" (UID: "2b9098b5-2e5d-4db3-a7f8-075369e47b59"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:21:18.284116 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.284076    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-data\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:21:18.284116 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.284113    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-credential-socket\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:21:18.284116 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.284123    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istiod-ca-cert\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:21:18.284116 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.284131    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-envoy\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:21:18.284354 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.284139    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-podinfo\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:21:18.284354 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.284147    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h69cj\" (UniqueName: \"kubernetes.io/projected/2b9098b5-2e5d-4db3-a7f8-075369e47b59-kube-api-access-h69cj\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:21:18.284354 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.284155    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b9098b5-2e5d-4db3-a7f8-075369e47b59-istio-token\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:21:18.711457 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.711420    2576 generic.go:356] "Generic (PLEG): container finished" podID="2b9098b5-2e5d-4db3-a7f8-075369e47b59" containerID="d3477a501757232fd568a5ef656306787a77352b369b1df2cd9be771451748bd" exitCode=0
Jul 25 02:21:18.711648 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.711478    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"
Jul 25 02:21:18.711648 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.711500    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh" event={"ID":"2b9098b5-2e5d-4db3-a7f8-075369e47b59","Type":"ContainerDied","Data":"d3477a501757232fd568a5ef656306787a77352b369b1df2cd9be771451748bd"}
Jul 25 02:21:18.711648 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.711550    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh" event={"ID":"2b9098b5-2e5d-4db3-a7f8-075369e47b59","Type":"ContainerDied","Data":"b0317fb8673f96b40cb1231b901be27773f4f59c994acc1ea7f89c554a9734e3"}
Jul 25 02:21:18.711648 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.711573    2576 scope.go:117] "RemoveContainer" containerID="d3477a501757232fd568a5ef656306787a77352b369b1df2cd9be771451748bd"
Jul 25 02:21:18.720829 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.720746    2576 scope.go:117] "RemoveContainer" containerID="ea6a8c41b0faae6b33c5e58e1fb9b0e92be191823153c4ce0c645805737f75d5"
Jul 25 02:21:18.728304 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.728289    2576 scope.go:117] "RemoveContainer" containerID="d3477a501757232fd568a5ef656306787a77352b369b1df2cd9be771451748bd"
Jul 25 02:21:18.728555 ip-10-0-134-238 kubenswrapper[2576]: E0725 02:21:18.728535    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3477a501757232fd568a5ef656306787a77352b369b1df2cd9be771451748bd\": container with ID starting with d3477a501757232fd568a5ef656306787a77352b369b1df2cd9be771451748bd not found: ID does not exist" containerID="d3477a501757232fd568a5ef656306787a77352b369b1df2cd9be771451748bd"
Jul 25 02:21:18.728598 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.728564    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3477a501757232fd568a5ef656306787a77352b369b1df2cd9be771451748bd"} err="failed to get container status \"d3477a501757232fd568a5ef656306787a77352b369b1df2cd9be771451748bd\": rpc error: code = NotFound desc = could not find container \"d3477a501757232fd568a5ef656306787a77352b369b1df2cd9be771451748bd\": container with ID starting with d3477a501757232fd568a5ef656306787a77352b369b1df2cd9be771451748bd not found: ID does not exist"
Jul 25 02:21:18.728598 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.728582    2576 scope.go:117] "RemoveContainer" containerID="ea6a8c41b0faae6b33c5e58e1fb9b0e92be191823153c4ce0c645805737f75d5"
Jul 25 02:21:18.728828 ip-10-0-134-238 kubenswrapper[2576]: E0725 02:21:18.728808    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea6a8c41b0faae6b33c5e58e1fb9b0e92be191823153c4ce0c645805737f75d5\": container with ID starting with ea6a8c41b0faae6b33c5e58e1fb9b0e92be191823153c4ce0c645805737f75d5 not found: ID does not exist" containerID="ea6a8c41b0faae6b33c5e58e1fb9b0e92be191823153c4ce0c645805737f75d5"
Jul 25 02:21:18.728890 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.728834    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea6a8c41b0faae6b33c5e58e1fb9b0e92be191823153c4ce0c645805737f75d5"} err="failed to get container status \"ea6a8c41b0faae6b33c5e58e1fb9b0e92be191823153c4ce0c645805737f75d5\": rpc error: code = NotFound desc = could not find container \"ea6a8c41b0faae6b33c5e58e1fb9b0e92be191823153c4ce0c645805737f75d5\": container with ID starting with ea6a8c41b0faae6b33c5e58e1fb9b0e92be191823153c4ce0c645805737f75d5 not found: ID does not exist"
Jul 25 02:21:18.731284 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.731264    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"]
Jul 25 02:21:18.734922 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.734903    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-dc8abe-openshift-default-b6788f558-5bcbh"]
Jul 25 02:21:18.996224 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:18.996151    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b9098b5-2e5d-4db3-a7f8-075369e47b59" path="/var/lib/kubelet/pods/2b9098b5-2e5d-4db3-a7f8-075369e47b59/volumes"
Jul 25 02:21:23.434938 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:23.434899    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e06ec-29749095-2czkm"]
Jul 25 02:21:23.438093 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:23.438068    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7e06ec-29749095-2czkm"]
Jul 25 02:21:24.996041 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:24.996005    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8229208-c671-4dd7-9201-2185d0c90d22" path="/var/lib/kubelet/pods/e8229208-c671-4dd7-9201-2185d0c90d22/volumes"
Jul 25 02:21:25.724685 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:25.724638    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:21:28.363531 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:28.363486    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2"]
Jul 25 02:21:28.364037 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:28.363547    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7e06ec-96qc2"]
Jul 25 02:21:28.996563 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:28.996528    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="277c8d82-3cd3-4689-8d20-4b0160c166e9" path="/var/lib/kubelet/pods/277c8d82-3cd3-4689-8d20-4b0160c166e9/volumes"
Jul 25 02:21:35.294770 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.294729    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw"]
Jul 25 02:21:35.295889 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.295861    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="277c8d82-3cd3-4689-8d20-4b0160c166e9" containerName="revoke-keys"
Jul 25 02:21:35.295889 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.295889    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="277c8d82-3cd3-4689-8d20-4b0160c166e9" containerName="revoke-keys"
Jul 25 02:21:35.296369 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.296343    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b9098b5-2e5d-4db3-a7f8-075369e47b59" containerName="istio-proxy"
Jul 25 02:21:35.296369 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.296369    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b9098b5-2e5d-4db3-a7f8-075369e47b59" containerName="istio-proxy"
Jul 25 02:21:35.296560 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.296393    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b9098b5-2e5d-4db3-a7f8-075369e47b59" containerName="istio-proxy"
Jul 25 02:21:35.296560 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.296402    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b9098b5-2e5d-4db3-a7f8-075369e47b59" containerName="istio-proxy"
Jul 25 02:21:35.296560 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.296517    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b9098b5-2e5d-4db3-a7f8-075369e47b59" containerName="istio-proxy"
Jul 25 02:21:35.296560 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.296528    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="277c8d82-3cd3-4689-8d20-4b0160c166e9" containerName="revoke-keys"
Jul 25 02:21:35.296560 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.296539    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b9098b5-2e5d-4db3-a7f8-075369e47b59" containerName="istio-proxy"
Jul 25 02:21:35.301338 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.301315    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw"]
Jul 25 02:21:35.301436 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.301411    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw"
Jul 25 02:21:35.303458 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.303435    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 25 02:21:35.303570 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.303458    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cjsrv\""
Jul 25 02:21:35.422163 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.422133    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/851f5f3d-9d56-46ee-a5c1-4d44dc7ede20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw\" (UID: \"851f5f3d-9d56-46ee-a5c1-4d44dc7ede20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw"
Jul 25 02:21:35.522564 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.522518    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/851f5f3d-9d56-46ee-a5c1-4d44dc7ede20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw\" (UID: \"851f5f3d-9d56-46ee-a5c1-4d44dc7ede20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw"
Jul 25 02:21:35.523154 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.523134    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/851f5f3d-9d56-46ee-a5c1-4d44dc7ede20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw\" (UID: \"851f5f3d-9d56-46ee-a5c1-4d44dc7ede20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw"
Jul 25 02:21:35.614604 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.614535    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw"
Jul 25 02:21:35.750892 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.750870    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw"]
Jul 25 02:21:35.753457 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:21:35.753429    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod851f5f3d_9d56_46ee_a5c1_4d44dc7ede20.slice/crio-bfbc396bdab808c369749574a537239c9144ab20730f51a18ae94c4125c30c01 WatchSource:0}: Error finding container bfbc396bdab808c369749574a537239c9144ab20730f51a18ae94c4125c30c01: Status 404 returned error can't find the container with id bfbc396bdab808c369749574a537239c9144ab20730f51a18ae94c4125c30c01
Jul 25 02:21:35.792598 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:35.792572    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw" event={"ID":"851f5f3d-9d56-46ee-a5c1-4d44dc7ede20","Type":"ContainerStarted","Data":"bfbc396bdab808c369749574a537239c9144ab20730f51a18ae94c4125c30c01"}
Jul 25 02:21:36.798434 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:36.798395    2576 generic.go:356] "Generic (PLEG): container finished" podID="851f5f3d-9d56-46ee-a5c1-4d44dc7ede20" containerID="464f3d5049118727bd40125d6277d9f90ad2634aafd592e8f11f08cefa1cb174" exitCode=0
Jul 25 02:21:36.798896 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:36.798463    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw" event={"ID":"851f5f3d-9d56-46ee-a5c1-4d44dc7ede20","Type":"ContainerDied","Data":"464f3d5049118727bd40125d6277d9f90ad2634aafd592e8f11f08cefa1cb174"}
Jul 25 02:21:37.934797 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:37.934777    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw"
Jul 25 02:21:38.043628 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:38.043595    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/851f5f3d-9d56-46ee-a5c1-4d44dc7ede20-maas-api-service-ca\") pod \"851f5f3d-9d56-46ee-a5c1-4d44dc7ede20\" (UID: \"851f5f3d-9d56-46ee-a5c1-4d44dc7ede20\") "
Jul 25 02:21:38.043971 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:38.043948    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/851f5f3d-9d56-46ee-a5c1-4d44dc7ede20-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "851f5f3d-9d56-46ee-a5c1-4d44dc7ede20" (UID: "851f5f3d-9d56-46ee-a5c1-4d44dc7ede20"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 02:21:38.144631 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:38.144549    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/851f5f3d-9d56-46ee-a5c1-4d44dc7ede20-maas-api-service-ca\") on node \"ip-10-0-134-238.ec2.internal\" DevicePath \"\""
Jul 25 02:21:38.809625 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:38.809595    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw"
Jul 25 02:21:38.809846 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:38.809632    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw" event={"ID":"851f5f3d-9d56-46ee-a5c1-4d44dc7ede20","Type":"ContainerDied","Data":"bfbc396bdab808c369749574a537239c9144ab20730f51a18ae94c4125c30c01"}
Jul 25 02:21:38.809846 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:38.809700    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bfbc396bdab808c369749574a537239c9144ab20730f51a18ae94c4125c30c01"
Jul 25 02:21:45.921801 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:45.921767    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-575aa7-29749095-r7psl"]
Jul 25 02:21:45.926113 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:45.926089    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-575aa7-29749095-r7psl"]
Jul 25 02:21:46.995483 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:46.995449    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35baa7d1-311c-4794-b637-90704ebedae1" path="/var/lib/kubelet/pods/35baa7d1-311c-4794-b637-90704ebedae1/volumes"
Jul 25 02:21:49.029403 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:49.029367    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kg9k"]
Jul 25 02:21:49.479435 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:49.479410    2576 scope.go:117] "RemoveContainer" containerID="0702bdcc7c37d90846793ca6979b47715c58a9800ee3c7bad879098ab4d43c2b"
Jul 25 02:21:49.487388 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:49.487369    2576 scope.go:117] "RemoveContainer" containerID="ce16c814eff7b460dc0897b4145910f05c70726303ddb5a5282c6daa7648ca69"
Jul 25 02:21:50.354672 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:50.354632    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw"]
Jul 25 02:21:50.356907 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:50.356882    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-575aa7-4k4rw"]
Jul 25 02:21:50.996141 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:50.996107    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="851f5f3d-9d56-46ee-a5c1-4d44dc7ede20" path="/var/lib/kubelet/pods/851f5f3d-9d56-46ee-a5c1-4d44dc7ede20/volumes"
Jul 25 02:21:57.484245 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:21:57.484214    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c997bfc44-rzjrt_bca994f4-be7f-4182-b54a-3a3685227b88/authorino/0.log"
Jul 25 02:22:03.192835 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:03.192806    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cwscn_e927bf51-6950-41c6-9a76-a578c5c782f2/server/0.log"
Jul 25 02:22:03.314681 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:03.314638    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7bz4g_d7221467-0e5d-4b32-ba02-9f57785a77c6/manager/0.log"
Jul 25 02:22:03.545605 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:03.545530    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9844cf678-ms57s_a852dc15-df04-4625-baa2-98e0a63b621a/copy-manifests/0.log"
Jul 25 02:22:03.556223 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:03.556196    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9844cf678-ms57s_a852dc15-df04-4625-baa2-98e0a63b621a/manager/0.log"
Jul 25 02:22:04.355879 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:04.355852    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29749095-98sq7_75ed5571-22d6-43f8-9cd8-97424a713dea/cleanup/0.log"
Jul 25 02:22:05.756494 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:05.756467    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c997bfc44-rzjrt_bca994f4-be7f-4182-b54a-3a3685227b88/authorino/0.log"
Jul 25 02:22:06.453904 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:06.453879    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-6kg9k_4e0bf5fd-a6bd-40ff-ac8f-db9f5247fb73/limitador/0.log"
Jul 25 02:22:07.268076 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:07.268051    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c6bd6744b-nwrb5_a6d8e3b8-9f54-4208-9295-e601f6c5da1d/kube-auth-proxy/0.log"
Jul 25 02:22:08.662331 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:08.662298    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-j77dk_9998c918-0a90-4f65-92a2-8fe3d8dcf622/main/0.log"
Jul 25 02:22:14.758025 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:14.757992    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-m4dqv/must-gather-9kjbc"]
Jul 25 02:22:14.758521 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:14.758480    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="851f5f3d-9d56-46ee-a5c1-4d44dc7ede20" containerName="revoke-keys"
Jul 25 02:22:14.758521 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:14.758499    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="851f5f3d-9d56-46ee-a5c1-4d44dc7ede20" containerName="revoke-keys"
Jul 25 02:22:14.758644 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:14.758582    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="851f5f3d-9d56-46ee-a5c1-4d44dc7ede20" containerName="revoke-keys"
Jul 25 02:22:14.761583 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:14.761563    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m4dqv/must-gather-9kjbc"
Jul 25 02:22:14.764182 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:14.764160    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m4dqv\"/\"openshift-service-ca.crt\""
Jul 25 02:22:14.764627 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:14.764612    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m4dqv\"/\"kube-root-ca.crt\""
Jul 25 02:22:14.764802 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:14.764788    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-m4dqv\"/\"default-dockercfg-nfx4k\""
Jul 25 02:22:14.785101 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:14.785079    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m4dqv/must-gather-9kjbc"]
Jul 25 02:22:14.830949 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:14.830914    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5cdn\" (UniqueName: \"kubernetes.io/projected/9d18df7d-11bb-46bc-baf4-d39ab6a44b30-kube-api-access-f5cdn\") pod \"must-gather-9kjbc\" (UID: \"9d18df7d-11bb-46bc-baf4-d39ab6a44b30\") " pod="openshift-must-gather-m4dqv/must-gather-9kjbc"
Jul 25 02:22:14.831102 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:14.830964    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9d18df7d-11bb-46bc-baf4-d39ab6a44b30-must-gather-output\") pod \"must-gather-9kjbc\" (UID: \"9d18df7d-11bb-46bc-baf4-d39ab6a44b30\") " pod="openshift-must-gather-m4dqv/must-gather-9kjbc"
Jul 25 02:22:14.931708 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:14.931684    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5cdn\" (UniqueName: \"kubernetes.io/projected/9d18df7d-11bb-46bc-baf4-d39ab6a44b30-kube-api-access-f5cdn\") pod \"must-gather-9kjbc\" (UID: \"9d18df7d-11bb-46bc-baf4-d39ab6a44b30\") " pod="openshift-must-gather-m4dqv/must-gather-9kjbc"
Jul 25 02:22:14.931861 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:14.931717    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9d18df7d-11bb-46bc-baf4-d39ab6a44b30-must-gather-output\") pod \"must-gather-9kjbc\" (UID: \"9d18df7d-11bb-46bc-baf4-d39ab6a44b30\") " pod="openshift-must-gather-m4dqv/must-gather-9kjbc"
Jul 25 02:22:14.931992 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:14.931975    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9d18df7d-11bb-46bc-baf4-d39ab6a44b30-must-gather-output\") pod \"must-gather-9kjbc\" (UID: \"9d18df7d-11bb-46bc-baf4-d39ab6a44b30\") " pod="openshift-must-gather-m4dqv/must-gather-9kjbc"
Jul 25 02:22:14.939125 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:14.939104    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5cdn\" (UniqueName: \"kubernetes.io/projected/9d18df7d-11bb-46bc-baf4-d39ab6a44b30-kube-api-access-f5cdn\") pod \"must-gather-9kjbc\" (UID: \"9d18df7d-11bb-46bc-baf4-d39ab6a44b30\") " pod="openshift-must-gather-m4dqv/must-gather-9kjbc"
Jul 25 02:22:15.070399 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:15.070316    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m4dqv/must-gather-9kjbc"
Jul 25 02:22:15.196997 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:15.196963    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m4dqv/must-gather-9kjbc"]
Jul 25 02:22:15.199919 ip-10-0-134-238 kubenswrapper[2576]: W0725 02:22:15.199886    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d18df7d_11bb_46bc_baf4_d39ab6a44b30.slice/crio-c924d890a33029d6a6790dc28a0966d834ad200de6b45b9ddbd6a98b6e8efd43 WatchSource:0}: Error finding container c924d890a33029d6a6790dc28a0966d834ad200de6b45b9ddbd6a98b6e8efd43: Status 404 returned error can't find the container with id c924d890a33029d6a6790dc28a0966d834ad200de6b45b9ddbd6a98b6e8efd43
Jul 25 02:22:15.203826 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:15.202065    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 02:22:15.996443 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:15.996414    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m4dqv/must-gather-9kjbc" event={"ID":"9d18df7d-11bb-46bc-baf4-d39ab6a44b30","Type":"ContainerStarted","Data":"32fca954db885eb8ab23318acce1bef40b056ad64096f4e1a363de710a5eb92d"}
Jul 25 02:22:15.996801 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:15.996459    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m4dqv/must-gather-9kjbc" event={"ID":"9d18df7d-11bb-46bc-baf4-d39ab6a44b30","Type":"ContainerStarted","Data":"c924d890a33029d6a6790dc28a0966d834ad200de6b45b9ddbd6a98b6e8efd43"}
Jul 25 02:22:17.006469 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:17.006433    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m4dqv/must-gather-9kjbc" event={"ID":"9d18df7d-11bb-46bc-baf4-d39ab6a44b30","Type":"ContainerStarted","Data":"2c7258a536c0bec0bff6e52329ba0f1f99989bbc6b9637e88fd1e9d447c60cc3"}
Jul 25 02:22:17.022463 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:17.022413    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-m4dqv/must-gather-9kjbc" podStartSLOduration=2.338906413 podStartE2EDuration="3.022398533s" totalImagesPullingTime="683.49212ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 02:22:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:22:17.020354636 +0000 UTC m=+4288.507651312" watchObservedRunningTime="2026-07-25 02:22:17.022398533 +0000 UTC m=+4288.509695207"
Jul 25 02:22:17.392484 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:17.392409    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bqzgx_056dc3b3-b95b-4d1f-8606-333625c397a3/global-pull-secret-syncer/0.log"
Jul 25 02:22:17.606849 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:17.606814    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-h7kwx_f8888b32-a7c5-4b96-b48d-133e3491fb81/konnectivity-agent/0.log"
Jul 25 02:22:17.652105 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:17.652071    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-238.ec2.internal_1b910c3459c38a928558d492ab28b0ef/haproxy/0.log"
Jul 25 02:22:22.013605 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:22.013220    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c997bfc44-rzjrt_bca994f4-be7f-4182-b54a-3a3685227b88/authorino/0.log"
Jul 25 02:22:22.395415 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:22.395330    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-6kg9k_4e0bf5fd-a6bd-40ff-ac8f-db9f5247fb73/limitador/0.log"
Jul 25 02:22:24.507494 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:24.507463    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rq6dv_2caa7660-548b-4bc2-bf73-446c447347ce/node-exporter/0.log"
Jul 25 02:22:24.526593 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:24.526554    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rq6dv_2caa7660-548b-4bc2-bf73-446c447347ce/kube-rbac-proxy/0.log"
Jul 25 02:22:24.546190 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:24.546168    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rq6dv_2caa7660-548b-4bc2-bf73-446c447347ce/init-textfile/0.log"
Jul 25 02:22:26.457993 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:26.457939    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-52w8s_0b0f3243-f94a-47e2-a884-3ae235a03f07/networking-console-plugin/0.log"
Jul 25 02:22:27.083841 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:27.083794    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/2.log"
Jul 25 02:22:27.090734 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:27.090707    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-28cvc_6acccfec-8c59-4af6-af57-fd9445830798/console-operator/3.log"
Jul 25 02:22:27.639015 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:27.638987    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6dc9685f5d-mmbth_bbba638f-b747-4abf-871f-6c393e2e05b7/console/0.log"
Jul 25 02:22:27.675608 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:27.675578    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-rswvx_4b696896-09b1-4599-85a3-c14700d2fd08/download-server/0.log"
Jul 25 02:22:28.078295 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.078262    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"]
Jul 25 02:22:28.086389 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.086356    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:28.093855 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.093822    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"]
Jul 25 02:22:28.167555 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.167252    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfkkb\" (UniqueName: \"kubernetes.io/projected/bb592186-7d90-42eb-954d-a3e08c8d7362-kube-api-access-bfkkb\") pod \"perf-node-gather-daemonset-4kx2d\" (UID: \"bb592186-7d90-42eb-954d-a3e08c8d7362\") " pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:28.167555 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.167297    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb592186-7d90-42eb-954d-a3e08c8d7362-lib-modules\") pod \"perf-node-gather-daemonset-4kx2d\" (UID: \"bb592186-7d90-42eb-954d-a3e08c8d7362\") " pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:28.167555 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.167331    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb592186-7d90-42eb-954d-a3e08c8d7362-sys\") pod \"perf-node-gather-daemonset-4kx2d\" (UID: \"bb592186-7d90-42eb-954d-a3e08c8d7362\") " pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:28.167555 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.167410    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bb592186-7d90-42eb-954d-a3e08c8d7362-proc\") pod \"perf-node-gather-daemonset-4kx2d\" (UID: \"bb592186-7d90-42eb-954d-a3e08c8d7362\") " pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:28.167555 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.167456    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bb592186-7d90-42eb-954d-a3e08c8d7362-podres\") pod \"perf-node-gather-daemonset-4kx2d\" (UID: \"bb592186-7d90-42eb-954d-a3e08c8d7362\") " pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:28.268153 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.268112    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bb592186-7d90-42eb-954d-a3e08c8d7362-podres\") pod \"perf-node-gather-daemonset-4kx2d\" (UID: \"bb592186-7d90-42eb-954d-a3e08c8d7362\") " pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:28.268343 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.268187    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfkkb\" (UniqueName: \"kubernetes.io/projected/bb592186-7d90-42eb-954d-a3e08c8d7362-kube-api-access-bfkkb\") pod \"perf-node-gather-daemonset-4kx2d\" (UID: \"bb592186-7d90-42eb-954d-a3e08c8d7362\") " pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:28.268343 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.268220    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb592186-7d90-42eb-954d-a3e08c8d7362-lib-modules\") pod \"perf-node-gather-daemonset-4kx2d\" (UID: \"bb592186-7d90-42eb-954d-a3e08c8d7362\") " pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:28.268343 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.268258    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb592186-7d90-42eb-954d-a3e08c8d7362-sys\") pod \"perf-node-gather-daemonset-4kx2d\" (UID: \"bb592186-7d90-42eb-954d-a3e08c8d7362\") " pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:28.268343 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.268334    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bb592186-7d90-42eb-954d-a3e08c8d7362-proc\") pod \"perf-node-gather-daemonset-4kx2d\" (UID: \"bb592186-7d90-42eb-954d-a3e08c8d7362\") " pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:28.268583 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.268418    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bb592186-7d90-42eb-954d-a3e08c8d7362-proc\") pod \"perf-node-gather-daemonset-4kx2d\" (UID: \"bb592186-7d90-42eb-954d-a3e08c8d7362\") " pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:28.268583 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.268553    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bb592186-7d90-42eb-954d-a3e08c8d7362-podres\") pod \"perf-node-gather-daemonset-4kx2d\" (UID: \"bb592186-7d90-42eb-954d-a3e08c8d7362\") " pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:28.269004 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.268977    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bb592186-7d90-42eb-954d-a3e08c8d7362-lib-modules\") pod \"perf-node-gather-daemonset-4kx2d\" (UID: \"bb592186-7d90-42eb-954d-a3e08c8d7362\") " pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:28.269088 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.269043    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb592186-7d90-42eb-954d-a3e08c8d7362-sys\") pod \"perf-node-gather-daemonset-4kx2d\" (UID: \"bb592186-7d90-42eb-954d-a3e08c8d7362\") " pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:28.283069 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.283043    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfkkb\" (UniqueName: \"kubernetes.io/projected/bb592186-7d90-42eb-954d-a3e08c8d7362-kube-api-access-bfkkb\") pod \"perf-node-gather-daemonset-4kx2d\" (UID: \"bb592186-7d90-42eb-954d-a3e08c8d7362\") " pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:28.400891 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.400804    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:28.432319 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.432285    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-59l68_405e24e0-1385-4056-82c3-bf7eca2bd7e4/volume-data-source-validator/0.log"
Jul 25 02:22:28.588329 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:28.587595    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"]
Jul 25 02:22:29.087647 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:29.087476    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d" event={"ID":"bb592186-7d90-42eb-954d-a3e08c8d7362","Type":"ContainerStarted","Data":"581711ba28bf445c22ab4fd8e6ff66350e28592d9e93f4ef8b00d62c37e21489"}
Jul 25 02:22:29.087647 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:29.087538    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d" event={"ID":"bb592186-7d90-42eb-954d-a3e08c8d7362","Type":"ContainerStarted","Data":"e3a586ffb566918509a0b29ed863e15d0b6719b404c34705ffaa60e153771456"}
Jul 25 02:22:29.087647 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:29.087571    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:29.105925 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:29.105871    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d" podStartSLOduration=1.105851551 podStartE2EDuration="1.105851551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 02:22:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:22:29.104517307 +0000 UTC m=+4300.591813983" watchObservedRunningTime="2026-07-25 02:22:29.105851551 +0000 UTC m=+4300.593148229"
Jul 25 02:22:29.634438 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:29.634408    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tqwn9_aa437a19-20d8-4209-9886-7c07c2b5bf75/dns/0.log"
Jul 25 02:22:29.652527 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:29.652503    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tqwn9_aa437a19-20d8-4209-9886-7c07c2b5bf75/kube-rbac-proxy/0.log"
Jul 25 02:22:29.719710 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:29.719681    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mttfq_c804a635-6110-4cea-b2d8-5e8660b449f8/dns-node-resolver/0.log"
Jul 25 02:22:30.266900 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:30.266873    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pmpt5_c6ebb66f-5622-481b-ae46-7b3743feec00/node-ca/0.log"
Jul 25 02:22:31.462052 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:31.462023    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c6bd6744b-nwrb5_a6d8e3b8-9f54-4208-9295-e601f6c5da1d/kube-auth-proxy/0.log"
Jul 25 02:22:32.408865 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:32.408833    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jhxl9_a0b30ace-19f7-4c3c-949e-7526c7a69d90/serve-healthcheck-canary/0.log"
Jul 25 02:22:32.970494 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:32.970441    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f4npq_8dab96cc-78b2-4915-b417-b04a84616d13/kube-rbac-proxy/0.log"
Jul 25 02:22:32.990454 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:32.990427    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f4npq_8dab96cc-78b2-4915-b417-b04a84616d13/exporter/0.log"
Jul 25 02:22:33.012981 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:33.012950    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f4npq_8dab96cc-78b2-4915-b417-b04a84616d13/extractor/0.log"
Jul 25 02:22:35.111389 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:35.111363    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-m4dqv/perf-node-gather-daemonset-4kx2d"
Jul 25 02:22:35.724364 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:35.724336    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cwscn_e927bf51-6950-41c6-9a76-a578c5c782f2/server/0.log"
Jul 25 02:22:35.821435 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:35.821399    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7bz4g_d7221467-0e5d-4b32-ba02-9f57785a77c6/manager/0.log"
Jul 25 02:22:35.980386 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:35.980309    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9844cf678-ms57s_a852dc15-df04-4625-baa2-98e0a63b621a/manager/0.log"
Jul 25 02:22:35.995479 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:35.995447    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9844cf678-ms57s_a852dc15-df04-4625-baa2-98e0a63b621a/copy-manifests/0.log"
Jul 25 02:22:37.377146 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:37.377112    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-66889bf7cd-rc5rx_ce0ca022-f93f-4269-85c1-5cc9b49abf24/manager/0.log"
Jul 25 02:22:38.212103 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:38.212073    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c6bd6744b-nwrb5_a6d8e3b8-9f54-4208-9295-e601f6c5da1d/kube-auth-proxy/0.log"
Jul 25 02:22:40.031915 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:40.031882    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c6bd6744b-nwrb5_a6d8e3b8-9f54-4208-9295-e601f6c5da1d/kube-auth-proxy/0.log"
Jul 25 02:22:42.724410 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:42.724307    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x_860680d1-a426-4fea-9c8f-892d44983420/kube-storage-version-migrator-operator/1.log"
Jul 25 02:22:42.725206 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:42.725189    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-qjk5x_860680d1-a426-4fea-9c8f-892d44983420/kube-storage-version-migrator-operator/0.log"
Jul 25 02:22:43.802803 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:43.802770    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8srzj_026f8bf6-bdcf-47a9-930e-59323c2cb9f3/kube-multus-additional-cni-plugins/0.log"
Jul 25 02:22:43.821961 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:43.821934    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8srzj_026f8bf6-bdcf-47a9-930e-59323c2cb9f3/egress-router-binary-copy/0.log"
Jul 25 02:22:43.841023 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:43.841001    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8srzj_026f8bf6-bdcf-47a9-930e-59323c2cb9f3/cni-plugins/0.log"
Jul 25 02:22:43.859845 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:43.859819    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8srzj_026f8bf6-bdcf-47a9-930e-59323c2cb9f3/bond-cni-plugin/0.log"
Jul 25 02:22:43.879097 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:43.879077    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8srzj_026f8bf6-bdcf-47a9-930e-59323c2cb9f3/routeoverride-cni/0.log"
Jul 25 02:22:43.897609 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:43.897590    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8srzj_026f8bf6-bdcf-47a9-930e-59323c2cb9f3/whereabouts-cni-bincopy/0.log"
Jul 25 02:22:43.915945 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:43.915921    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8srzj_026f8bf6-bdcf-47a9-930e-59323c2cb9f3/whereabouts-cni/0.log"
Jul 25 02:22:44.195852 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:44.195819    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-swm4p_58dcfb48-9812-4413-b614-dd5cca8dd63a/kube-multus/0.log"
Jul 25 02:22:44.217361 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:44.217332    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-87t4z_6f9c02f7-779b-41bb-8260-3244475a0f20/network-metrics-daemon/0.log"
Jul 25 02:22:44.233986 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:44.233960    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-87t4z_6f9c02f7-779b-41bb-8260-3244475a0f20/kube-rbac-proxy/0.log"
Jul 25 02:22:45.151889 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:45.151853    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-controller/0.log"
Jul 25 02:22:45.166973 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:45.166889    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/0.log"
Jul 25 02:22:45.166973 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:45.166922    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/ovn-acl-logging/1.log"
Jul 25 02:22:45.184281 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:45.184248    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/kube-rbac-proxy-node/0.log"
Jul 25 02:22:45.203563 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:45.203527    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/kube-rbac-proxy-ovn-metrics/0.log"
Jul 25 02:22:45.223029 ip-10-0-134-238 kubenswrapper[2576]: I0725 02:22:45.223007    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcf67_bf4b06fe-fa8a-4281-8995-d92fb33486fc/northd/0.log"