Sep 05 06:26:23.298216 ip-10-0-130-170 systemd[1]: Starting Kubernetes Kubelet...
Sep 05 06:26:23.671676 ip-10-0-130-170 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 05 06:26:23.671676 ip-10-0-130-170 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 05 06:26:23.671676 ip-10-0-130-170 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 05 06:26:23.671676 ip-10-0-130-170 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 05 06:26:23.671676 ip-10-0-130-170 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 05 06:26:23.673268 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.673173    2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 05 06:26:23.676767 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676743    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 06:26:23.676767 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676767    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676771    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676775    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676781    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676784    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676789    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676792    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676797    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676800    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676804    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676815    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676821    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676824    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676827    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676834    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676838    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676841    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676845    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676848    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676853    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 06:26:23.676859 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676856    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676859    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676863    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676867    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676870    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676873    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676876    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676892    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676895    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676898    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676901    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676904    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676908    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676913    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676915    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676918    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676921    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676929    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676932    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676937    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 06:26:23.677406 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676939    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676944    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676947    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676950    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676955    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676959    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676963    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676966    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676969    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676972    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676975    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676978    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676980    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676984    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676988    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676991    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676993    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.676998    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677003    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677006    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 06:26:23.677994 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677009    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677012    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677015    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677018    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677021    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677024    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677026    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677030    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677033    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677036    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677039    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677043    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677045    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677049    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677051    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677055    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677057    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677064    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677070    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 06:26:23.678572 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677073    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677076    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677079    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677081    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677084    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677087    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677090    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677094    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677098    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677102    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677105    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677108    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677110    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677113    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677116    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677119    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677122    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677127    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 06:26:23.679120 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.677131    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679207    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679229    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679233    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679241    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679245    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679249    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679252    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679256    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679259    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679263    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679268    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679271    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679274    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679277    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679282    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679286    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679293    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679300    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679303    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 06:26:23.679621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679307    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679311    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679316    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679320    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679323    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679326    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679329    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679332    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679335    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679338    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679341    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679345    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679349    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679352    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679358    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679361    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679364    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679367    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679370    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 06:26:23.680378 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679374    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679377    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679380    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679382    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679385    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679388    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679391    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679394    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679397    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679399    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679402    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679405    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679408    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679411    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679413    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679416    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679419    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679422    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679425    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679427    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 06:26:23.680922 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679430    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679433    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679435    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679437    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679440    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679443    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679446    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679450    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679453    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679456    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679459    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679463    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679466    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679470    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679472    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679475    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679480    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679484    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679487    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679490    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 06:26:23.681467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679493    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679496    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679498    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679502    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679506    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679510    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679512    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679515    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679517    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679520    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679523    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679526    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679529    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679531    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679534    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679537    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679541    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679543    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679546    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 06:26:23.682058 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679549    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.679552    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680091    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680104    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680108    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680112    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680117    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680120    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680123    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680126    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680129    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680132    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680135    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680138    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680143    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680146    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680148    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680151    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680154    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680157    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 06:26:23.682570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680162    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680166    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680170    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680173    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680176    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680179    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680182    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680185    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680189    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680193    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680195    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680198    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680201    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680204    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680207    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680210    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680212    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680215    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680218    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680221    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 06:26:23.683141 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680224    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680226    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680229    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680232    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680234    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680237    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680240    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680242    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680245    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680248    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680250    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680254    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680256    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680259    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680262    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680265    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680267    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680270    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680273    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 06:26:23.683791 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680277    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680282    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680287    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680291    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680294    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680298    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680301    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680304    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680307    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680310    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680313    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680316    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680318    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680326    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680329    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680333    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680336    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680339    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680342    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 06:26:23.684322 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680345    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680348    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680351    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680354    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680357    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680360    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680363    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680365    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680369    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680373    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680376    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680380    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680383    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680387    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680390    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680394    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680398    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680402    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680406    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680409    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 06:26:23.684829 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680412    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680415    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.680418    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680561    2559 flags.go:64] FLAG: --address="0.0.0.0"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680620    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680628    2559 flags.go:64] FLAG: --anonymous-auth="true"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680633    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680639    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680643    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680648    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680653    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680656    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680660    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680664    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680668    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680672    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680676    2559 flags.go:64] FLAG: --cgroup-root=""
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680680    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680683    2559 flags.go:64] FLAG: --client-ca-file=""
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680687    2559 flags.go:64] FLAG: --cloud-provider="external"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680690    2559 flags.go:64] FLAG: --cluster-dns="[]"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680695    2559 flags.go:64] FLAG: --cluster-domain=""
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680698    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680702    2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 05 06:26:23.685439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680706    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680710    2559 flags.go:64] FLAG: --container-log-max-files="5"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680715    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680718    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680721    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680725    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680729    2559 flags.go:64] FLAG: --contention-profiling="false"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680731    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680735    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680738    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680741    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680745    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680753    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680756    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680760    2559 flags.go:64] FLAG: --enable-load-reader="false"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680763    2559 flags.go:64] FLAG: --enable-server="true"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680767    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680772    2559 flags.go:64] FLAG: --event-burst="100"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680775    2559 flags.go:64] FLAG: --event-qps="50"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680779    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680783    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680786    2559 flags.go:64] FLAG: --eviction-hard=""
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680790    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680793    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680797    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 05 06:26:23.686061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680800    2559 flags.go:64] FLAG: --eviction-soft=""
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680803    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680806    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680809    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680813    2559 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680816    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680819    2559 flags.go:64] FLAG: --fail-swap-on="true"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680822    2559 flags.go:64] FLAG: --feature-gates=""
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680826    2559 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680832    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680835    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680839    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680843    2559 flags.go:64] FLAG: --healthz-port="10248"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680846    2559 flags.go:64] FLAG: --help="false"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680850    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-130-170.ec2.internal"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680854    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680857    2559 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680860    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680864    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680870    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680873    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680877    2559 flags.go:64] FLAG: --image-service-endpoint=""
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680896    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 05 06:26:23.686669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680901    2559 flags.go:64] FLAG: --kube-api-burst="100"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680905    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680909    2559 flags.go:64] FLAG: --kube-api-qps="50"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680913    2559 flags.go:64] FLAG: --kube-reserved=""
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680916    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680919    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680923    2559 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680926    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680930    2559 flags.go:64] FLAG: --lock-file=""
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680933    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680936    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680940    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680947    2559 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680951    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680954    2559 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680958    2559 flags.go:64] FLAG: --logging-format="text"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680961    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680965    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680972    2559 flags.go:64] FLAG: --manifest-url=""
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680976    2559 flags.go:64] FLAG: --manifest-url-header=""
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680982    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680985    2559 flags.go:64] FLAG: --max-open-files="1000000"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680991    2559 flags.go:64] FLAG: --max-pods="110"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680994    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.680998    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 05 06:26:23.687276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681002    2559 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681006    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681009    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681014    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681018    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681030    2559 flags.go:64] FLAG: --node-status-max-images="50"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681033    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681036    2559 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681039    2559 flags.go:64] FLAG: --pod-cidr=""
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681043    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681050    2559 flags.go:64] FLAG: --pod-manifest-path=""
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681054    2559 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681057    2559 flags.go:64] FLAG: --pods-per-core="0"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681062    2559 flags.go:64] FLAG: --port="10250"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681065    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681069    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-069ceefeb9f061be7"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681073    2559 flags.go:64] FLAG: --qos-reserved=""
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681076    2559 flags.go:64] FLAG: --read-only-port="10255"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681080    2559 flags.go:64] FLAG: --register-node="true"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681083    2559 flags.go:64] FLAG: --register-with-taints=""
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681087    2559 flags.go:64] FLAG: --registry-burst="10"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681090    2559 flags.go:64] FLAG: --registry-qps="5"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681093    2559 flags.go:64] FLAG: --reserved-cpus=""
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681096    2559 flags.go:64] FLAG: --reserved-memory=""
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681100    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 05 06:26:23.687927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681104    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681111    2559 flags.go:64] FLAG: --rotate-certificates="false"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681114    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681118    2559 flags.go:64] FLAG: --runonce="false"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681121    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681124    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681128    2559 flags.go:64] FLAG: --seccomp-default="false"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681132    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681136    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681139    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681145    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681149    2559 flags.go:64] FLAG: --storage-driver-password="root"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681152    2559 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681155    2559 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681158    2559 flags.go:64] FLAG: --storage-driver-user="root"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681161    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681165    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681170    2559 flags.go:64] FLAG: --system-cgroups=""
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681173    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681180    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681184    2559 flags.go:64] FLAG: --tls-cert-file=""
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681187    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681193    2559 flags.go:64] FLAG: --tls-min-version=""
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681195    2559 flags.go:64] FLAG: --tls-private-key-file=""
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681199    2559 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 05 06:26:23.688551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681202    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681205    2559 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681209    2559 flags.go:64] FLAG: --v="2"
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681215    2559 flags.go:64] FLAG: --version="false"
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681221    2559 flags.go:64] FLAG: --vmodule=""
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681225    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681229    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681402    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681415    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681420    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681426    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681429    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681432    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681436    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681439    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681441    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681445    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681450    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681454    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681458    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681461    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681463    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 06:26:23.689272 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681467    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681470    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681473    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681477    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681481    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681484    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681486    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681490    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681493    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681496    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681498    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681501    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681504    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681507    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681512    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681516    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681519    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681522    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681526    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 06:26:23.689832 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681528    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681531    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681533    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681536    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681539    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681542    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681544    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681547    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681550    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681553    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681555    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681558    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681560    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681563    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681565    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681568    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681571    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681581    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681584    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681587    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 06:26:23.690337 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681589    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681592    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681595    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681597    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681600    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681603    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681606    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681608    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681611    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681613    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681616    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681619    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681622    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681626    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681630    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681634    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681638    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681641    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681643    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 06:26:23.690843 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681646    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681649    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681651    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681654    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681657    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681659    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681662    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681665    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681668    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681671    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681674    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681678    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681681    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681685    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681688    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681690    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681693    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681696    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681698    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681701    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 06:26:23.691328 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681703    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 06:26:23.691854 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681706    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 06:26:23.691854 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681711    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 06:26:23.691854 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681715    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 06:26:23.691854 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681718    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 06:26:23.691854 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.681722    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 06:26:23.691854 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.681729    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 05 06:26:23.691854 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.691187    2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 05 06:26:23.691854 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.691212    2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 05 06:26:23.692075 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.691967    2559 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 05 06:26:23.692075 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.691977    2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 05 06:26:23.692075 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692048    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 06:26:23.692075 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692055    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 06:26:23.692075 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692058    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 06:26:23.692075 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692061    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 06:26:23.692075 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692064    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 06:26:23.692075 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692067    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 06:26:23.692075 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692070    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 06:26:23.692075 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692074    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 06:26:23.692075 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692077    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 06:26:23.692075 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692080    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692083    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692086    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692088    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692093    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692096    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692098    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692101    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692104    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692106    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692109    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692112    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692115    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692117    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692120    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692123    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692126    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692128    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692131    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692134    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 06:26:23.692416 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692146    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692149    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692151    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692154    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692157    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692159    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692162    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692164    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692167    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692169    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692172    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692175    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692177    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692179    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692183    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692187    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692191    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692194    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692197    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 06:26:23.693013 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692199    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692202    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692205    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692208    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692210    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692214    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692216    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692219    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692222    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692226    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692229    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692232    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692235    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692238    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692246    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692249    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692251    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692254    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692257    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 06:26:23.693488 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692259    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692262    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692265    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692267    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692270    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692274    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692277    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692281    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692285    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692288    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692291    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692294    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692297    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692300    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692303    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692305    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692308    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692311    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692314    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692316    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 06:26:23.693976 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692319    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 06:26:23.694467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692321    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 06:26:23.694467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692324    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 06:26:23.694467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692327    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 06:26:23.694467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692329    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 06:26:23.694467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692332    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 06:26:23.694467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692334    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 06:26:23.694467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692337    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 06:26:23.694467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692346    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 06:26:23.694467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692350    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 06:26:23.694467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692352    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 06:26:23.694467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692354    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 06:26:23.694467 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.692360    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 05 06:26:23.694467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692524    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 05 06:26:23.694467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692530    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 05 06:26:23.694467 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692533    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692536    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692539    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692542    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692545    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692548    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692551    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692554    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692557    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692559    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692562    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692564    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692567    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692569    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692572    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692575    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692577    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692580    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692582    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692585    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 05 06:26:23.694852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692588    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692590    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692593    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692595    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692598    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692600    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692609    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692611    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692614    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692617    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692619    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692622    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692625    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692627    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692632    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692636    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692639    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692642    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692644    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 05 06:26:23.695463 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692647    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692650    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692652    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692655    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692658    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692660    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692663    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692666    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692670    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692673    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692676    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692679    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692681    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692684    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692686    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692689    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692692    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692694    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692697    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 05 06:26:23.695967 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692699    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692709    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692712    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692714    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692717    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692720    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692722    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692725    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692728    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692731    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692734    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692736    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692739    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692741    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692744    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692747    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692749    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692752    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692755    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 05 06:26:23.696560 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692757    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692760    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692762    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692765    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692767    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692769    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692772    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692774    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692777    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692780    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692782    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692785    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692787    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692790    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692792    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692801    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692804    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692808    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692811    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 05 06:26:23.697363 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:23.692814    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 05 06:26:23.698053 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.692819    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 05 06:26:23.698053 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.693026    2559 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 05 06:26:23.698053 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.695000    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 05 06:26:23.698053 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.696191    2559 server.go:1019] "Starting client certificate rotation"
Sep 05 06:26:23.698053 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.696313    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 05 06:26:23.698053 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.696369    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 05 06:26:23.698678 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.698658    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 05 06:26:23.702950 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.702918    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 05 06:26:23.716428 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.716406    2559 log.go:25] "Validated CRI v1 runtime API"
Sep 05 06:26:23.719475 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.719461    2559 log.go:25] "Validated CRI v1 image API"
Sep 05 06:26:23.720372 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.720357    2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 05 06:26:23.723869 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.723842    2559 fs.go:135] Filesystem UUIDs: map[1b40afa0-b2f0-4fad-80e0-375d86d51932:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a4ebc638-855c-4763-a2fc-faa9fef1b09f:/dev/nvme0n1p4]
Sep 05 06:26:23.723973 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.723865    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 05 06:26:23.728031 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.728008    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 05 06:26:23.729414 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.729303    2559 manager.go:216] Machine: {Timestamp:2026-09-05 06:26:23.728027454 +0000 UTC m=+0.318586397 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103847 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21fa8ced1bdc1c32bb86ca9a573992 SystemUUID:ec21fa8c-ed1b-dc1c-32bb-86ca9a573992 BootID:0a3d1756-835b-4322-9580-60ab18b0de4d Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2a:55:54:93:1d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2a:55:54:93:1d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9a:40:3b:e2:2f:c3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 05 06:26:23.729414 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.729408    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 05 06:26:23.729532 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.729502    2559 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 05 06:26:23.732344 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.732318    2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 05 06:26:23.732504 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.732345    2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-170.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 05 06:26:23.733279 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.733268    2559 topology_manager.go:138] "Creating topology manager with none policy"
Sep 05 06:26:23.733317 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.733282    2559 container_manager_linux.go:309] "Creating device plugin manager"
Sep 05 06:26:23.733317 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.733291    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 05 06:26:23.734019 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.734009    2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 05 06:26:23.734703 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.734694    2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 05 06:26:23.737702 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.737691    2559 state_mem.go:36] "Initialized new in-memory state store"
Sep 05 06:26:23.737823 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.737814    2559 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 05 06:26:23.737866 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.737858    2559 kubelet.go:486] "Attempting to sync node with API server"
Sep 05 06:26:23.737910 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.737871    2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 05 06:26:23.737910 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.737896    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 05 06:26:23.737910 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.737911    2559 kubelet.go:398] "Adding apiserver pod source"
Sep 05 06:26:23.738042 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.737920    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 05 06:26:23.738345 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.738327    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 05 06:26:23.738444 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.738353    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 05 06:26:23.741005 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.740985    2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 05 06:26:23.743107 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.743089    2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 05 06:26:23.743175 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.743122    2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 05 06:26:23.744125 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.744112    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 05 06:26:23.744125 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.744133    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 05 06:26:23.744214 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.744141    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 05 06:26:23.744214 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.744149    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 05 06:26:23.744214 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.744160    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 05 06:26:23.744214 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.744166    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 05 06:26:23.744214 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.744179    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 05 06:26:23.744214 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.744184    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 05 06:26:23.744214 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.744190    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 05 06:26:23.744214 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.744197    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 05 06:26:23.744214 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.744205    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 05 06:26:23.744451 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.744219    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 05 06:26:23.744920 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.744910    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 05 06:26:23.744920 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.744919    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 05 06:26:23.748875 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.748855    2559 server.go:1268] "Started kubelet"
Sep 05 06:26:23.748989 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.748922    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 05 06:26:23.749046 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.749006    2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 05 06:26:23.749087 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.749072    2559 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 05 06:26:23.749326 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.749309    2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-170.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 05 06:26:23.752328 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.752308    2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 05 06:26:23.752555 ip-10-0-130-170 systemd[1]: Started Kubernetes Kubelet.
Sep 05 06:26:23.752960 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:23.752931    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-170.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 05 06:26:23.753924 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:23.753796    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 05 06:26:23.754919 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.754899    2559 server.go:310] "Adding debug handlers to kubelet server"
Sep 05 06:26:23.756923 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:23.756874    2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 05 06:26:23.759715 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:23.758792    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-170.ec2.internal.18d258c0044f2f90  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-170.ec2.internal,UID:ip-10-0-130-170.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-170.ec2.internal,},FirstTimestamp:2026-09-05 06:26:23.748771728 +0000 UTC m=+0.339330671,LastTimestamp:2026-09-05 06:26:23.748771728 +0000 UTC m=+0.339330671,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-170.ec2.internal,}"
Sep 05 06:26:23.760045 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.760021    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 05 06:26:23.760493 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.760474    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 05 06:26:23.760688 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:23.760667    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-170.ec2.internal\" not found"
Sep 05 06:26:23.761334 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.761318    2559 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 05 06:26:23.761334 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.761335    2559 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 05 06:26:23.761334 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.761322    2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 05 06:26:23.761504 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.761426    2559 reconstruct.go:97] "Volume reconstruction finished"
Sep 05 06:26:23.761504 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.761434    2559 reconciler.go:29] "Reconciler: start to sync state"
Sep 05 06:26:23.761571 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.761521    2559 factory.go:55] Registering systemd factory
Sep 05 06:26:23.761571 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.761537    2559 factory.go:223] Registration of the systemd container factory successfully
Sep 05 06:26:23.761786 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.761770    2559 factory.go:153] Registering CRI-O factory
Sep 05 06:26:23.761786 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.761788    2559 factory.go:223] Registration of the crio container factory successfully
Sep 05 06:26:23.761919 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.761839    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 05 06:26:23.761919 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.761863    2559 factory.go:103] Registering Raw factory
Sep 05 06:26:23.761919 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.761897    2559 manager.go:1196] Started watching for new ooms in manager
Sep 05 06:26:23.761919 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.761902    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nbwf2"
Sep 05 06:26:23.762309 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.762297    2559 manager.go:349] Starting recovery of all containers
Sep 05 06:26:23.769348 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:23.769305    2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-170.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 05 06:26:23.769348 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.769333    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 05 06:26:23.769568 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:23.769543    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 05 06:26:23.771039 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.771015    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nbwf2"
Sep 05 06:26:23.772874 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.772854    2559 manager.go:354] Recovery completed
Sep 05 06:26:23.778127 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.778115    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 06:26:23.780191 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.780172    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 06:26:23.780262 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.780205    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 06:26:23.780262 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.780215    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasSufficientPID"
Sep 05 06:26:23.780852 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.780841    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 05 06:26:23.780904 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.780852    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 05 06:26:23.780904 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.780860    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 06:26:23.780904 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.780869    2559 state_mem.go:36] "Initialized new in-memory state store"
Sep 05 06:26:23.781635 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.781620    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 06:26:23.781717 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.781649    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 06:26:23.781717 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.781663    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasSufficientPID"
Sep 05 06:26:23.782071 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.782057    2559 policy_none.go:49] "None policy: Start"
Sep 05 06:26:23.782105 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.782078    2559 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 05 06:26:23.782105 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.782091    2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 05 06:26:23.783221 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.783204    2559 policy_none.go:47] "Start"
Sep 05 06:26:23.783298 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:23.783162    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-170.ec2.internal.18d258c0062e91c4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-170.ec2.internal,UID:ip-10-0-130-170.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-170.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-170.ec2.internal,},FirstTimestamp:2026-09-05 06:26:23.780188612 +0000 UTC m=+0.370747566,LastTimestamp:2026-09-05 06:26:23.780188612 +0000 UTC m=+0.370747566,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-170.ec2.internal,}"
Sep 05 06:26:23.812751 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.812570    2559 manager.go:337] "Starting Device Plugin manager"
Sep 05 06:26:23.824973 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:23.812780    2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 05 06:26:23.824973 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.812794    2559 server.go:87] "Starting device plugin registration server"
Sep 05 06:26:23.824973 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.813145    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 05 06:26:23.824973 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.813157    2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 05 06:26:23.824973 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.813248    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 05 06:26:23.824973 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.813384    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 05 06:26:23.824973 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.813395    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 05 06:26:23.824973 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:23.814028    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 05 06:26:23.824973 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:23.814084    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-170.ec2.internal\" not found"
Sep 05 06:26:23.885187 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.885151    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 05 06:26:23.885333 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.885229    2559 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 05 06:26:23.885333 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.885258    2559 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 05 06:26:23.885430 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:23.885360    2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 05 06:26:23.890272 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.890252    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 06:26:23.913627 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.913604    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 06:26:23.914699 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.914680    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 06:26:23.914779 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.914716    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 06:26:23.914779 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.914726    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasSufficientPID"
Sep 05 06:26:23.914779 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.914757    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-170.ec2.internal"
Sep 05 06:26:23.927976 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.927908    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-170.ec2.internal"
Sep 05 06:26:23.927976 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:23.927931    2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-170.ec2.internal\": node \"ip-10-0-130-170.ec2.internal\" not found"
Sep 05 06:26:23.956176 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:23.956152    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-170.ec2.internal\" not found"
Sep 05 06:26:23.985443 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.985415    2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-170.ec2.internal"]
Sep 05 06:26:23.985572 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.985492    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 06:26:23.987075 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.987057    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 06:26:23.987141 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.987092    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 06:26:23.987141 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.987107    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasSufficientPID"
Sep 05 06:26:23.988411 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.988398    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 06:26:23.988521 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.988507    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal"
Sep 05 06:26:23.988615 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.988538    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 06:26:23.989217 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.989200    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 06:26:23.989217 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.989213    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 06:26:23.989360 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.989230    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 06:26:23.989360 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.989237    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 06:26:23.989360 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.989243    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasSufficientPID"
Sep 05 06:26:23.989360 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.989251    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasSufficientPID"
Sep 05 06:26:23.990607 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.990591    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-170.ec2.internal"
Sep 05 06:26:23.990688 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.990620    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 05 06:26:23.991633 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.991618    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasSufficientMemory"
Sep 05 06:26:23.991701 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.991650    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasNoDiskPressure"
Sep 05 06:26:23.991701 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:23.991666    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeHasSufficientPID"
Sep 05 06:26:24.015676 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.015648    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-170.ec2.internal\" not found" node="ip-10-0-130-170.ec2.internal"
Sep 05 06:26:24.020020 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.020001    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-170.ec2.internal\" not found" node="ip-10-0-130-170.ec2.internal"
Sep 05 06:26:24.056836 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.056787    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-170.ec2.internal\" not found"
Sep 05 06:26:24.062475 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.062447    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd0e8a512a9396826908b8226fb7b755-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal\" (UID: \"bd0e8a512a9396826908b8226fb7b755\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal"
Sep 05 06:26:24.062604 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.062494    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd0e8a512a9396826908b8226fb7b755-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal\" (UID: \"bd0e8a512a9396826908b8226fb7b755\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal"
Sep 05 06:26:24.062604 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.062536    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50db2b2cd4cf113e73677eb1ebb83d24-config\") pod \"kube-apiserver-proxy-ip-10-0-130-170.ec2.internal\" (UID: \"50db2b2cd4cf113e73677eb1ebb83d24\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-170.ec2.internal"
Sep 05 06:26:24.157736 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.157674    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-170.ec2.internal\" not found"
Sep 05 06:26:24.162988 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.162964    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd0e8a512a9396826908b8226fb7b755-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal\" (UID: \"bd0e8a512a9396826908b8226fb7b755\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal"
Sep 05 06:26:24.163095 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.162998    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd0e8a512a9396826908b8226fb7b755-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal\" (UID: \"bd0e8a512a9396826908b8226fb7b755\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal"
Sep 05 06:26:24.163095 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.163027    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50db2b2cd4cf113e73677eb1ebb83d24-config\") pod \"kube-apiserver-proxy-ip-10-0-130-170.ec2.internal\" (UID: \"50db2b2cd4cf113e73677eb1ebb83d24\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-170.ec2.internal"
Sep 05 06:26:24.163095 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.163078    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50db2b2cd4cf113e73677eb1ebb83d24-config\") pod \"kube-apiserver-proxy-ip-10-0-130-170.ec2.internal\" (UID: \"50db2b2cd4cf113e73677eb1ebb83d24\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-170.ec2.internal"
Sep 05 06:26:24.163095 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.163084    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd0e8a512a9396826908b8226fb7b755-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal\" (UID: \"bd0e8a512a9396826908b8226fb7b755\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal"
Sep 05 06:26:24.163241 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.163084    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bd0e8a512a9396826908b8226fb7b755-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal\" (UID: \"bd0e8a512a9396826908b8226fb7b755\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal"
Sep 05 06:26:24.258492 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.258404    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-170.ec2.internal\" not found"
Sep 05 06:26:24.317480 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.317446    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal"
Sep 05 06:26:24.323817 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.323795    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-170.ec2.internal"
Sep 05 06:26:24.358578 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.358532    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-170.ec2.internal\" not found"
Sep 05 06:26:24.465359 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.465299    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-170.ec2.internal\" not found"
Sep 05 06:26:24.566036 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.565943    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-170.ec2.internal\" not found"
Sep 05 06:26:24.624573 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.624539    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 06:26:24.662759 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.662708    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal"
Sep 05 06:26:24.684062 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.684032    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 05 06:26:24.685929 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.685911    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-170.ec2.internal"
Sep 05 06:26:24.696373 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.696354    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 05 06:26:24.696548 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.696511    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 05 06:26:24.696548 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.696532    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 05 06:26:24.696548 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.696519    2559 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a0ef36a2aa03544bd81d5ca6ffaadd89-a687463c307ce165.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.130.170:44384->54.152.81.252:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-170.ec2.internal"
Sep 05 06:26:24.739693 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.739646    2559 apiserver.go:52] "Watching apiserver"
Sep 05 06:26:24.756126 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.756093    2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 05 06:26:24.756517 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.756496    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-jdbcm","kube-system/kube-apiserver-proxy-ip-10-0-130-170.ec2.internal","openshift-dns/node-resolver-zs2d7","openshift-image-registry/node-ca-j8x2c","openshift-multus/multus-ztv4g","openshift-multus/network-metrics-daemon-jslwg","openshift-network-operator/iptables-alerter-kp6pd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv","openshift-cluster-node-tuning-operator/tuned-92xkt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal","openshift-multus/multus-additional-cni-plugins-j4q5t","openshift-network-diagnostics/network-check-target-hsthh","openshift-ovn-kubernetes/ovnkube-node-vrv5q","kube-system/global-pull-secret-syncer-tcnw5"]
Sep 05 06:26:24.758218 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.758197    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j8x2c"
Sep 05 06:26:24.763029 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.759863    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.763029 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.761222    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-h7dgb\""
Sep 05 06:26:24.763029 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.761252    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 05 06:26:24.763029 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.761363    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 05 06:26:24.763029 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.761500    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 05 06:26:24.763029 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.761500    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 05 06:26:24.763029 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.761765    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.764171 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.763660    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 05 06:26:24.764171 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.763741    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-l9cqq\""
Sep 05 06:26:24.764171 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.764041    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 05 06:26:24.764171 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.764107    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 05 06:26:24.764441 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.764263    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kp6pd"
Sep 05 06:26:24.764441 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.764386    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 05 06:26:24.764556 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.764542    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 05 06:26:24.765016 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.764994    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-njcnk\""
Sep 05 06:26:24.765392 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.765373    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.766328 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766310    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c806dc77-8c41-4bbb-95a6-718556fc41eb-host-slash\") pod \"iptables-alerter-kp6pd\" (UID: \"c806dc77-8c41-4bbb-95a6-718556fc41eb\") " pod="openshift-network-operator/iptables-alerter-kp6pd"
Sep 05 06:26:24.766408 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766337    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/561b2ddc-4c98-434e-995b-a8338ac6cfd5-host\") pod \"node-ca-j8x2c\" (UID: \"561b2ddc-4c98-434e-995b-a8338ac6cfd5\") " pod="openshift-image-registry/node-ca-j8x2c"
Sep 05 06:26:24.766408 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766356    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-sysconfig\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.766408 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766378    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-systemd\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.766408 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766394    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-host\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.766547 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766410    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-sys-fs\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.766547 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766458    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6qh4\" (UniqueName: \"kubernetes.io/projected/561b2ddc-4c98-434e-995b-a8338ac6cfd5-kube-api-access-k6qh4\") pod \"node-ca-j8x2c\" (UID: \"561b2ddc-4c98-434e-995b-a8338ac6cfd5\") " pod="openshift-image-registry/node-ca-j8x2c"
Sep 05 06:26:24.766547 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766496    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kww8j\" (UniqueName: \"kubernetes.io/projected/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-kube-api-access-kww8j\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.766547 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766528    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-lib-modules\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.766673 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766558    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-tuned\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.766673 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766583    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.766673 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766610    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kfvq\" (UniqueName: \"kubernetes.io/projected/57a89e6a-809b-466d-9857-9a62db191f39-kube-api-access-2kfvq\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.766673 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766641    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-sysctl-d\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.766673 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766665    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-run\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.766848 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766719    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-kubernetes\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.766848 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766745    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-device-dir\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.766848 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766769    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-etc-selinux\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.766848 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766803    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-var-lib-kubelet\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.766848 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766827    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-sysctl-conf\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.766848 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766842    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-tmp\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.767253 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766878    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-socket-dir\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.767253 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766916    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-registration-dir\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.767253 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766969    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c806dc77-8c41-4bbb-95a6-718556fc41eb-iptables-alerter-script\") pod \"iptables-alerter-kp6pd\" (UID: \"c806dc77-8c41-4bbb-95a6-718556fc41eb\") " pod="openshift-network-operator/iptables-alerter-kp6pd"
Sep 05 06:26:24.767253 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.766991    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cstf8\" (UniqueName: \"kubernetes.io/projected/c806dc77-8c41-4bbb-95a6-718556fc41eb-kube-api-access-cstf8\") pod \"iptables-alerter-kp6pd\" (UID: \"c806dc77-8c41-4bbb-95a6-718556fc41eb\") " pod="openshift-network-operator/iptables-alerter-kp6pd"
Sep 05 06:26:24.767253 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.767030    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/561b2ddc-4c98-434e-995b-a8338ac6cfd5-serviceca\") pod \"node-ca-j8x2c\" (UID: \"561b2ddc-4c98-434e-995b-a8338ac6cfd5\") " pod="openshift-image-registry/node-ca-j8x2c"
Sep 05 06:26:24.767253 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.767055    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-modprobe-d\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.767253 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.767072    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-sys\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.767253 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.767122    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:24.767253 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.767175    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:24.767750 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.767729    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 05 06:26:24.768063 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.768043    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 05 06:26:24.768179 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.768097    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 05 06:26:24.768266 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.768245    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 05 06:26:24.768790 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.768771    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 05 06:26:24.768927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.768794    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 05 06:26:24.768927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.768832    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 05 06:26:24.768927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.768848    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kz7bm\""
Sep 05 06:26:24.768927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.768904    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7pqgb\""
Sep 05 06:26:24.769208 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.769188    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jdbcm"
Sep 05 06:26:24.771283 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.771266    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zs2d7"
Sep 05 06:26:24.771783 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.771765    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2gjp5\""
Sep 05 06:26:24.771846 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.771765    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 05 06:26:24.771846 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.771829    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 05 06:26:24.773588 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.773572    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 05 06:26:24.773986 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.773971    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.774299 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.774257    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-04 06:21:24 +0000 UTC" deadline="2028-04-11 14:48:43.48553058 +0000 UTC"
Sep 05 06:26:24.774299 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.774296    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14024h22m18.711238081s"
Sep 05 06:26:24.774547 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.774532    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 05 06:26:24.774621 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.774558    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lv4l9\""
Sep 05 06:26:24.776185 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.776169    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 05 06:26:24.776366 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.776354    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gsqzl\""
Sep 05 06:26:24.776453 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.776438    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 05 06:26:24.776781 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.776764    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 05 06:26:24.776853 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.776845    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:24.777107 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.776906    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:24.779816 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.779780    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.782696 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.782677    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:24.782794 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.782733    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:24.782873 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.782819    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 05 06:26:24.782873 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.782850    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 05 06:26:24.782971 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.782931    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8x8kl\""
Sep 05 06:26:24.783062 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.783048    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 05 06:26:24.783832 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.783814    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 05 06:26:24.783905 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.783848    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 05 06:26:24.784418 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.784404    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 05 06:26:24.802424 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.802405    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zvvtq"
Sep 05 06:26:24.812794 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.812775    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zvvtq"
Sep 05 06:26:24.862766 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.862746    2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 05 06:26:24.867350 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867331    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-run-netns\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.867432 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867359    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-var-lib-cni-bin\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.867432 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867380    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-run-multus-certs\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.867432 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867398    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs\") pod \"network-metrics-daemon-jslwg\" (UID: \"99bf56fa-eafd-489b-9d5d-75f31b076c45\") " pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:24.867432 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867421    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-run-netns\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.867589 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867460    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret\") pod \"global-pull-secret-syncer-tcnw5\" (UID: \"c85b9378-0915-419f-9b29-47bb580dc22a\") " pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:24.867589 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867502    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-var-lib-kubelet\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.867589 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867524    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a357932-d881-47a6-b6d9-ec8dab12d087-system-cni-dir\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.867589 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867546    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-sysctl-conf\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.867759 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867601    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-var-lib-kubelet\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.867759 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867612    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-cni-binary-copy\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.867759 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867664    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-run-k8s-cni-cncf-io\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.867759 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867698    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-multus-daemon-config\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.867759 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867711    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-sysctl-conf\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.867759 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867718    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db932df4-aa61-4dd9-b590-ba22dabce334-tmp-dir\") pod \"node-resolver-zs2d7\" (UID: \"db932df4-aa61-4dd9-b590-ba22dabce334\") " pod="openshift-dns/node-resolver-zs2d7"
Sep 05 06:26:24.867999 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867745    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a357932-d881-47a6-b6d9-ec8dab12d087-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.867999 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867796    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-slash\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.867999 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867816    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-run-ovn-kubernetes\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.867999 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867834    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-registration-dir\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.867999 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867851    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cstf8\" (UniqueName: \"kubernetes.io/projected/c806dc77-8c41-4bbb-95a6-718556fc41eb-kube-api-access-cstf8\") pod \"iptables-alerter-kp6pd\" (UID: \"c806dc77-8c41-4bbb-95a6-718556fc41eb\") " pod="openshift-network-operator/iptables-alerter-kp6pd"
Sep 05 06:26:24.867999 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867873    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-modprobe-d\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.867999 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867928    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-host\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.867999 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867939    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-registration-dir\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.867999 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867951    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-sys-fs\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.867999 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867979    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-hostroot\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.867999 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867993    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-host\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.868587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.867997    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42g8k\" (UniqueName: \"kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k\") pod \"network-check-target-hsthh\" (UID: \"9c9536b7-5174-42f5-981d-1a6d1a14317e\") " pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:24.868587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868048    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-etc-openvswitch\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.868587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868050    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-modprobe-d\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.868587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868055    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-sys-fs\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.868587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868083    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c806dc77-8c41-4bbb-95a6-718556fc41eb-host-slash\") pod \"iptables-alerter-kp6pd\" (UID: \"c806dc77-8c41-4bbb-95a6-718556fc41eb\") " pod="openshift-network-operator/iptables-alerter-kp6pd"
Sep 05 06:26:24.868587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868102    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/561b2ddc-4c98-434e-995b-a8338ac6cfd5-host\") pod \"node-ca-j8x2c\" (UID: \"561b2ddc-4c98-434e-995b-a8338ac6cfd5\") " pod="openshift-image-registry/node-ca-j8x2c"
Sep 05 06:26:24.868587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868121    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-systemd\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.868587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868153    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2d9b77a8-583d-4d7e-b62e-60fe17d73f24-konnectivity-ca\") pod \"konnectivity-agent-jdbcm\" (UID: \"2d9b77a8-583d-4d7e-b62e-60fe17d73f24\") " pod="kube-system/konnectivity-agent-jdbcm"
Sep 05 06:26:24.868587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868168    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c806dc77-8c41-4bbb-95a6-718556fc41eb-host-slash\") pod \"iptables-alerter-kp6pd\" (UID: \"c806dc77-8c41-4bbb-95a6-718556fc41eb\") " pod="openshift-network-operator/iptables-alerter-kp6pd"
Sep 05 06:26:24.868587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868182    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27svw\" (UniqueName: \"kubernetes.io/projected/db932df4-aa61-4dd9-b590-ba22dabce334-kube-api-access-27svw\") pod \"node-resolver-zs2d7\" (UID: \"db932df4-aa61-4dd9-b590-ba22dabce334\") " pod="openshift-dns/node-resolver-zs2d7"
Sep 05 06:26:24.868587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868183    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/561b2ddc-4c98-434e-995b-a8338ac6cfd5-host\") pod \"node-ca-j8x2c\" (UID: \"561b2ddc-4c98-434e-995b-a8338ac6cfd5\") " pod="openshift-image-registry/node-ca-j8x2c"
Sep 05 06:26:24.868587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868212    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf8nf\" (UniqueName: \"kubernetes.io/projected/0a357932-d881-47a6-b6d9-ec8dab12d087-kube-api-access-pf8nf\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.868587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868217    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-systemd\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.868587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868254    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-systemd-units\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.868587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868283    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-cni-bin\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.868587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868314    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kww8j\" (UniqueName: \"kubernetes.io/projected/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-kube-api-access-kww8j\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.868587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868339    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a357932-d881-47a6-b6d9-ec8dab12d087-cni-binary-copy\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.869256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868369    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a357932-d881-47a6-b6d9-ec8dab12d087-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.869256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868397    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.869256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868427    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-lib-modules\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.869256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868454    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a357932-d881-47a6-b6d9-ec8dab12d087-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.869256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868480    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-cni-netd\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.869256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868506    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvxzf\" (UniqueName: \"kubernetes.io/projected/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-kube-api-access-lvxzf\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.869256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868532    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-sysctl-d\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.869256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868554    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-lib-modules\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.869256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868566    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2d9b77a8-583d-4d7e-b62e-60fe17d73f24-agent-certs\") pod \"konnectivity-agent-jdbcm\" (UID: \"2d9b77a8-583d-4d7e-b62e-60fe17d73f24\") " pod="kube-system/konnectivity-agent-jdbcm"
Sep 05 06:26:24.869256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868592    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-run-ovn\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.869256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868610    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-ovnkube-config\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.869256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868636    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-kubernetes\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.869256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868660    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-device-dir\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.869256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868687    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl2q7\" (UniqueName: \"kubernetes.io/projected/99bf56fa-eafd-489b-9d5d-75f31b076c45-kube-api-access-rl2q7\") pod \"network-metrics-daemon-jslwg\" (UID: \"99bf56fa-eafd-489b-9d5d-75f31b076c45\") " pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:24.869256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868687    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-sysctl-d\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.869256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868745    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-kubernetes\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.870085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868746    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-ovn-node-metrics-cert\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.870085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868780    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-var-lib-cni-multus\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.870085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868792    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-device-dir\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.870085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868798    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/db932df4-aa61-4dd9-b590-ba22dabce334-hosts-file\") pod \"node-resolver-zs2d7\" (UID: \"db932df4-aa61-4dd9-b590-ba22dabce334\") " pod="openshift-dns/node-resolver-zs2d7"
Sep 05 06:26:24.870085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868843    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-node-log\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.870085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868868    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-system-cni-dir\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.870085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868884    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-var-lib-kubelet\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.870085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868922    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-kubelet\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.870085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868939    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-log-socket\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.870085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868953    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-multus-conf-dir\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.870085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.868973    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-tmp\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.870085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869028    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-socket-dir\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.870085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869045    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c806dc77-8c41-4bbb-95a6-718556fc41eb-iptables-alerter-script\") pod \"iptables-alerter-kp6pd\" (UID: \"c806dc77-8c41-4bbb-95a6-718556fc41eb\") " pod="openshift-network-operator/iptables-alerter-kp6pd"
Sep 05 06:26:24.870085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869165    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/561b2ddc-4c98-434e-995b-a8338ac6cfd5-serviceca\") pod \"node-ca-j8x2c\" (UID: \"561b2ddc-4c98-434e-995b-a8338ac6cfd5\") " pod="openshift-image-registry/node-ca-j8x2c"
Sep 05 06:26:24.870085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869179    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-socket-dir\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.870085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869198    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-sys\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.870085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869272    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-sys\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.870757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869271    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-multus-socket-dir-parent\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.870757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869311    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c85b9378-0915-419f-9b29-47bb580dc22a-dbus\") pod \"global-pull-secret-syncer-tcnw5\" (UID: \"c85b9378-0915-419f-9b29-47bb580dc22a\") " pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:24.870757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869346    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-sysconfig\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.870757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869374    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-var-lib-openvswitch\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.870757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869403    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-run-openvswitch\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.870757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869414    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-sysconfig\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.870757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869431    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6qh4\" (UniqueName: \"kubernetes.io/projected/561b2ddc-4c98-434e-995b-a8338ac6cfd5-kube-api-access-k6qh4\") pod \"node-ca-j8x2c\" (UID: \"561b2ddc-4c98-434e-995b-a8338ac6cfd5\") " pod="openshift-image-registry/node-ca-j8x2c"
Sep 05 06:26:24.870757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869461    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-multus-cni-dir\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.870757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869439    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 05 06:26:24.870757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869490    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-etc-kubernetes\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.870757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869590    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8lwn\" (UniqueName: \"kubernetes.io/projected/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-kube-api-access-g8lwn\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.870757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869622    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-env-overrides\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.870757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869660    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-ovnkube-script-lib\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.870757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869686    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c806dc77-8c41-4bbb-95a6-718556fc41eb-iptables-alerter-script\") pod \"iptables-alerter-kp6pd\" (UID: \"c806dc77-8c41-4bbb-95a6-718556fc41eb\") " pod="openshift-network-operator/iptables-alerter-kp6pd"
Sep 05 06:26:24.870757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869692    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-tuned\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.870757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869751    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.870757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869807    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kfvq\" (UniqueName: \"kubernetes.io/projected/57a89e6a-809b-466d-9857-9a62db191f39-kube-api-access-2kfvq\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.871402 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869842    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a357932-d881-47a6-b6d9-ec8dab12d087-cnibin\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.871402 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869867    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-run-systemd\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.871402 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869920    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c85b9378-0915-419f-9b29-47bb580dc22a-kubelet-config\") pod \"global-pull-secret-syncer-tcnw5\" (UID: \"c85b9378-0915-419f-9b29-47bb580dc22a\") " pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:24.871402 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869959    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-run\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.871402 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.869988    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-cnibin\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.871402 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.870051    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-run\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.871402 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.870116    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-etc-selinux\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.871402 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.870108    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.871402 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.870167    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/561b2ddc-4c98-434e-995b-a8338ac6cfd5-serviceca\") pod \"node-ca-j8x2c\" (UID: \"561b2ddc-4c98-434e-995b-a8338ac6cfd5\") " pod="openshift-image-registry/node-ca-j8x2c"
Sep 05 06:26:24.871402 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.870183    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/57a89e6a-809b-466d-9857-9a62db191f39-etc-selinux\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.873633 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.873608    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-tmp\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.873768 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.873654    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-etc-tuned\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.881847 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.881818    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kww8j\" (UniqueName: \"kubernetes.io/projected/b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a-kube-api-access-kww8j\") pod \"tuned-92xkt\" (UID: \"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a\") " pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:24.882011 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.881820    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6qh4\" (UniqueName: \"kubernetes.io/projected/561b2ddc-4c98-434e-995b-a8338ac6cfd5-kube-api-access-k6qh4\") pod \"node-ca-j8x2c\" (UID: \"561b2ddc-4c98-434e-995b-a8338ac6cfd5\") " pod="openshift-image-registry/node-ca-j8x2c"
Sep 05 06:26:24.882245 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.882228    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kfvq\" (UniqueName: \"kubernetes.io/projected/57a89e6a-809b-466d-9857-9a62db191f39-kube-api-access-2kfvq\") pod \"aws-ebs-csi-driver-node-tdjrv\" (UID: \"57a89e6a-809b-466d-9857-9a62db191f39\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:24.882293 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.882239    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cstf8\" (UniqueName: \"kubernetes.io/projected/c806dc77-8c41-4bbb-95a6-718556fc41eb-kube-api-access-cstf8\") pod \"iptables-alerter-kp6pd\" (UID: \"c806dc77-8c41-4bbb-95a6-718556fc41eb\") " pod="openshift-network-operator/iptables-alerter-kp6pd"
Sep 05 06:26:24.971053 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.970998    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-run-netns\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.971053 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971040    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-var-lib-cni-bin\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.971336 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971071    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-run-multus-certs\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.971336 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971105    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs\") pod \"network-metrics-daemon-jslwg\" (UID: \"99bf56fa-eafd-489b-9d5d-75f31b076c45\") " pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:24.971336 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971117    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-run-netns\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.971336 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971118    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-var-lib-cni-bin\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.971336 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971124    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-run-netns\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.971336 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971170    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-run-multus-certs\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.971336 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971175    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret\") pod \"global-pull-secret-syncer-tcnw5\" (UID: \"c85b9378-0915-419f-9b29-47bb580dc22a\") " pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:24.971336 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971214    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a357932-d881-47a6-b6d9-ec8dab12d087-system-cni-dir\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.971336 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.971222    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:24.971336 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.971235    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:24.971336 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971240    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-cni-binary-copy\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.971336 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.971297    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs podName:99bf56fa-eafd-489b-9d5d-75f31b076c45 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:25.471271021 +0000 UTC m=+2.061829970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs") pod "network-metrics-daemon-jslwg" (UID: "99bf56fa-eafd-489b-9d5d-75f31b076c45") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:24.971336 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971302    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a357932-d881-47a6-b6d9-ec8dab12d087-system-cni-dir\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.971336 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971293    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-run-netns\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.971336 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971351    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-run-k8s-cni-cncf-io\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.972176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971387    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-run-k8s-cni-cncf-io\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.972176 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.971366    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret podName:c85b9378-0915-419f-9b29-47bb580dc22a nodeName:}" failed. No retries permitted until 2026-09-05 06:26:25.471346201 +0000 UTC m=+2.061905143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret") pod "global-pull-secret-syncer-tcnw5" (UID: "c85b9378-0915-419f-9b29-47bb580dc22a") : object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:24.972176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971439    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-multus-daemon-config\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.972176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971468    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db932df4-aa61-4dd9-b590-ba22dabce334-tmp-dir\") pod \"node-resolver-zs2d7\" (UID: \"db932df4-aa61-4dd9-b590-ba22dabce334\") " pod="openshift-dns/node-resolver-zs2d7"
Sep 05 06:26:24.972176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971495    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a357932-d881-47a6-b6d9-ec8dab12d087-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.972176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971523    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-slash\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.972176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971550    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-run-ovn-kubernetes\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.972176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971582    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-hostroot\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.972176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971608    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42g8k\" (UniqueName: \"kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k\") pod \"network-check-target-hsthh\" (UID: \"9c9536b7-5174-42f5-981d-1a6d1a14317e\") " pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:24.972176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971627    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-slash\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.972176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971634    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-etc-openvswitch\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.972176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971679    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-etc-openvswitch\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.972176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971682    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2d9b77a8-583d-4d7e-b62e-60fe17d73f24-konnectivity-ca\") pod \"konnectivity-agent-jdbcm\" (UID: \"2d9b77a8-583d-4d7e-b62e-60fe17d73f24\") " pod="kube-system/konnectivity-agent-jdbcm"
Sep 05 06:26:24.972176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971715    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27svw\" (UniqueName: \"kubernetes.io/projected/db932df4-aa61-4dd9-b590-ba22dabce334-kube-api-access-27svw\") pod \"node-resolver-zs2d7\" (UID: \"db932df4-aa61-4dd9-b590-ba22dabce334\") " pod="openshift-dns/node-resolver-zs2d7"
Sep 05 06:26:24.972176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971721    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-run-ovn-kubernetes\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.972176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971751    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf8nf\" (UniqueName: \"kubernetes.io/projected/0a357932-d881-47a6-b6d9-ec8dab12d087-kube-api-access-pf8nf\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.972870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971804    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-systemd-units\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.972870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971822    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db932df4-aa61-4dd9-b590-ba22dabce334-tmp-dir\") pod \"node-resolver-zs2d7\" (UID: \"db932df4-aa61-4dd9-b590-ba22dabce334\") " pod="openshift-dns/node-resolver-zs2d7"
Sep 05 06:26:24.972870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971830    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-cni-bin\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.972870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971821    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-hostroot\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.972870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971852    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-cni-binary-copy\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.972870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971864    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a357932-d881-47a6-b6d9-ec8dab12d087-cni-binary-copy\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.972870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971908    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a357932-d881-47a6-b6d9-ec8dab12d087-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.972870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971966    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.972870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.971995    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a357932-d881-47a6-b6d9-ec8dab12d087-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.972870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972020    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-multus-daemon-config\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.972870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972027    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-cni-netd\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.972870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972059    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-cni-netd\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.972870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972081    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvxzf\" (UniqueName: \"kubernetes.io/projected/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-kube-api-access-lvxzf\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.972870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972092    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.972870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972114    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2d9b77a8-583d-4d7e-b62e-60fe17d73f24-agent-certs\") pod \"konnectivity-agent-jdbcm\" (UID: \"2d9b77a8-583d-4d7e-b62e-60fe17d73f24\") " pod="kube-system/konnectivity-agent-jdbcm"
Sep 05 06:26:24.972870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972146    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-run-ovn\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.972870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972173    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-ovnkube-config\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.973806 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972205    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl2q7\" (UniqueName: \"kubernetes.io/projected/99bf56fa-eafd-489b-9d5d-75f31b076c45-kube-api-access-rl2q7\") pod \"network-metrics-daemon-jslwg\" (UID: \"99bf56fa-eafd-489b-9d5d-75f31b076c45\") " pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:24.973806 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972238    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-ovn-node-metrics-cert\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.973806 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972272    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-var-lib-cni-multus\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.973806 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972300    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/db932df4-aa61-4dd9-b590-ba22dabce334-hosts-file\") pod \"node-resolver-zs2d7\" (UID: \"db932df4-aa61-4dd9-b590-ba22dabce334\") " pod="openshift-dns/node-resolver-zs2d7"
Sep 05 06:26:24.973806 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972326    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-node-log\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.973806 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972353    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-system-cni-dir\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.973806 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972379    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-var-lib-kubelet\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.973806 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972411    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-kubelet\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.973806 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972439    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-log-socket\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.973806 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972465    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-multus-conf-dir\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.973806 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972480    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a357932-d881-47a6-b6d9-ec8dab12d087-cni-binary-copy\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.973806 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972500    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-multus-socket-dir-parent\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.973806 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972529    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c85b9378-0915-419f-9b29-47bb580dc22a-dbus\") pod \"global-pull-secret-syncer-tcnw5\" (UID: \"c85b9378-0915-419f-9b29-47bb580dc22a\") " pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:24.973806 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972553    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2d9b77a8-583d-4d7e-b62e-60fe17d73f24-konnectivity-ca\") pod \"konnectivity-agent-jdbcm\" (UID: \"2d9b77a8-583d-4d7e-b62e-60fe17d73f24\") " pod="kube-system/konnectivity-agent-jdbcm"
Sep 05 06:26:24.973806 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972568    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-var-lib-openvswitch\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.973806 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972045    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a357932-d881-47a6-b6d9-ec8dab12d087-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.973806 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972591    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-cni-bin\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972599    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-run-openvswitch\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972639    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a357932-d881-47a6-b6d9-ec8dab12d087-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972649    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-multus-conf-dir\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972655    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-multus-cni-dir\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972670    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-run-openvswitch\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972689    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-etc-kubernetes\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972721    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8lwn\" (UniqueName: \"kubernetes.io/projected/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-kube-api-access-g8lwn\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972723    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-multus-cni-dir\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972741    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-multus-socket-dir-parent\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972757    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-env-overrides\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972787    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-ovnkube-script-lib\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972807    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c85b9378-0915-419f-9b29-47bb580dc22a-dbus\") pod \"global-pull-secret-syncer-tcnw5\" (UID: \"c85b9378-0915-419f-9b29-47bb580dc22a\") " pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972819    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a357932-d881-47a6-b6d9-ec8dab12d087-cnibin\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972847    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-systemd-units\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972849    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-run-systemd\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972856    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-run-ovn\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972897    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-var-lib-openvswitch\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.975096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972640    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-host-kubelet\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.976061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972979    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a357932-d881-47a6-b6d9-ec8dab12d087-cnibin\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.976061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.973062    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-etc-kubernetes\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.976061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972690    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-log-socket\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.976061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.973619    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c85b9378-0915-419f-9b29-47bb580dc22a-kubelet-config\") pod \"global-pull-secret-syncer-tcnw5\" (UID: \"c85b9378-0915-419f-9b29-47bb580dc22a\") " pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:24.976061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.973723    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-run-systemd\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.976061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.972822    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-ovnkube-config\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.976061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.974034    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-ovnkube-script-lib\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.976061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.974042    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-env-overrides\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.976061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.974096    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-cnibin\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.976061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.974161    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-system-cni-dir\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.976061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.974214    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-node-log\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.976061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.974265    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-var-lib-kubelet\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.976061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.974325    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-host-var-lib-cni-multus\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.976061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.974372    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-cnibin\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.976061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.974396    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/db932df4-aa61-4dd9-b590-ba22dabce334-hosts-file\") pod \"node-resolver-zs2d7\" (UID: \"db932df4-aa61-4dd9-b590-ba22dabce334\") " pod="openshift-dns/node-resolver-zs2d7"
Sep 05 06:26:24.976061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.974542    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c85b9378-0915-419f-9b29-47bb580dc22a-kubelet-config\") pod \"global-pull-secret-syncer-tcnw5\" (UID: \"c85b9378-0915-419f-9b29-47bb580dc22a\") " pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:24.976061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.975419    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-ovn-node-metrics-cert\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.977307 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.977278    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a357932-d881-47a6-b6d9-ec8dab12d087-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.978164 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.977457    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2d9b77a8-583d-4d7e-b62e-60fe17d73f24-agent-certs\") pod \"konnectivity-agent-jdbcm\" (UID: \"2d9b77a8-583d-4d7e-b62e-60fe17d73f24\") " pod="kube-system/konnectivity-agent-jdbcm"
Sep 05 06:26:24.982878 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.982852    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 06:26:24.983021 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.982885    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 06:26:24.983021 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.982900    2559 projected.go:196] Error preparing data for projected volume kube-api-access-42g8k for pod openshift-network-diagnostics/network-check-target-hsthh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:24.983021 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:24.983013    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k podName:9c9536b7-5174-42f5-981d-1a6d1a14317e nodeName:}" failed. No retries permitted until 2026-09-05 06:26:25.482991928 +0000 UTC m=+2.073550860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-42g8k" (UniqueName: "kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k") pod "network-check-target-hsthh" (UID: "9c9536b7-5174-42f5-981d-1a6d1a14317e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:24.985584 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.985552    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27svw\" (UniqueName: \"kubernetes.io/projected/db932df4-aa61-4dd9-b590-ba22dabce334-kube-api-access-27svw\") pod \"node-resolver-zs2d7\" (UID: \"db932df4-aa61-4dd9-b590-ba22dabce334\") " pod="openshift-dns/node-resolver-zs2d7"
Sep 05 06:26:24.986666 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.986638    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8lwn\" (UniqueName: \"kubernetes.io/projected/9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7-kube-api-access-g8lwn\") pod \"multus-ztv4g\" (UID: \"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7\") " pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:24.986783 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.986758    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl2q7\" (UniqueName: \"kubernetes.io/projected/99bf56fa-eafd-489b-9d5d-75f31b076c45-kube-api-access-rl2q7\") pod \"network-metrics-daemon-jslwg\" (UID: \"99bf56fa-eafd-489b-9d5d-75f31b076c45\") " pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:24.986960 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.986942    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf8nf\" (UniqueName: \"kubernetes.io/projected/0a357932-d881-47a6-b6d9-ec8dab12d087-kube-api-access-pf8nf\") pod \"multus-additional-cni-plugins-j4q5t\" (UID: \"0a357932-d881-47a6-b6d9-ec8dab12d087\") " pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:24.987135 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:24.987112    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvxzf\" (UniqueName: \"kubernetes.io/projected/e5d95fe0-a12b-4f61-83ab-fb7ccab64e01-kube-api-access-lvxzf\") pod \"ovnkube-node-vrv5q\" (UID: \"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01\") " pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:24.999815 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:24.999776    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd0e8a512a9396826908b8226fb7b755.slice/crio-8203791a8ad47d1b5ef72ae5ad3414f4ce7b00b03508e22b4417dcc79ddf5553 WatchSource:0}: Error finding container 8203791a8ad47d1b5ef72ae5ad3414f4ce7b00b03508e22b4417dcc79ddf5553: Status 404 returned error can't find the container with id 8203791a8ad47d1b5ef72ae5ad3414f4ce7b00b03508e22b4417dcc79ddf5553
Sep 05 06:26:25.002902 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:25.002861    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50db2b2cd4cf113e73677eb1ebb83d24.slice/crio-e436b0e7cc8f18517a634de74d2f91994de2e85a9e962684d977fd0d5e8dd85e WatchSource:0}: Error finding container e436b0e7cc8f18517a634de74d2f91994de2e85a9e962684d977fd0d5e8dd85e: Status 404 returned error can't find the container with id e436b0e7cc8f18517a634de74d2f91994de2e85a9e962684d977fd0d5e8dd85e
Sep 05 06:26:25.005940 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.005923    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 06:26:25.054172 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.054134    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 06:26:25.098822 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.098699    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j8x2c"
Sep 05 06:26:25.109468 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.109440    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-92xkt"
Sep 05 06:26:25.109751 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:25.109728    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod561b2ddc_4c98_434e_995b_a8338ac6cfd5.slice/crio-c56ea75f747704aab8fda36f4e109f973b02aad219c69401882ba9aebff971e0 WatchSource:0}: Error finding container c56ea75f747704aab8fda36f4e109f973b02aad219c69401882ba9aebff971e0: Status 404 returned error can't find the container with id c56ea75f747704aab8fda36f4e109f973b02aad219c69401882ba9aebff971e0
Sep 05 06:26:25.119284 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:25.119258    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb74c33d0_93d3_4135_b98e_1e9c3ee8ab8a.slice/crio-23e7ecd4b85298f1a1044cf32038010e7ddda5c27cba02f2549ffbbfb84d5689 WatchSource:0}: Error finding container 23e7ecd4b85298f1a1044cf32038010e7ddda5c27cba02f2549ffbbfb84d5689: Status 404 returned error can't find the container with id 23e7ecd4b85298f1a1044cf32038010e7ddda5c27cba02f2549ffbbfb84d5689
Sep 05 06:26:25.125672 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.125651    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv"
Sep 05 06:26:25.134588 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:25.134558    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57a89e6a_809b_466d_9857_9a62db191f39.slice/crio-f82b4bb1d8c37df039f1090c083206e80fef11e69e80a415dd606bbf8bc22c53 WatchSource:0}: Error finding container f82b4bb1d8c37df039f1090c083206e80fef11e69e80a415dd606bbf8bc22c53: Status 404 returned error can't find the container with id f82b4bb1d8c37df039f1090c083206e80fef11e69e80a415dd606bbf8bc22c53
Sep 05 06:26:25.142168 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.142124    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kp6pd"
Sep 05 06:26:25.153738 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:25.153698    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc806dc77_8c41_4bbb_95a6_718556fc41eb.slice/crio-1b47c13d0816253028811dbfd5cef941ca22f8a862012cd0d980d6bcf7af8e5a WatchSource:0}: Error finding container 1b47c13d0816253028811dbfd5cef941ca22f8a862012cd0d980d6bcf7af8e5a: Status 404 returned error can't find the container with id 1b47c13d0816253028811dbfd5cef941ca22f8a862012cd0d980d6bcf7af8e5a
Sep 05 06:26:25.154697 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.154670    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ztv4g"
Sep 05 06:26:25.163621 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:25.163597    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb52357_c7cb_4f1c_a4a7_9a927fe0d7a7.slice/crio-24a0a05c4dd940dc1f81714006bdd5c0d24e3fd4bd6df25d8da2a730d7aaa02e WatchSource:0}: Error finding container 24a0a05c4dd940dc1f81714006bdd5c0d24e3fd4bd6df25d8da2a730d7aaa02e: Status 404 returned error can't find the container with id 24a0a05c4dd940dc1f81714006bdd5c0d24e3fd4bd6df25d8da2a730d7aaa02e
Sep 05 06:26:25.171094 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.171068    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jdbcm"
Sep 05 06:26:25.180291 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:25.180269    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d9b77a8_583d_4d7e_b62e_60fe17d73f24.slice/crio-3b3c686d8424158609e6e693c1786114e27535c84a9a60123b398d3246e8ef07 WatchSource:0}: Error finding container 3b3c686d8424158609e6e693c1786114e27535c84a9a60123b398d3246e8ef07: Status 404 returned error can't find the container with id 3b3c686d8424158609e6e693c1786114e27535c84a9a60123b398d3246e8ef07
Sep 05 06:26:25.192987 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.192968    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zs2d7"
Sep 05 06:26:25.199272 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.199254    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j4q5t"
Sep 05 06:26:25.202269 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:25.202245    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb932df4_aa61_4dd9_b590_ba22dabce334.slice/crio-9dfa5af64b74f0013443ea7464bbff1401edb53f59b1a74d737a01d9c7d562d0 WatchSource:0}: Error finding container 9dfa5af64b74f0013443ea7464bbff1401edb53f59b1a74d737a01d9c7d562d0: Status 404 returned error can't find the container with id 9dfa5af64b74f0013443ea7464bbff1401edb53f59b1a74d737a01d9c7d562d0
Sep 05 06:26:25.203635 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.203614    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:25.208569 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:25.208545    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a357932_d881_47a6_b6d9_ec8dab12d087.slice/crio-898fd4974729d71bf86a8696d4737efb9e45249839be74abe9e1f979ae54c53e WatchSource:0}: Error finding container 898fd4974729d71bf86a8696d4737efb9e45249839be74abe9e1f979ae54c53e: Status 404 returned error can't find the container with id 898fd4974729d71bf86a8696d4737efb9e45249839be74abe9e1f979ae54c53e
Sep 05 06:26:25.214769 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:25.214748    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d95fe0_a12b_4f61_83ab_fb7ccab64e01.slice/crio-19932933fbf23fd9b6329df9ad3b7dda3b73f17ed98b03b7080d0c18a244e7be WatchSource:0}: Error finding container 19932933fbf23fd9b6329df9ad3b7dda3b73f17ed98b03b7080d0c18a244e7be: Status 404 returned error can't find the container with id 19932933fbf23fd9b6329df9ad3b7dda3b73f17ed98b03b7080d0c18a244e7be
Sep 05 06:26:25.254751 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.254718    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 06:26:25.478141 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.478049    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs\") pod \"network-metrics-daemon-jslwg\" (UID: \"99bf56fa-eafd-489b-9d5d-75f31b076c45\") " pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:25.478141 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.478102    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret\") pod \"global-pull-secret-syncer-tcnw5\" (UID: \"c85b9378-0915-419f-9b29-47bb580dc22a\") " pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:25.478381 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:25.478256    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:25.478381 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:25.478362    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret podName:c85b9378-0915-419f-9b29-47bb580dc22a nodeName:}" failed. No retries permitted until 2026-09-05 06:26:26.478335175 +0000 UTC m=+3.068894108 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret") pod "global-pull-secret-syncer-tcnw5" (UID: "c85b9378-0915-419f-9b29-47bb580dc22a") : object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:25.478512 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:25.478432    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:25.478566 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:25.478515    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs podName:99bf56fa-eafd-489b-9d5d-75f31b076c45 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:26.478496288 +0000 UTC m=+3.069055234 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs") pod "network-metrics-daemon-jslwg" (UID: "99bf56fa-eafd-489b-9d5d-75f31b076c45") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:25.579062 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.578877    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42g8k\" (UniqueName: \"kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k\") pod \"network-check-target-hsthh\" (UID: \"9c9536b7-5174-42f5-981d-1a6d1a14317e\") " pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:25.579269 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:25.579143    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 06:26:25.579269 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:25.579202    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 06:26:25.579269 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:25.579220    2559 projected.go:196] Error preparing data for projected volume kube-api-access-42g8k for pod openshift-network-diagnostics/network-check-target-hsthh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:25.579442 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:25.579334    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k podName:9c9536b7-5174-42f5-981d-1a6d1a14317e nodeName:}" failed. No retries permitted until 2026-09-05 06:26:26.579296462 +0000 UTC m=+3.169855397 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-42g8k" (UniqueName: "kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k") pod "network-check-target-hsthh" (UID: "9c9536b7-5174-42f5-981d-1a6d1a14317e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:25.816066 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.815821    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-04 06:21:25 +0000 UTC" deadline="2028-05-14 01:00:38.969732622 +0000 UTC"
Sep 05 06:26:25.816066 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.815910    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14802h34m13.153827799s"
Sep 05 06:26:25.930710 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.930636    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q" event={"ID":"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01","Type":"ContainerStarted","Data":"19932933fbf23fd9b6329df9ad3b7dda3b73f17ed98b03b7080d0c18a244e7be"}
Sep 05 06:26:25.956926 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.956829    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4q5t" event={"ID":"0a357932-d881-47a6-b6d9-ec8dab12d087","Type":"ContainerStarted","Data":"898fd4974729d71bf86a8696d4737efb9e45249839be74abe9e1f979ae54c53e"}
Sep 05 06:26:25.960119 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.960077    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 06:26:25.968974 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.968902    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jdbcm" event={"ID":"2d9b77a8-583d-4d7e-b62e-60fe17d73f24","Type":"ContainerStarted","Data":"3b3c686d8424158609e6e693c1786114e27535c84a9a60123b398d3246e8ef07"}
Sep 05 06:26:25.977610 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.977569    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv" event={"ID":"57a89e6a-809b-466d-9857-9a62db191f39","Type":"ContainerStarted","Data":"f82b4bb1d8c37df039f1090c083206e80fef11e69e80a415dd606bbf8bc22c53"}
Sep 05 06:26:25.987461 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.987387    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-92xkt" event={"ID":"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a","Type":"ContainerStarted","Data":"23e7ecd4b85298f1a1044cf32038010e7ddda5c27cba02f2549ffbbfb84d5689"}
Sep 05 06:26:25.999154 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:25.999100    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal" event={"ID":"bd0e8a512a9396826908b8226fb7b755","Type":"ContainerStarted","Data":"8203791a8ad47d1b5ef72ae5ad3414f4ce7b00b03508e22b4417dcc79ddf5553"}
Sep 05 06:26:26.013845 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:26.013783    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zs2d7" event={"ID":"db932df4-aa61-4dd9-b590-ba22dabce334","Type":"ContainerStarted","Data":"9dfa5af64b74f0013443ea7464bbff1401edb53f59b1a74d737a01d9c7d562d0"}
Sep 05 06:26:26.020843 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:26.020802    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ztv4g" event={"ID":"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7","Type":"ContainerStarted","Data":"24a0a05c4dd940dc1f81714006bdd5c0d24e3fd4bd6df25d8da2a730d7aaa02e"}
Sep 05 06:26:26.038926 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:26.038883    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kp6pd" event={"ID":"c806dc77-8c41-4bbb-95a6-718556fc41eb","Type":"ContainerStarted","Data":"1b47c13d0816253028811dbfd5cef941ca22f8a862012cd0d980d6bcf7af8e5a"}
Sep 05 06:26:26.052526 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:26.052484    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j8x2c" event={"ID":"561b2ddc-4c98-434e-995b-a8338ac6cfd5","Type":"ContainerStarted","Data":"c56ea75f747704aab8fda36f4e109f973b02aad219c69401882ba9aebff971e0"}
Sep 05 06:26:26.063708 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:26.063669    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-170.ec2.internal" event={"ID":"50db2b2cd4cf113e73677eb1ebb83d24","Type":"ContainerStarted","Data":"e436b0e7cc8f18517a634de74d2f91994de2e85a9e962684d977fd0d5e8dd85e"}
Sep 05 06:26:26.489325 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:26.489279    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs\") pod \"network-metrics-daemon-jslwg\" (UID: \"99bf56fa-eafd-489b-9d5d-75f31b076c45\") " pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:26.489562 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:26.489335    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret\") pod \"global-pull-secret-syncer-tcnw5\" (UID: \"c85b9378-0915-419f-9b29-47bb580dc22a\") " pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:26.489562 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:26.489493    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:26.489737 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:26.489563    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret podName:c85b9378-0915-419f-9b29-47bb580dc22a nodeName:}" failed. No retries permitted until 2026-09-05 06:26:28.489542394 +0000 UTC m=+5.080101326 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret") pod "global-pull-secret-syncer-tcnw5" (UID: "c85b9378-0915-419f-9b29-47bb580dc22a") : object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:26.490097 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:26.490072    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:26.490240 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:26.490140    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs podName:99bf56fa-eafd-489b-9d5d-75f31b076c45 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:28.490121891 +0000 UTC m=+5.080680844 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs") pod "network-metrics-daemon-jslwg" (UID: "99bf56fa-eafd-489b-9d5d-75f31b076c45") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:26.590625 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:26.590578    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42g8k\" (UniqueName: \"kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k\") pod \"network-check-target-hsthh\" (UID: \"9c9536b7-5174-42f5-981d-1a6d1a14317e\") " pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:26.591209 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:26.590813    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 06:26:26.591209 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:26.590837    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 06:26:26.591209 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:26.590852    2559 projected.go:196] Error preparing data for projected volume kube-api-access-42g8k for pod openshift-network-diagnostics/network-check-target-hsthh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:26.591209 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:26.590923    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k podName:9c9536b7-5174-42f5-981d-1a6d1a14317e nodeName:}" failed. No retries permitted until 2026-09-05 06:26:28.590899983 +0000 UTC m=+5.181458966 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-42g8k" (UniqueName: "kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k") pod "network-check-target-hsthh" (UID: "9c9536b7-5174-42f5-981d-1a6d1a14317e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:26.725916 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:26.725881    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 05 06:26:26.816285 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:26.816148    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-04 06:21:25 +0000 UTC" deadline="2028-03-16 11:39:55.491693856 +0000 UTC"
Sep 05 06:26:26.816285 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:26.816219    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13397h13m28.675479917s"
Sep 05 06:26:26.886261 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:26.886211    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:26.886468 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:26.886365    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:26.886872 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:26.886849    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:26.887024 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:26.886976    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:26.887087 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:26.887056    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:26.887216 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:26.887148    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:28.508264 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:28.507057    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs\") pod \"network-metrics-daemon-jslwg\" (UID: \"99bf56fa-eafd-489b-9d5d-75f31b076c45\") " pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:28.508264 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:28.507383    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret\") pod \"global-pull-secret-syncer-tcnw5\" (UID: \"c85b9378-0915-419f-9b29-47bb580dc22a\") " pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:28.508264 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:28.507601    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:28.508264 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:28.507686    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret podName:c85b9378-0915-419f-9b29-47bb580dc22a nodeName:}" failed. No retries permitted until 2026-09-05 06:26:32.507664405 +0000 UTC m=+9.098223337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret") pod "global-pull-secret-syncer-tcnw5" (UID: "c85b9378-0915-419f-9b29-47bb580dc22a") : object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:28.508264 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:28.508164    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:28.508264 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:28.508224    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs podName:99bf56fa-eafd-489b-9d5d-75f31b076c45 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:32.508204748 +0000 UTC m=+9.098763686 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs") pod "network-metrics-daemon-jslwg" (UID: "99bf56fa-eafd-489b-9d5d-75f31b076c45") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:28.609013 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:28.608964    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42g8k\" (UniqueName: \"kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k\") pod \"network-check-target-hsthh\" (UID: \"9c9536b7-5174-42f5-981d-1a6d1a14317e\") " pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:28.609205 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:28.609178    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 06:26:28.609205 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:28.609204    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 06:26:28.609334 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:28.609261    2559 projected.go:196] Error preparing data for projected volume kube-api-access-42g8k for pod openshift-network-diagnostics/network-check-target-hsthh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:28.609401 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:28.609358    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k podName:9c9536b7-5174-42f5-981d-1a6d1a14317e nodeName:}" failed. No retries permitted until 2026-09-05 06:26:32.609334428 +0000 UTC m=+9.199893381 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-42g8k" (UniqueName: "kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k") pod "network-check-target-hsthh" (UID: "9c9536b7-5174-42f5-981d-1a6d1a14317e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:28.887030 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:28.886170    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:28.887030 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:28.886225    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:28.887030 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:28.886330    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:28.887030 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:28.886711    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:28.887030 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:28.886828    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:28.887030 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:28.886923    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:30.885835 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:30.885791    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:30.886421 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:30.885979    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:30.886508 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:30.886428    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:30.886570 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:30.886545    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:30.886633 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:30.886428    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:30.886769 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:30.886729    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:32.549580 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:32.549535    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs\") pod \"network-metrics-daemon-jslwg\" (UID: \"99bf56fa-eafd-489b-9d5d-75f31b076c45\") " pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:32.549985 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:32.549588    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret\") pod \"global-pull-secret-syncer-tcnw5\" (UID: \"c85b9378-0915-419f-9b29-47bb580dc22a\") " pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:32.549985 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:32.549663    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:32.549985 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:32.549722    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:32.549985 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:32.549736    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs podName:99bf56fa-eafd-489b-9d5d-75f31b076c45 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:40.549714544 +0000 UTC m=+17.140273494 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs") pod "network-metrics-daemon-jslwg" (UID: "99bf56fa-eafd-489b-9d5d-75f31b076c45") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:32.549985 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:32.549760    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret podName:c85b9378-0915-419f-9b29-47bb580dc22a nodeName:}" failed. No retries permitted until 2026-09-05 06:26:40.549749737 +0000 UTC m=+17.140308694 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret") pod "global-pull-secret-syncer-tcnw5" (UID: "c85b9378-0915-419f-9b29-47bb580dc22a") : object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:32.650793 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:32.650730    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42g8k\" (UniqueName: \"kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k\") pod \"network-check-target-hsthh\" (UID: \"9c9536b7-5174-42f5-981d-1a6d1a14317e\") " pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:32.651020 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:32.650946    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 06:26:32.651020 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:32.650971    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 06:26:32.651020 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:32.650987    2559 projected.go:196] Error preparing data for projected volume kube-api-access-42g8k for pod openshift-network-diagnostics/network-check-target-hsthh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:32.651197 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:32.651068    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k podName:9c9536b7-5174-42f5-981d-1a6d1a14317e nodeName:}" failed. No retries permitted until 2026-09-05 06:26:40.651043485 +0000 UTC m=+17.241602424 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-42g8k" (UniqueName: "kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k") pod "network-check-target-hsthh" (UID: "9c9536b7-5174-42f5-981d-1a6d1a14317e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:32.886403 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:32.886295    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:32.886596 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:32.886464    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:32.886658 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:32.886593    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:32.886720 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:32.886599    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:32.886777 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:32.886724    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:32.886837 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:32.886775    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:34.886243 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:34.886186    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:34.886777 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:34.886195    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:34.886777 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:34.886345    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:34.886777 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:34.886419    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:34.886777 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:34.886215    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:34.886777 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:34.886528    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:36.885875 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:36.885830    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:36.885875 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:36.885844    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:36.886394 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:36.885966    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:36.886394 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:36.885978    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:36.886394 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:36.886102    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:36.886394 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:36.886208    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:38.886204 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:38.886157    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:38.886204 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:38.886181    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:38.886791 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:38.886301    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:38.886791 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:38.886361    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:38.886791 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:38.886423    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:38.886791 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:38.886500    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:40.614835 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:40.614796    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs\") pod \"network-metrics-daemon-jslwg\" (UID: \"99bf56fa-eafd-489b-9d5d-75f31b076c45\") " pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:40.615530 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:40.614845    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret\") pod \"global-pull-secret-syncer-tcnw5\" (UID: \"c85b9378-0915-419f-9b29-47bb580dc22a\") " pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:40.615530 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:40.614980    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:40.615530 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:40.615057    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret podName:c85b9378-0915-419f-9b29-47bb580dc22a nodeName:}" failed. No retries permitted until 2026-09-05 06:26:56.615039697 +0000 UTC m=+33.205598642 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret") pod "global-pull-secret-syncer-tcnw5" (UID: "c85b9378-0915-419f-9b29-47bb580dc22a") : object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:40.615530 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:40.614980    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:40.615530 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:40.615145    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs podName:99bf56fa-eafd-489b-9d5d-75f31b076c45 nodeName:}" failed. No retries permitted until 2026-09-05 06:26:56.615125084 +0000 UTC m=+33.205684033 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs") pod "network-metrics-daemon-jslwg" (UID: "99bf56fa-eafd-489b-9d5d-75f31b076c45") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:40.715264 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:40.715203    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42g8k\" (UniqueName: \"kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k\") pod \"network-check-target-hsthh\" (UID: \"9c9536b7-5174-42f5-981d-1a6d1a14317e\") " pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:40.715459 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:40.715394    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 06:26:40.715459 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:40.715419    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 06:26:40.715459 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:40.715429    2559 projected.go:196] Error preparing data for projected volume kube-api-access-42g8k for pod openshift-network-diagnostics/network-check-target-hsthh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:40.715602 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:40.715490    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k podName:9c9536b7-5174-42f5-981d-1a6d1a14317e nodeName:}" failed. No retries permitted until 2026-09-05 06:26:56.715470712 +0000 UTC m=+33.306029645 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-42g8k" (UniqueName: "kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k") pod "network-check-target-hsthh" (UID: "9c9536b7-5174-42f5-981d-1a6d1a14317e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:40.886550 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:40.886461    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:40.886708 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:40.886461    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:40.886708 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:40.886601    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:40.886708 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:40.886469    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:40.886823 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:40.886641    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:40.886823 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:40.886736    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:42.885811 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:42.885750    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:42.886316 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:42.885816    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:42.886316 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:42.885816    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:42.886316 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:42.885928    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:42.886316 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:42.885981    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:42.886316 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:42.886095    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:44.885440 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:44.885409    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:44.885755 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:44.885448    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:44.885755 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:44.885457    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:44.885755 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:44.885533    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:44.885755 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:44.885608    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:44.885755 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:44.885672    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:45.113044 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:45.113006    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-170.ec2.internal" event={"ID":"50db2b2cd4cf113e73677eb1ebb83d24","Type":"ContainerStarted","Data":"ec2fdc7be8df514ba5ab4e77f7752aea5f3a4ea7100cc5dc3dd6dd3f6123c60d"}
Sep 05 06:26:45.129854 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:45.129752    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-170.ec2.internal" podStartSLOduration=20.129729882 podStartE2EDuration="20.129729882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:26:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:45.129192532 +0000 UTC m=+21.719751484" watchObservedRunningTime="2026-09-05 06:26:45.129729882 +0000 UTC m=+21.720288835"
Sep 05 06:26:46.117470 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.117169    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j8x2c" event={"ID":"561b2ddc-4c98-434e-995b-a8338ac6cfd5","Type":"ContainerStarted","Data":"8424b1613650fed9bb1bf5e3afd07843744c8a664139e986b2b2fb4e1839e9ed"}
Sep 05 06:26:46.122690 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.122656    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q" event={"ID":"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01","Type":"ContainerStarted","Data":"6484a8c217b30da42ae0e053abba54a4e9eb500c0cd9cc400748b2601156428c"}
Sep 05 06:26:46.122868 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.122747    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q" event={"ID":"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01","Type":"ContainerStarted","Data":"d6f094aaf713dc476fda5b8472312173e0c73b4b09c65f16d4be91ce13ad8bef"}
Sep 05 06:26:46.122868 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.122799    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q" event={"ID":"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01","Type":"ContainerStarted","Data":"3a2353ce981eb21fad96fe98fe9e26ce37ebc58bb94a239a2f7e2e8e1d00bfb8"}
Sep 05 06:26:46.122868 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.122846    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q" event={"ID":"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01","Type":"ContainerStarted","Data":"13faf051b9401f07e36ed647112e3cea1eb7edae49e124a8f604f717ec3d96d3"}
Sep 05 06:26:46.123038 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.122878    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q" event={"ID":"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01","Type":"ContainerStarted","Data":"2fbaa9453d02bf5a53716bd6fdcfe0a60959a76fe5ed61c622ba111011a72129"}
Sep 05 06:26:46.123038 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.122928    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q" event={"ID":"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01","Type":"ContainerStarted","Data":"4518a2e43caa483bb4642c420525d1dc15230ae35fd09858444e82c9635e8e78"}
Sep 05 06:26:46.124785 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.124750    2559 generic.go:356] "Generic (PLEG): container finished" podID="0a357932-d881-47a6-b6d9-ec8dab12d087" containerID="a7c1c22dc9cd063eba4817d2d64aaabc6c35d6e30e1cfd3bf688bf94305a627c" exitCode=0
Sep 05 06:26:46.124941 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.124833    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4q5t" event={"ID":"0a357932-d881-47a6-b6d9-ec8dab12d087","Type":"ContainerDied","Data":"a7c1c22dc9cd063eba4817d2d64aaabc6c35d6e30e1cfd3bf688bf94305a627c"}
Sep 05 06:26:46.127305 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.127278    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jdbcm" event={"ID":"2d9b77a8-583d-4d7e-b62e-60fe17d73f24","Type":"ContainerStarted","Data":"e49520660767765fd7a1426a2dea976ad5c67b8ba83a9384754caab27f322275"}
Sep 05 06:26:46.129115 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.128969    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv" event={"ID":"57a89e6a-809b-466d-9857-9a62db191f39","Type":"ContainerStarted","Data":"23acd7c99ea777b06dec5193dd0493fe7daae731370eb684593b6d2385b2e3ed"}
Sep 05 06:26:46.131109 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.130804    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-92xkt" event={"ID":"b74c33d0-93d3-4135-b98e-1e9c3ee8ab8a","Type":"ContainerStarted","Data":"73d66d29e0c2c5c6b9c9dd3666e8d645634bb13ea5daf47450d16b51a6b83c0e"}
Sep 05 06:26:46.132621 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.132595    2559 generic.go:356] "Generic (PLEG): container finished" podID="bd0e8a512a9396826908b8226fb7b755" containerID="60d0376f6327904fe708d18f0f6cc7c9a0c0fcc41f84997f5933d63d61cae7ec" exitCode=0
Sep 05 06:26:46.132742 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.132686    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal" event={"ID":"bd0e8a512a9396826908b8226fb7b755","Type":"ContainerDied","Data":"60d0376f6327904fe708d18f0f6cc7c9a0c0fcc41f84997f5933d63d61cae7ec"}
Sep 05 06:26:46.134634 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.134601    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zs2d7" event={"ID":"db932df4-aa61-4dd9-b590-ba22dabce334","Type":"ContainerStarted","Data":"2ff84bb2d5520a04c0578e3fc38a76ecc0d8195b5d9dc03df514ab1855f772bd"}
Sep 05 06:26:46.136330 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.136291    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ztv4g" event={"ID":"9cb52357-c7cb-4f1c-a4a7-9a927fe0d7a7","Type":"ContainerStarted","Data":"ca00cb29cd7913eb9786029def2890b18e836b942a6d4c88d42bbe53d2dc28f7"}
Sep 05 06:26:46.137900 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.137839    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-j8x2c" podStartSLOduration=2.358316874 podStartE2EDuration="22.137827202s" totalImagesPullingTime="19.779510328s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:26:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:46.137748811 +0000 UTC m=+22.728307758" watchObservedRunningTime="2026-09-05 06:26:46.137827202 +0000 UTC m=+22.728386154"
Sep 05 06:26:46.170252 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.170190    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-92xkt" podStartSLOduration=2.39510024 podStartE2EDuration="22.170168912s" totalImagesPullingTime="19.775068672s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:26:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:46.170127059 +0000 UTC m=+22.760686011" watchObservedRunningTime="2026-09-05 06:26:46.170168912 +0000 UTC m=+22.760727890"
Sep 05 06:26:46.211471 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.211407    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ztv4g" podStartSLOduration=1.229188007 podStartE2EDuration="21.211386774s" totalImagesPullingTime="19.982198767s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:46.211254174 +0000 UTC m=+22.801813323" watchObservedRunningTime="2026-09-05 06:26:46.211386774 +0000 UTC m=+22.801945728"
Sep 05 06:26:46.229488 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.229442    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jdbcm" podStartSLOduration=1.5192194749999999 podStartE2EDuration="21.229425435s" totalImagesPullingTime="19.71020596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:46.229011458 +0000 UTC m=+22.819570409" watchObservedRunningTime="2026-09-05 06:26:46.229425435 +0000 UTC m=+22.819984387"
Sep 05 06:26:46.245828 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.245783    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zs2d7" podStartSLOduration=1.5668863389999999 podStartE2EDuration="21.245770944s" totalImagesPullingTime="19.678884605s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:46.245417552 +0000 UTC m=+22.835976503" watchObservedRunningTime="2026-09-05 06:26:46.245770944 +0000 UTC m=+22.836329896"
Sep 05 06:26:46.886378 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.886347    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:46.886502 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:46.886463    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:46.886740 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.886570    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:46.886844 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:46.886785    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:46.886844 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:46.886793    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:46.886982 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:46.886898    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:47.030402 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:47.030380    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 05 06:26:47.141575 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:47.141484    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv" event={"ID":"57a89e6a-809b-466d-9857-9a62db191f39","Type":"ContainerStarted","Data":"84ab4a01207aa08539a2a88d0879150b8d2898173e379df31e97d395c4049012"}
Sep 05 06:26:47.143937 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:47.143874    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal" event={"ID":"bd0e8a512a9396826908b8226fb7b755","Type":"ContainerStarted","Data":"eebb5917cc8ae2f1bca177f2dc0e282e896ce3aadd73c1783bf51e82e81eb67c"}
Sep 05 06:26:47.145761 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:47.145728    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kp6pd" event={"ID":"c806dc77-8c41-4bbb-95a6-718556fc41eb","Type":"ContainerStarted","Data":"ec5c42471c5201235640b4c261331892665121f206c1ccdf57183764780bad14"}
Sep 05 06:26:47.193275 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:47.193187    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-170.ec2.internal" podStartSLOduration=22.193148239 podStartE2EDuration="22.193148239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:26:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:47.17169368 +0000 UTC m=+23.762252614" watchObservedRunningTime="2026-09-05 06:26:47.193148239 +0000 UTC m=+23.783707191"
Sep 05 06:26:47.825667 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:47.825548    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-05T06:26:47.03039965Z","UUID":"69e119d4-6856-4b22-91c1-9465aa811cc5","Handler":null,"Name":"","Endpoint":""}
Sep 05 06:26:47.827483 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:47.827453    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 05 06:26:47.827645 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:47.827493    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 05 06:26:48.154489 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:48.154261    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q" event={"ID":"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01","Type":"ContainerStarted","Data":"accbc171b6271630033e9f9afc219cc675cbe380391ab710985057997ad19061"}
Sep 05 06:26:48.157597 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:48.157570    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv" event={"ID":"57a89e6a-809b-466d-9857-9a62db191f39","Type":"ContainerStarted","Data":"d455f9cc31afd12c1e56f2f837194e838179f3f1d5ef5cabf94f220e7758f811"}
Sep 05 06:26:48.179575 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:48.179523    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kp6pd" podStartSLOduration=3.443809229 podStartE2EDuration="23.179502216s" totalImagesPullingTime="19.735692987s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:47.193149739 +0000 UTC m=+23.783708694" watchObservedRunningTime="2026-09-05 06:26:48.179502216 +0000 UTC m=+24.770061176"
Sep 05 06:26:48.179760 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:48.179611    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tdjrv" podStartSLOduration=2.13490545 podStartE2EDuration="24.17960625s" totalImagesPullingTime="22.0447008s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:26:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:48.17913026 +0000 UTC m=+24.769689212" watchObservedRunningTime="2026-09-05 06:26:48.17960625 +0000 UTC m=+24.770165201"
Sep 05 06:26:48.886544 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:48.886498    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:48.886544 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:48.886524    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:48.886766 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:48.886524    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:48.886766 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:48.886647    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:48.886766 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:48.886726    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:48.886912 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:48.886808    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:50.885817 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:50.885627    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:50.886237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:50.885627    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:50.886237 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:50.885913    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:50.886237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:50.885676    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:50.886237 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:50.885974    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:50.886237 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:50.886113    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:50.910384 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:50.910350    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jdbcm"
Sep 05 06:26:50.911002 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:50.910979    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jdbcm"
Sep 05 06:26:51.167582 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:51.167479    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q" event={"ID":"e5d95fe0-a12b-4f61-83ab-fb7ccab64e01","Type":"ContainerStarted","Data":"2fdb6bf37cfbf4eec2fbd52fe8647b85cb9f80f2671aa564bbe6a008749ad510"}
Sep 05 06:26:51.167853 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:51.167755    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:51.167853 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:51.167818    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:51.169565 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:51.169532    2559 generic.go:356] "Generic (PLEG): container finished" podID="0a357932-d881-47a6-b6d9-ec8dab12d087" containerID="852db7540530605afb5e0d38159fb36c9f7d8b2ff6cf71ee7eae50c329ea8689" exitCode=0
Sep 05 06:26:51.169676 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:51.169608    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4q5t" event={"ID":"0a357932-d881-47a6-b6d9-ec8dab12d087","Type":"ContainerDied","Data":"852db7540530605afb5e0d38159fb36c9f7d8b2ff6cf71ee7eae50c329ea8689"}
Sep 05 06:26:51.169991 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:51.169974    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jdbcm"
Sep 05 06:26:51.170934 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:51.170355    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jdbcm"
Sep 05 06:26:51.184194 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:51.184171    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:51.201781 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:51.201734    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q" podStartSLOduration=6.49181932 podStartE2EDuration="26.201720392s" totalImagesPullingTime="19.709901072s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:26:51.200265472 +0000 UTC m=+27.790824444" watchObservedRunningTime="2026-09-05 06:26:51.201720392 +0000 UTC m=+27.792279416"
Sep 05 06:26:52.175333 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:52.175246    2559 generic.go:356] "Generic (PLEG): container finished" podID="0a357932-d881-47a6-b6d9-ec8dab12d087" containerID="5b0172f7c5f8ea77ea7456d60389ca722c300f7156401ed78c9ca73324dc1d16" exitCode=0
Sep 05 06:26:52.175922 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:52.175339    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4q5t" event={"ID":"0a357932-d881-47a6-b6d9-ec8dab12d087","Type":"ContainerDied","Data":"5b0172f7c5f8ea77ea7456d60389ca722c300f7156401ed78c9ca73324dc1d16"}
Sep 05 06:26:52.176441 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:52.176190    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:52.193953 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:52.193927    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:26:52.523661 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:52.523040    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tcnw5"]
Sep 05 06:26:52.523661 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:52.523194    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:52.523661 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:52.523304    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:52.523971 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:52.523798    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jslwg"]
Sep 05 06:26:52.523971 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:52.523953    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:52.524084 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:52.524054    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:52.524952 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:52.524925    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hsthh"]
Sep 05 06:26:52.525054 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:52.525039    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:52.525152 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:52.525129    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:53.180970 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:53.180847    2559 generic.go:356] "Generic (PLEG): container finished" podID="0a357932-d881-47a6-b6d9-ec8dab12d087" containerID="4f0aa22391b008007a63928989617f2aa97fcb394587bd038127a93edc7e4995" exitCode=0
Sep 05 06:26:53.180970 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:53.180911    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4q5t" event={"ID":"0a357932-d881-47a6-b6d9-ec8dab12d087","Type":"ContainerDied","Data":"4f0aa22391b008007a63928989617f2aa97fcb394587bd038127a93edc7e4995"}
Sep 05 06:26:53.890337 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:53.888244    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:53.890337 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:53.888678    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:53.890337 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:53.889256    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:53.890337 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:53.889398    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:53.890337 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:53.889452    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:53.890337 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:53.889553    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:55.886562 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:55.886495    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:55.886562 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:55.886531    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:55.887107 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:55.886495    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:55.887107 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:55.886644    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:55.887107 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:55.886776    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:55.887107 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:55.886866    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:56.632278 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:56.632234    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs\") pod \"network-metrics-daemon-jslwg\" (UID: \"99bf56fa-eafd-489b-9d5d-75f31b076c45\") " pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:56.632278 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:56.632274    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret\") pod \"global-pull-secret-syncer-tcnw5\" (UID: \"c85b9378-0915-419f-9b29-47bb580dc22a\") " pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:56.632536 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:56.632384    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:56.632536 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:56.632396    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:56.632536 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:56.632450    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret podName:c85b9378-0915-419f-9b29-47bb580dc22a nodeName:}" failed. No retries permitted until 2026-09-05 06:27:28.632435487 +0000 UTC m=+65.222994424 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret") pod "global-pull-secret-syncer-tcnw5" (UID: "c85b9378-0915-419f-9b29-47bb580dc22a") : object "kube-system"/"original-pull-secret" not registered
Sep 05 06:26:56.632536 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:56.632469    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs podName:99bf56fa-eafd-489b-9d5d-75f31b076c45 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:28.632462188 +0000 UTC m=+65.223021123 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs") pod "network-metrics-daemon-jslwg" (UID: "99bf56fa-eafd-489b-9d5d-75f31b076c45") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 05 06:26:56.733550 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:56.733509    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42g8k\" (UniqueName: \"kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k\") pod \"network-check-target-hsthh\" (UID: \"9c9536b7-5174-42f5-981d-1a6d1a14317e\") " pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:56.733729 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:56.733679    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 05 06:26:56.733729 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:56.733700    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 05 06:26:56.733729 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:56.733711    2559 projected.go:196] Error preparing data for projected volume kube-api-access-42g8k for pod openshift-network-diagnostics/network-check-target-hsthh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:56.733894 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:56.733771    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k podName:9c9536b7-5174-42f5-981d-1a6d1a14317e nodeName:}" failed. No retries permitted until 2026-09-05 06:27:28.733750933 +0000 UTC m=+65.324309863 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-42g8k" (UniqueName: "kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k") pod "network-check-target-hsthh" (UID: "9c9536b7-5174-42f5-981d-1a6d1a14317e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 05 06:26:57.885928 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:57.885868    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:57.886498 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:57.885868    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:57.886498 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:57.886006    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hsthh" podUID="9c9536b7-5174-42f5-981d-1a6d1a14317e"
Sep 05 06:26:57.886498 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:57.885868    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:57.886498 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:57.886068    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcnw5" podUID="c85b9378-0915-419f-9b29-47bb580dc22a"
Sep 05 06:26:57.886498 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:57.886151    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jslwg" podUID="99bf56fa-eafd-489b-9d5d-75f31b076c45"
Sep 05 06:26:58.196671 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:58.196630    2559 generic.go:356] "Generic (PLEG): container finished" podID="0a357932-d881-47a6-b6d9-ec8dab12d087" containerID="ec2ec7d00a3ca373d5cb9666e48e2762e5acaea7da72efdaf2e02f273975f3ad" exitCode=0
Sep 05 06:26:58.196671 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:58.196673    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4q5t" event={"ID":"0a357932-d881-47a6-b6d9-ec8dab12d087","Type":"ContainerDied","Data":"ec2ec7d00a3ca373d5cb9666e48e2762e5acaea7da72efdaf2e02f273975f3ad"}
Sep 05 06:26:59.202457 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.202417    2559 generic.go:356] "Generic (PLEG): container finished" podID="0a357932-d881-47a6-b6d9-ec8dab12d087" containerID="5bd52dbe90a600ad0a878822e3e6691bf4487e55ac5af9ce91b4e7ac69408fba" exitCode=0
Sep 05 06:26:59.203006 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.202474    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4q5t" event={"ID":"0a357932-d881-47a6-b6d9-ec8dab12d087","Type":"ContainerDied","Data":"5bd52dbe90a600ad0a878822e3e6691bf4487e55ac5af9ce91b4e7ac69408fba"}
Sep 05 06:26:59.212701 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.212673    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-170.ec2.internal" event="NodeReady"
Sep 05 06:26:59.212935 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.212921    2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 05 06:26:59.273682 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.273648    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-s2wh5"]
Sep 05 06:26:59.299230 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.299022    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-74htm"]
Sep 05 06:26:59.299370 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.299148    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:26:59.302033 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.301994    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 05 06:26:59.302164 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.302041    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v76tl\""
Sep 05 06:26:59.302376 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.302354    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 05 06:26:59.315223 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.315197    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-s2wh5"]
Sep 05 06:26:59.315359 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.315236    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-74htm"]
Sep 05 06:26:59.315359 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.315324    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-74htm"
Sep 05 06:26:59.318423 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.318383    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 05 06:26:59.318423 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.318392    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 05 06:26:59.318618 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.318393    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 05 06:26:59.319073 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.318955    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qnm9m\""
Sep 05 06:26:59.453207 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.453105    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9dv2\" (UniqueName: \"kubernetes.io/projected/3df3774c-273c-4b5b-9c63-785f0f683d72-kube-api-access-w9dv2\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:26:59.453207 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.453164    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3df3774c-273c-4b5b-9c63-785f0f683d72-tmp-dir\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:26:59.453407 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.453256    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/543b8b05-067c-49ff-81da-00f05d7e4454-cert\") pod \"ingress-canary-74htm\" (UID: \"543b8b05-067c-49ff-81da-00f05d7e4454\") " pod="openshift-ingress-canary/ingress-canary-74htm"
Sep 05 06:26:59.453407 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.453289    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz2fc\" (UniqueName: \"kubernetes.io/projected/543b8b05-067c-49ff-81da-00f05d7e4454-kube-api-access-dz2fc\") pod \"ingress-canary-74htm\" (UID: \"543b8b05-067c-49ff-81da-00f05d7e4454\") " pod="openshift-ingress-canary/ingress-canary-74htm"
Sep 05 06:26:59.453407 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.453308    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:26:59.453407 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.453397    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3df3774c-273c-4b5b-9c63-785f0f683d72-config-volume\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:26:59.554825 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.554792    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3df3774c-273c-4b5b-9c63-785f0f683d72-tmp-dir\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:26:59.555048 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.554858    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/543b8b05-067c-49ff-81da-00f05d7e4454-cert\") pod \"ingress-canary-74htm\" (UID: \"543b8b05-067c-49ff-81da-00f05d7e4454\") " pod="openshift-ingress-canary/ingress-canary-74htm"
Sep 05 06:26:59.555048 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.554912    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz2fc\" (UniqueName: \"kubernetes.io/projected/543b8b05-067c-49ff-81da-00f05d7e4454-kube-api-access-dz2fc\") pod \"ingress-canary-74htm\" (UID: \"543b8b05-067c-49ff-81da-00f05d7e4454\") " pod="openshift-ingress-canary/ingress-canary-74htm"
Sep 05 06:26:59.555048 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.554937    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:26:59.555048 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.554992    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3df3774c-273c-4b5b-9c63-785f0f683d72-config-volume\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:26:59.555048 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.555017    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9dv2\" (UniqueName: \"kubernetes.io/projected/3df3774c-273c-4b5b-9c63-785f0f683d72-kube-api-access-w9dv2\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:26:59.555279 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:59.555081    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 06:26:59.555279 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:26:59.555181    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls podName:3df3774c-273c-4b5b-9c63-785f0f683d72 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:00.055157332 +0000 UTC m=+36.645716282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls") pod "dns-default-s2wh5" (UID: "3df3774c-273c-4b5b-9c63-785f0f683d72") : secret "dns-default-metrics-tls" not found
Sep 05 06:26:59.555279 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.555227    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3df3774c-273c-4b5b-9c63-785f0f683d72-tmp-dir\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:26:59.555547 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.555523    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3df3774c-273c-4b5b-9c63-785f0f683d72-config-volume\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:26:59.559254 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.559234    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/543b8b05-067c-49ff-81da-00f05d7e4454-cert\") pod \"ingress-canary-74htm\" (UID: \"543b8b05-067c-49ff-81da-00f05d7e4454\") " pod="openshift-ingress-canary/ingress-canary-74htm"
Sep 05 06:26:59.566351 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.566322    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9dv2\" (UniqueName: \"kubernetes.io/projected/3df3774c-273c-4b5b-9c63-785f0f683d72-kube-api-access-w9dv2\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:26:59.566490 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.566394    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz2fc\" (UniqueName: \"kubernetes.io/projected/543b8b05-067c-49ff-81da-00f05d7e4454-kube-api-access-dz2fc\") pod \"ingress-canary-74htm\" (UID: \"543b8b05-067c-49ff-81da-00f05d7e4454\") " pod="openshift-ingress-canary/ingress-canary-74htm"
Sep 05 06:26:59.624644 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.624575    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-74htm"
Sep 05 06:26:59.885960 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.885915    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:26:59.886147 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.885924    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:26:59.886147 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.885933    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:26:59.889946 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.889921    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 05 06:26:59.889946 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.889927    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4m5kh\""
Sep 05 06:26:59.890197 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.889978    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 05 06:26:59.890197 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.890025    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-24sqj\""
Sep 05 06:26:59.890197 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.889975    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 05 06:26:59.890197 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.889983    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 05 06:26:59.938045 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:26:59.938009    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-74htm"]
Sep 05 06:26:59.941758 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:26:59.941735    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod543b8b05_067c_49ff_81da_00f05d7e4454.slice/crio-f51d818b6b30bea8c52cae8ee5336f94cbfaa6867c56c008b340769b8d973ca3 WatchSource:0}: Error finding container f51d818b6b30bea8c52cae8ee5336f94cbfaa6867c56c008b340769b8d973ca3: Status 404 returned error can't find the container with id f51d818b6b30bea8c52cae8ee5336f94cbfaa6867c56c008b340769b8d973ca3
Sep 05 06:27:00.059569 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:00.059474    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:27:00.059742 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:00.059614    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 06:27:00.059742 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:00.059685    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls podName:3df3774c-273c-4b5b-9c63-785f0f683d72 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:01.059657449 +0000 UTC m=+37.650216384 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls") pod "dns-default-s2wh5" (UID: "3df3774c-273c-4b5b-9c63-785f0f683d72") : secret "dns-default-metrics-tls" not found
Sep 05 06:27:00.206190 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:00.206148    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-74htm" event={"ID":"543b8b05-067c-49ff-81da-00f05d7e4454","Type":"ContainerStarted","Data":"f51d818b6b30bea8c52cae8ee5336f94cbfaa6867c56c008b340769b8d973ca3"}
Sep 05 06:27:00.209612 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:00.209580    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j4q5t" event={"ID":"0a357932-d881-47a6-b6d9-ec8dab12d087","Type":"ContainerStarted","Data":"06b6232b84d52447fcadf7402c08afcb36e3af113734f85ebf51a31c48c8f7a5"}
Sep 05 06:27:00.233862 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:00.233809    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j4q5t" podStartSLOduration=5.709543391 podStartE2EDuration="35.233791934s" totalImagesPullingTime="29.524248543s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:26:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:00.233010207 +0000 UTC m=+36.823569159" watchObservedRunningTime="2026-09-05 06:27:00.233791934 +0000 UTC m=+36.824350877"
Sep 05 06:27:01.069304 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:01.069253    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:27:01.069507 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:01.069433    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 06:27:01.069572 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:01.069520    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls podName:3df3774c-273c-4b5b-9c63-785f0f683d72 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:03.069498437 +0000 UTC m=+39.660057381 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls") pod "dns-default-s2wh5" (UID: "3df3774c-273c-4b5b-9c63-785f0f683d72") : secret "dns-default-metrics-tls" not found
Sep 05 06:27:03.085750 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:03.085703    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:27:03.086281 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:03.085911    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 06:27:03.086281 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:03.085991    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls podName:3df3774c-273c-4b5b-9c63-785f0f683d72 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:07.085973701 +0000 UTC m=+43.676532631 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls") pod "dns-default-s2wh5" (UID: "3df3774c-273c-4b5b-9c63-785f0f683d72") : secret "dns-default-metrics-tls" not found
Sep 05 06:27:03.219010 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:03.218972    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-74htm" event={"ID":"543b8b05-067c-49ff-81da-00f05d7e4454","Type":"ContainerStarted","Data":"a3e1b3e8028711b6513b08bf86e9885800772952c0f5afe804d9952419c4ee7c"}
Sep 05 06:27:03.238001 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:03.237937    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-74htm" podStartSLOduration=1.602567364 podStartE2EDuration="4.237919301s" totalImagesPullingTime="2.635351937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:26:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:03.236448222 +0000 UTC m=+39.827007198" watchObservedRunningTime="2026-09-05 06:27:03.237919301 +0000 UTC m=+39.828478252"
Sep 05 06:27:07.114732 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:07.114667    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:27:07.115150 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:07.114827    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 06:27:07.115150 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:07.114914    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls podName:3df3774c-273c-4b5b-9c63-785f0f683d72 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:15.114895291 +0000 UTC m=+51.705454236 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls") pod "dns-default-s2wh5" (UID: "3df3774c-273c-4b5b-9c63-785f0f683d72") : secret "dns-default-metrics-tls" not found
Sep 05 06:27:15.172176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:15.172125    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:27:15.172818 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:15.172300    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 06:27:15.172818 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:15.172394    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls podName:3df3774c-273c-4b5b-9c63-785f0f683d72 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:31.172369271 +0000 UTC m=+67.762928215 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls") pod "dns-default-s2wh5" (UID: "3df3774c-273c-4b5b-9c63-785f0f683d72") : secret "dns-default-metrics-tls" not found
Sep 05 06:27:24.196200 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:24.196163    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vrv5q"
Sep 05 06:27:28.265699 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.265657    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2kg8p"]
Sep 05 06:27:28.268295 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.268259    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-74c88b6858-trsfj"]
Sep 05 06:27:28.268446 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.268429    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2kg8p"
Sep 05 06:27:28.270033 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.270009    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-gjwjg"]
Sep 05 06:27:28.270157 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.270115    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:28.270836 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.270813    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-t7fxv\""
Sep 05 06:27:28.270836 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.270825    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 05 06:27:28.271100 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.270823    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 05 06:27:28.271868 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.271849    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.272647 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.272627    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 05 06:27:28.272749 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.272661    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 05 06:27:28.272749 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.272714    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 05 06:27:28.272845 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.272831    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-hsm2d\""
Sep 05 06:27:28.273002 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.272976    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 05 06:27:28.273746 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.273720    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 05 06:27:28.273844 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.273821    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 05 06:27:28.274993 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.274973    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-qxkr2\""
Sep 05 06:27:28.275504 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.275485    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 05 06:27:28.276846 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.275784    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 05 06:27:28.276846 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.276052    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 05 06:27:28.276846 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.276193    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 05 06:27:28.281155 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.280873    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2kg8p"]
Sep 05 06:27:28.284038 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.284016    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-gjwjg"]
Sep 05 06:27:28.284900 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.284863    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74c88b6858-trsfj"]
Sep 05 06:27:28.285172 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.285154    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 05 06:27:28.369544 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.369507    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-default-certificate\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:28.369544 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.369545    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:28.369800 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.369566    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggcc5\" (UniqueName: \"kubernetes.io/projected/81a69703-95b0-49e9-97e7-62fd2199968e-kube-api-access-ggcc5\") pod \"volume-data-source-validator-695c895bd6-2kg8p\" (UID: \"81a69703-95b0-49e9-97e7-62fd2199968e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2kg8p"
Sep 05 06:27:28.369800 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.369616    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-serving-cert\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.369800 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.369655    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-stats-auth\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:28.369800 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.369673    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-snapshots\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.369800 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.369736    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:28.369800 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.369752    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn7dk\" (UniqueName: \"kubernetes.io/projected/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-kube-api-access-fn7dk\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:28.369800 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.369782    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-tmp\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.370070 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.369812    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.370070 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.369842    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt9fd\" (UniqueName: \"kubernetes.io/projected/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-kube-api-access-dt9fd\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.370070 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.369862    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.470969 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.470922    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggcc5\" (UniqueName: \"kubernetes.io/projected/81a69703-95b0-49e9-97e7-62fd2199968e-kube-api-access-ggcc5\") pod \"volume-data-source-validator-695c895bd6-2kg8p\" (UID: \"81a69703-95b0-49e9-97e7-62fd2199968e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2kg8p"
Sep 05 06:27:28.470969 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.470971    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-serving-cert\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.471213 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.470996    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-stats-auth\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:28.471213 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.471014    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-snapshots\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.471213 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.471202    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:28.471346 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.471237    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn7dk\" (UniqueName: \"kubernetes.io/projected/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-kube-api-access-fn7dk\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:28.471346 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.471260    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-tmp\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.471346 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.471300    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.471346 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.471329    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt9fd\" (UniqueName: \"kubernetes.io/projected/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-kube-api-access-dt9fd\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.471528 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:28.471338    2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 05 06:27:28.471528 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.471361    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.471528 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.471394    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-default-certificate\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:28.471528 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:28.471410    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs podName:f9b3185c-0e0e-43aa-8e47-1090bfb5f016 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:28.971390392 +0000 UTC m=+65.561949341 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs") pod "router-default-74c88b6858-trsfj" (UID: "f9b3185c-0e0e-43aa-8e47-1090bfb5f016") : secret "router-metrics-certs-default" not found
Sep 05 06:27:28.471528 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.471455    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:28.471791 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:28.471581    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle podName:f9b3185c-0e0e-43aa-8e47-1090bfb5f016 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:28.971565491 +0000 UTC m=+65.562124434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle") pod "router-default-74c88b6858-trsfj" (UID: "f9b3185c-0e0e-43aa-8e47-1090bfb5f016") : configmap references non-existent config key: service-ca.crt
Sep 05 06:27:28.472404 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.471838    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-snapshots\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.472404 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.472123    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-tmp\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.472404 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.472282    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.472404 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.472312    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.474569 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.474542    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-default-certificate\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:28.474672 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.474656    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-serving-cert\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.474777 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.474756    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-stats-auth\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:28.480451 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.480414    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggcc5\" (UniqueName: \"kubernetes.io/projected/81a69703-95b0-49e9-97e7-62fd2199968e-kube-api-access-ggcc5\") pod \"volume-data-source-validator-695c895bd6-2kg8p\" (UID: \"81a69703-95b0-49e9-97e7-62fd2199968e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2kg8p"
Sep 05 06:27:28.481290 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.481266    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt9fd\" (UniqueName: \"kubernetes.io/projected/f8057dec-2f84-4037-9fc9-b7e0d68c5cc2-kube-api-access-dt9fd\") pod \"insights-operator-76bcd6ccf5-gjwjg\" (UID: \"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.481290 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.481270    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn7dk\" (UniqueName: \"kubernetes.io/projected/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-kube-api-access-fn7dk\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:28.584895 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.584786    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2kg8p"
Sep 05 06:27:28.597159 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.597128    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg"
Sep 05 06:27:28.672424 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.672386    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs\") pod \"network-metrics-daemon-jslwg\" (UID: \"99bf56fa-eafd-489b-9d5d-75f31b076c45\") " pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:27:28.672558 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.672445    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret\") pod \"global-pull-secret-syncer-tcnw5\" (UID: \"c85b9378-0915-419f-9b29-47bb580dc22a\") " pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:27:28.676077 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.676046    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 05 06:27:28.676198 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.676114    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 05 06:27:28.683293 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:28.683258    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 05 06:27:28.683409 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:28.683362    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs podName:99bf56fa-eafd-489b-9d5d-75f31b076c45 nodeName:}" failed. No retries permitted until 2026-09-05 06:28:32.683335446 +0000 UTC m=+129.273894382 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs") pod "network-metrics-daemon-jslwg" (UID: "99bf56fa-eafd-489b-9d5d-75f31b076c45") : secret "metrics-daemon-secret" not found
Sep 05 06:27:28.686394 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.686317    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c85b9378-0915-419f-9b29-47bb580dc22a-original-pull-secret\") pod \"global-pull-secret-syncer-tcnw5\" (UID: \"c85b9378-0915-419f-9b29-47bb580dc22a\") " pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:27:28.702692 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.702662    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcnw5"
Sep 05 06:27:28.773963 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.773809    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42g8k\" (UniqueName: \"kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k\") pod \"network-check-target-hsthh\" (UID: \"9c9536b7-5174-42f5-981d-1a6d1a14317e\") " pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:27:28.776770 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.776739    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 05 06:27:28.789421 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.789393    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 05 06:27:28.797798 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.797745    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42g8k\" (UniqueName: \"kubernetes.io/projected/9c9536b7-5174-42f5-981d-1a6d1a14317e-kube-api-access-42g8k\") pod \"network-check-target-hsthh\" (UID: \"9c9536b7-5174-42f5-981d-1a6d1a14317e\") " pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:27:28.889005 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.888925    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2kg8p"]
Sep 05 06:27:28.892317 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:27:28.892291    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81a69703_95b0_49e9_97e7_62fd2199968e.slice/crio-afcfcdae8dddf4432a5700bff218fb7aa677e0785a6f70d32433d9e2b467a93f WatchSource:0}: Error finding container afcfcdae8dddf4432a5700bff218fb7aa677e0785a6f70d32433d9e2b467a93f: Status 404 returned error can't find the container with id afcfcdae8dddf4432a5700bff218fb7aa677e0785a6f70d32433d9e2b467a93f
Sep 05 06:27:28.917034 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:27:28.916996    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8057dec_2f84_4037_9fc9_b7e0d68c5cc2.slice/crio-b581e3b1b8c18d8834f6316568c3c3dbd345491493e9178c124b4d8786a74d94 WatchSource:0}: Error finding container b581e3b1b8c18d8834f6316568c3c3dbd345491493e9178c124b4d8786a74d94: Status 404 returned error can't find the container with id b581e3b1b8c18d8834f6316568c3c3dbd345491493e9178c124b4d8786a74d94
Sep 05 06:27:28.917246 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.917228    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-gjwjg"]
Sep 05 06:27:28.975256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.975218    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:28.975460 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.975292    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:28.975460 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:28.975400    2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 05 06:27:28.975460 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:28.975415    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle podName:f9b3185c-0e0e-43aa-8e47-1090bfb5f016 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:29.975397145 +0000 UTC m=+66.565956076 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle") pod "router-default-74c88b6858-trsfj" (UID: "f9b3185c-0e0e-43aa-8e47-1090bfb5f016") : configmap references non-existent config key: service-ca.crt
Sep 05 06:27:28.975591 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:28.975483    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs podName:f9b3185c-0e0e-43aa-8e47-1090bfb5f016 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:29.975467025 +0000 UTC m=+66.566025969 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs") pod "router-default-74c88b6858-trsfj" (UID: "f9b3185c-0e0e-43aa-8e47-1090bfb5f016") : secret "router-metrics-certs-default" not found
Sep 05 06:27:28.994624 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:28.994592    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tcnw5"]
Sep 05 06:27:28.997709 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:27:28.997685    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc85b9378_0915_419f_9b29_47bb580dc22a.slice/crio-3258144a5bf9532e32f680454fe90c6b6397731e4fef3b433505bc839d03af97 WatchSource:0}: Error finding container 3258144a5bf9532e32f680454fe90c6b6397731e4fef3b433505bc839d03af97: Status 404 returned error can't find the container with id 3258144a5bf9532e32f680454fe90c6b6397731e4fef3b433505bc839d03af97
Sep 05 06:27:29.009706 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:29.009682    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-24sqj\""
Sep 05 06:27:29.017763 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:29.017742    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:27:29.287927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:29.287870    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2kg8p" event={"ID":"81a69703-95b0-49e9-97e7-62fd2199968e","Type":"ContainerStarted","Data":"afcfcdae8dddf4432a5700bff218fb7aa677e0785a6f70d32433d9e2b467a93f"}
Sep 05 06:27:29.289081 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:29.289055    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg" event={"ID":"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2","Type":"ContainerStarted","Data":"b581e3b1b8c18d8834f6316568c3c3dbd345491493e9178c124b4d8786a74d94"}
Sep 05 06:27:29.290133 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:29.290109    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tcnw5" event={"ID":"c85b9378-0915-419f-9b29-47bb580dc22a","Type":"ContainerStarted","Data":"3258144a5bf9532e32f680454fe90c6b6397731e4fef3b433505bc839d03af97"}
Sep 05 06:27:29.322878 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:29.322842    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hsthh"]
Sep 05 06:27:29.336894 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:27:29.336851    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c9536b7_5174_42f5_981d_1a6d1a14317e.slice/crio-b313584f2cb127e80af432525bdbab3a046e36f0f52de196bf0316bb82827545 WatchSource:0}: Error finding container b313584f2cb127e80af432525bdbab3a046e36f0f52de196bf0316bb82827545: Status 404 returned error can't find the container with id b313584f2cb127e80af432525bdbab3a046e36f0f52de196bf0316bb82827545
Sep 05 06:27:29.983212 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:29.982258    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:29.983212 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:29.982391    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:29.983212 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:29.982583    2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 05 06:27:29.983212 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:29.982652    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs podName:f9b3185c-0e0e-43aa-8e47-1090bfb5f016 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:31.98263227 +0000 UTC m=+68.573191206 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs") pod "router-default-74c88b6858-trsfj" (UID: "f9b3185c-0e0e-43aa-8e47-1090bfb5f016") : secret "router-metrics-certs-default" not found
Sep 05 06:27:29.983212 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:29.983133    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle podName:f9b3185c-0e0e-43aa-8e47-1090bfb5f016 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:31.98311592 +0000 UTC m=+68.573674857 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle") pod "router-default-74c88b6858-trsfj" (UID: "f9b3185c-0e0e-43aa-8e47-1090bfb5f016") : configmap references non-existent config key: service-ca.crt
Sep 05 06:27:30.295816 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:30.295767    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2kg8p" event={"ID":"81a69703-95b0-49e9-97e7-62fd2199968e","Type":"ContainerStarted","Data":"7743c4b0c6afba6f54156e00e6577ea459ac56d273df1d2ba902a0b83251afc7"}
Sep 05 06:27:30.297596 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:30.297569    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hsthh" event={"ID":"9c9536b7-5174-42f5-981d-1a6d1a14317e","Type":"ContainerStarted","Data":"b313584f2cb127e80af432525bdbab3a046e36f0f52de196bf0316bb82827545"}
Sep 05 06:27:30.315155 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:30.315090    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-2kg8p" podStartSLOduration=1.01752355 podStartE2EDuration="2.315068199s" totalImagesPullingTime="1.297544649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:30.314274126 +0000 UTC m=+66.904833079" watchObservedRunningTime="2026-09-05 06:27:30.315068199 +0000 UTC m=+66.905627151"
Sep 05 06:27:31.193858 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:31.193814    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:27:31.194259 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:31.194009    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 05 06:27:31.194259 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:31.194096    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls podName:3df3774c-273c-4b5b-9c63-785f0f683d72 nodeName:}" failed. No retries permitted until 2026-09-05 06:28:03.19407173 +0000 UTC m=+99.784630678 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls") pod "dns-default-s2wh5" (UID: "3df3774c-273c-4b5b-9c63-785f0f683d72") : secret "dns-default-metrics-tls" not found
Sep 05 06:27:32.000817 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.000772    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:32.001333 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.000845    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:32.001333 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:32.000966    2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 05 06:27:32.001333 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:32.001026    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle podName:f9b3185c-0e0e-43aa-8e47-1090bfb5f016 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:36.001004505 +0000 UTC m=+72.591563463 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle") pod "router-default-74c88b6858-trsfj" (UID: "f9b3185c-0e0e-43aa-8e47-1090bfb5f016") : configmap references non-existent config key: service-ca.crt
Sep 05 06:27:32.001333 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:32.001048    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs podName:f9b3185c-0e0e-43aa-8e47-1090bfb5f016 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:36.001039616 +0000 UTC m=+72.591598567 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs") pod "router-default-74c88b6858-trsfj" (UID: "f9b3185c-0e0e-43aa-8e47-1090bfb5f016") : secret "router-metrics-certs-default" not found
Sep 05 06:27:32.307250 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.307128    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg" event={"ID":"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2","Type":"ContainerStarted","Data":"4f1fcb0e041a07f0445d4531ba5c3e1bbe0679d480e6b536b424d4ac5ae40872"}
Sep 05 06:27:32.327540 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.327473    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg" podStartSLOduration=1.8761614039999999 podStartE2EDuration="4.32744969s" totalImagesPullingTime="2.451288286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:32.32730201 +0000 UTC m=+68.917860963" watchObservedRunningTime="2026-09-05 06:27:32.32744969 +0000 UTC m=+68.918008642"
Sep 05 06:27:32.567275 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.567184    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-h6j9h"]
Sep 05 06:27:32.569451 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.569425    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:32.572072 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.572041    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rb49r\""
Sep 05 06:27:32.572808 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.572615    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 05 06:27:32.572808 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.572722    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 05 06:27:32.583090 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.583062    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-h6j9h"]
Sep 05 06:27:32.606584 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.606543    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:32.606784 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.606628    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kftxx\" (UniqueName: \"kubernetes.io/projected/04da8f82-4950-40f1-b128-5c1fe193f655-kube-api-access-kftxx\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:32.606784 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.606676    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/04da8f82-4950-40f1-b128-5c1fe193f655-crio-socket\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:32.607061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.607016    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/04da8f82-4950-40f1-b128-5c1fe193f655-data-volume\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:32.607061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.607067    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:32.708087 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.708049    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/04da8f82-4950-40f1-b128-5c1fe193f655-data-volume\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:32.708087 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.708088    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:32.708335 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.708137    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:32.708335 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.708177    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kftxx\" (UniqueName: \"kubernetes.io/projected/04da8f82-4950-40f1-b128-5c1fe193f655-kube-api-access-kftxx\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:32.708335 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.708198    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/04da8f82-4950-40f1-b128-5c1fe193f655-crio-socket\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:32.708335 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.708276    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/04da8f82-4950-40f1-b128-5c1fe193f655-crio-socket\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:32.708335 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:32.708292    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 05 06:27:32.708553 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:32.708371    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:32.708553 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.708400    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/04da8f82-4950-40f1-b128-5c1fe193f655-data-volume\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:32.708553 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:32.708378    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm podName:04da8f82-4950-40f1-b128-5c1fe193f655 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:33.208354481 +0000 UTC m=+69.798913434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h6j9h" (UID: "04da8f82-4950-40f1-b128-5c1fe193f655") : configmap "kube-rbac-proxy" not found
Sep 05 06:27:32.708553 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:32.708454    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls podName:04da8f82-4950-40f1-b128-5c1fe193f655 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:33.20843941 +0000 UTC m=+69.798998339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h6j9h" (UID: "04da8f82-4950-40f1-b128-5c1fe193f655") : secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:32.717441 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.717410    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kftxx\" (UniqueName: \"kubernetes.io/projected/04da8f82-4950-40f1-b128-5c1fe193f655-kube-api-access-kftxx\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:32.982438 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.982401    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-z8lqv"]
Sep 05 06:27:32.984736 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.984712    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-z8lqv"
Sep 05 06:27:32.987918 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.987870    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 05 06:27:32.988066 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.987915    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 05 06:27:32.988066 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:32.987935    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-l8bfn\""
Sep 05 06:27:33.001013 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:33.000986    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-z8lqv"]
Sep 05 06:27:33.010950 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:33.010922    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7tfz\" (UniqueName: \"kubernetes.io/projected/2c624fb9-258f-4f9d-9dfc-00fa2e6ee87f-kube-api-access-x7tfz\") pod \"migrator-fdbb8fdff-z8lqv\" (UID: \"2c624fb9-258f-4f9d-9dfc-00fa2e6ee87f\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-z8lqv"
Sep 05 06:27:33.111974 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:33.111934    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7tfz\" (UniqueName: \"kubernetes.io/projected/2c624fb9-258f-4f9d-9dfc-00fa2e6ee87f-kube-api-access-x7tfz\") pod \"migrator-fdbb8fdff-z8lqv\" (UID: \"2c624fb9-258f-4f9d-9dfc-00fa2e6ee87f\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-z8lqv"
Sep 05 06:27:33.122196 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:33.122155    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7tfz\" (UniqueName: \"kubernetes.io/projected/2c624fb9-258f-4f9d-9dfc-00fa2e6ee87f-kube-api-access-x7tfz\") pod \"migrator-fdbb8fdff-z8lqv\" (UID: \"2c624fb9-258f-4f9d-9dfc-00fa2e6ee87f\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-z8lqv"
Sep 05 06:27:33.212507 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:33.212454    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:33.212711 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:33.212540    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:33.212711 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:33.212613    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 05 06:27:33.212711 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:33.212677    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:33.212711 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:33.212704    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm podName:04da8f82-4950-40f1-b128-5c1fe193f655 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:34.212683665 +0000 UTC m=+70.803242615 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h6j9h" (UID: "04da8f82-4950-40f1-b128-5c1fe193f655") : configmap "kube-rbac-proxy" not found
Sep 05 06:27:33.212938 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:33.212733    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls podName:04da8f82-4950-40f1-b128-5c1fe193f655 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:34.212717298 +0000 UTC m=+70.803276228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h6j9h" (UID: "04da8f82-4950-40f1-b128-5c1fe193f655") : secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:33.295948 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:33.295832    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-z8lqv"
Sep 05 06:27:34.223045 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:34.222979    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:34.223691 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:34.223085    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:34.223691 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:34.223152    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 05 06:27:34.223691 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:34.223251    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm podName:04da8f82-4950-40f1-b128-5c1fe193f655 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:36.223226492 +0000 UTC m=+72.813785422 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h6j9h" (UID: "04da8f82-4950-40f1-b128-5c1fe193f655") : configmap "kube-rbac-proxy" not found
Sep 05 06:27:34.223691 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:34.223321    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:34.223691 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:34.223409    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls podName:04da8f82-4950-40f1-b128-5c1fe193f655 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:36.223387608 +0000 UTC m=+72.813946568 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h6j9h" (UID: "04da8f82-4950-40f1-b128-5c1fe193f655") : secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:34.314672 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:34.314631    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tcnw5" event={"ID":"c85b9378-0915-419f-9b29-47bb580dc22a","Type":"ContainerStarted","Data":"15e65b452633fd3982bd128dc6b544133488be20e18e6a32c14971680240cfaf"}
Sep 05 06:27:34.320460 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:34.320411    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hsthh" event={"ID":"9c9536b7-5174-42f5-981d-1a6d1a14317e","Type":"ContainerStarted","Data":"d614e0535919437fdbd09bd58fdea0a1db30daefc2c6607c12593639973045b0"}
Sep 05 06:27:34.320641 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:34.320513    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:27:34.332174 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:34.332117    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tcnw5" podStartSLOduration=64.187890431 podStartE2EDuration="1m9.332100851s" totalImagesPullingTime="5.14421042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:34.331524194 +0000 UTC m=+70.922083149" watchObservedRunningTime="2026-09-05 06:27:34.332100851 +0000 UTC m=+70.922659803"
Sep 05 06:27:34.349466 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:34.349410    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hsthh" podStartSLOduration=64.550625888 podStartE2EDuration="1m9.349394183s" totalImagesPullingTime="4.798768295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:34.349045497 +0000 UTC m=+70.939604475" watchObservedRunningTime="2026-09-05 06:27:34.349394183 +0000 UTC m=+70.939953191"
Sep 05 06:27:34.405710 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:34.405621    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-z8lqv"]
Sep 05 06:27:34.728623 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:34.728594    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zs2d7_db932df4-aa61-4dd9-b590-ba22dabce334/dns-node-resolver/0.log"
Sep 05 06:27:35.324616 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:35.324581    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-z8lqv" event={"ID":"2c624fb9-258f-4f9d-9dfc-00fa2e6ee87f","Type":"ContainerStarted","Data":"0f52b3bf470caffd97dbdd61c9a99dcc684ebd9f773adcfbd4036ae1de04d6e1"}
Sep 05 06:27:35.528900 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:35.528859    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j8x2c_561b2ddc-4c98-434e-995b-a8338ac6cfd5/node-ca/0.log"
Sep 05 06:27:36.037540 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:36.037501    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:36.037722 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:36.037561    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:36.037722 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:36.037650    2559 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 05 06:27:36.037722 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:36.037700    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle podName:f9b3185c-0e0e-43aa-8e47-1090bfb5f016 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:44.037678532 +0000 UTC m=+80.628237462 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle") pod "router-default-74c88b6858-trsfj" (UID: "f9b3185c-0e0e-43aa-8e47-1090bfb5f016") : configmap references non-existent config key: service-ca.crt
Sep 05 06:27:36.037833 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:36.037728    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs podName:f9b3185c-0e0e-43aa-8e47-1090bfb5f016 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:44.037720713 +0000 UTC m=+80.628279643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs") pod "router-default-74c88b6858-trsfj" (UID: "f9b3185c-0e0e-43aa-8e47-1090bfb5f016") : secret "router-metrics-certs-default" not found
Sep 05 06:27:36.130533 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:36.130503    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-74htm_543b8b05-067c-49ff-81da-00f05d7e4454/serve-healthcheck-canary/0.log"
Sep 05 06:27:36.239135 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:36.239096    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:36.239316 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:36.239169    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:36.239316 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:36.239253    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 05 06:27:36.239316 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:36.239268    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:36.239316 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:36.239309    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm podName:04da8f82-4950-40f1-b128-5c1fe193f655 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:40.239294137 +0000 UTC m=+76.829853067 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h6j9h" (UID: "04da8f82-4950-40f1-b128-5c1fe193f655") : configmap "kube-rbac-proxy" not found
Sep 05 06:27:36.239494 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:36.239327    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls podName:04da8f82-4950-40f1-b128-5c1fe193f655 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:40.239314109 +0000 UTC m=+76.829873039 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h6j9h" (UID: "04da8f82-4950-40f1-b128-5c1fe193f655") : secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:36.329716 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:36.329619    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-z8lqv" event={"ID":"2c624fb9-258f-4f9d-9dfc-00fa2e6ee87f","Type":"ContainerStarted","Data":"686a4488d2457e7d559dcfa5092edba679018ed627aa1ec65b98d281203e4a24"}
Sep 05 06:27:36.329716 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:36.329680    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-z8lqv" event={"ID":"2c624fb9-258f-4f9d-9dfc-00fa2e6ee87f","Type":"ContainerStarted","Data":"d5112494ea6fc2c2bd5bb5eb73bb8a7b069fd2d1de2f5656a61fabe42aea82d6"}
Sep 05 06:27:36.347746 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:36.347679    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-z8lqv" podStartSLOduration=3.41590051 podStartE2EDuration="4.347662274s" totalImagesPullingTime="931.761764ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:27:36.347095687 +0000 UTC m=+72.937654640" watchObservedRunningTime="2026-09-05 06:27:36.347662274 +0000 UTC m=+72.938221226"
Sep 05 06:27:40.275140 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:40.275087    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:40.275626 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:40.275228    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 05 06:27:40.275626 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:40.275279    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:40.275626 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:40.275305    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm podName:04da8f82-4950-40f1-b128-5c1fe193f655 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:48.275287762 +0000 UTC m=+84.865846718 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm") pod "insights-runtime-extractor-h6j9h" (UID: "04da8f82-4950-40f1-b128-5c1fe193f655") : configmap "kube-rbac-proxy" not found
Sep 05 06:27:40.275626 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:40.275367    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:40.275626 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:40.275424    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls podName:04da8f82-4950-40f1-b128-5c1fe193f655 nodeName:}" failed. No retries permitted until 2026-09-05 06:27:48.275412105 +0000 UTC m=+84.865971035 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h6j9h" (UID: "04da8f82-4950-40f1-b128-5c1fe193f655") : secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:44.105989 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:44.105945    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:44.106374 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:44.106042    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:44.106374 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:44.106123    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle podName:f9b3185c-0e0e-43aa-8e47-1090bfb5f016 nodeName:}" failed. No retries permitted until 2026-09-05 06:28:00.106098963 +0000 UTC m=+96.696657906 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle") pod "router-default-74c88b6858-trsfj" (UID: "f9b3185c-0e0e-43aa-8e47-1090bfb5f016") : configmap references non-existent config key: service-ca.crt
Sep 05 06:27:44.108452 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:44.108435    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-metrics-certs\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:27:48.338553 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:48.338386    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:48.338553 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:48.338471    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:48.339210 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:48.338603    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:48.339210 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:27:48.338691    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls podName:04da8f82-4950-40f1-b128-5c1fe193f655 nodeName:}" failed. No retries permitted until 2026-09-05 06:28:04.338668387 +0000 UTC m=+100.929227333 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls") pod "insights-runtime-extractor-h6j9h" (UID: "04da8f82-4950-40f1-b128-5c1fe193f655") : secret "insights-runtime-extractor-tls" not found
Sep 05 06:27:48.339210 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:48.338987    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04da8f82-4950-40f1-b128-5c1fe193f655-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:27:56.918720 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:56.918682    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"]
Sep 05 06:27:56.923089 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:56.923065    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:56.926769 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:56.926737    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 05 06:27:56.926952 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:56.926789    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 05 06:27:56.926952 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:56.926813    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 05 06:27:56.926952 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:56.926867    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 05 06:27:56.926952 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:56.926813    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 05 06:27:56.926952 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:56.926904    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 05 06:27:56.927178 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:56.926909    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 05 06:27:56.933840 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:56.933818    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"]
Sep 05 06:27:57.006692 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.006649    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.006942 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.006709    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-hub\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.006942 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.006776    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.006942 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.006850    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.006942 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.006909    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-ca\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.007169 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.007055    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tktgh\" (UniqueName: \"kubernetes.io/projected/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-kube-api-access-tktgh\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.107424 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.107376    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.107424 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.107423    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-hub\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.107667 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.107447    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.107667 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.107646    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.107771 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.107684    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-ca\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.107933 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.107906    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tktgh\" (UniqueName: \"kubernetes.io/projected/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-kube-api-access-tktgh\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.108208 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.108186    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.110140 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.110107    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.110238 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.110150    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-ca\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.110478 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.110461    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-hub\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.110522 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.110496    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.124717 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.124690    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tktgh\" (UniqueName: \"kubernetes.io/projected/ceb957e4-b987-443b-9a60-3ad46a1ca8bc-kube-api-access-tktgh\") pod \"cluster-proxy-proxy-agent-6b5c9448fd-5x62g\" (UID: \"ceb957e4-b987-443b-9a60-3ad46a1ca8bc\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.240593 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.240554    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"
Sep 05 06:27:57.519037 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:57.518952    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g"]
Sep 05 06:27:57.522814 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:27:57.522780    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podceb957e4_b987_443b_9a60_3ad46a1ca8bc.slice/crio-68cc5712568bce4be366e12e6efd9981f0f4913889bf07e036836cee1784d24d WatchSource:0}: Error finding container 68cc5712568bce4be366e12e6efd9981f0f4913889bf07e036836cee1784d24d: Status 404 returned error can't find the container with id 68cc5712568bce4be366e12e6efd9981f0f4913889bf07e036836cee1784d24d
Sep 05 06:27:58.402410 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:27:58.402342    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g" event={"ID":"ceb957e4-b987-443b-9a60-3ad46a1ca8bc","Type":"ContainerStarted","Data":"68cc5712568bce4be366e12e6efd9981f0f4913889bf07e036836cee1784d24d"}
Sep 05 06:28:00.135095 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:00.135044    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:28:00.135763 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:00.135734    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9b3185c-0e0e-43aa-8e47-1090bfb5f016-service-ca-bundle\") pod \"router-default-74c88b6858-trsfj\" (UID: \"f9b3185c-0e0e-43aa-8e47-1090bfb5f016\") " pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:28:00.391502 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:00.391405    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:28:01.292819 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:01.292740    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-74c88b6858-trsfj"]
Sep 05 06:28:01.296050 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:28:01.296021    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9b3185c_0e0e_43aa_8e47_1090bfb5f016.slice/crio-4d7910091f6f3441de799280b888da06fb7301909e43e679ddb49715174a750c WatchSource:0}: Error finding container 4d7910091f6f3441de799280b888da06fb7301909e43e679ddb49715174a750c: Status 404 returned error can't find the container with id 4d7910091f6f3441de799280b888da06fb7301909e43e679ddb49715174a750c
Sep 05 06:28:01.415372 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:01.415333    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g" event={"ID":"ceb957e4-b987-443b-9a60-3ad46a1ca8bc","Type":"ContainerStarted","Data":"b79f4b6cc5ccbcd411dc15ad9884f6a5797412184409b917aa622c07560195fd"}
Sep 05 06:28:01.415372 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:01.415389    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g" event={"ID":"ceb957e4-b987-443b-9a60-3ad46a1ca8bc","Type":"ContainerStarted","Data":"b96709bcdc3e5d2482c391e9d40cdc623d27e79eb4e76410f5d25e08e669444a"}
Sep 05 06:28:01.415650 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:01.415410    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g" event={"ID":"ceb957e4-b987-443b-9a60-3ad46a1ca8bc","Type":"ContainerStarted","Data":"4737bd523284673f77a379e57dc877efea1af1bda4e5d4f810b8e966ea5e9843"}
Sep 05 06:28:01.416933 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:01.416904    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74c88b6858-trsfj" event={"ID":"f9b3185c-0e0e-43aa-8e47-1090bfb5f016","Type":"ContainerStarted","Data":"45c5796300cbee77669c067d6a4b0025645b0fe10405459564b486d9b2b5333e"}
Sep 05 06:28:01.417044 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:01.416952    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-74c88b6858-trsfj" event={"ID":"f9b3185c-0e0e-43aa-8e47-1090bfb5f016","Type":"ContainerStarted","Data":"4d7910091f6f3441de799280b888da06fb7301909e43e679ddb49715174a750c"}
Sep 05 06:28:01.437899 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:01.437846    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6b5c9448fd-5x62g" podStartSLOduration=1.865868283 podStartE2EDuration="5.437828928s" totalImagesPullingTime="3.571960645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:01.436875215 +0000 UTC m=+98.027434166" watchObservedRunningTime="2026-09-05 06:28:01.437828928 +0000 UTC m=+98.028387983"
Sep 05 06:28:01.456449 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:01.456402    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-74c88b6858-trsfj" podStartSLOduration=33.456387234 podStartE2EDuration="33.456387234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:27:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:01.455571517 +0000 UTC m=+98.046130470" watchObservedRunningTime="2026-09-05 06:28:01.456387234 +0000 UTC m=+98.046946185"
Sep 05 06:28:02.392229 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:02.392137    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:28:02.394832 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:02.394805    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:28:02.420772 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:02.420741    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:28:02.422146 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:02.422117    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-74c88b6858-trsfj"
Sep 05 06:28:03.265552 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:03.265499    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:28:03.267910 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:03.267872    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3df3774c-273c-4b5b-9c63-785f0f683d72-metrics-tls\") pod \"dns-default-s2wh5\" (UID: \"3df3774c-273c-4b5b-9c63-785f0f683d72\") " pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:28:03.511072 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:03.511036    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v76tl\""
Sep 05 06:28:03.519174 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:03.519119    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:28:03.819031 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:03.818934    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-s2wh5"]
Sep 05 06:28:03.823101 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:28:03.823072    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3df3774c_273c_4b5b_9c63_785f0f683d72.slice/crio-6516ed3b9f6fe3a2938f7e146c56f4365ced1c1266c553b9e8b52b6cd50b7da6 WatchSource:0}: Error finding container 6516ed3b9f6fe3a2938f7e146c56f4365ced1c1266c553b9e8b52b6cd50b7da6: Status 404 returned error can't find the container with id 6516ed3b9f6fe3a2938f7e146c56f4365ced1c1266c553b9e8b52b6cd50b7da6
Sep 05 06:28:04.373697 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:04.373648    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:28:04.376237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:04.376212    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04da8f82-4950-40f1-b128-5c1fe193f655-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-h6j9h\" (UID: \"04da8f82-4950-40f1-b128-5c1fe193f655\") " pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:28:04.382205 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:04.382182    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-h6j9h"
Sep 05 06:28:04.427645 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:04.427608    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s2wh5" event={"ID":"3df3774c-273c-4b5b-9c63-785f0f683d72","Type":"ContainerStarted","Data":"6516ed3b9f6fe3a2938f7e146c56f4365ced1c1266c553b9e8b52b6cd50b7da6"}
Sep 05 06:28:04.683232 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:04.682833    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-h6j9h"]
Sep 05 06:28:04.685946 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:28:04.685909    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04da8f82_4950_40f1_b128_5c1fe193f655.slice/crio-efed09c67a13ce2e772b97b4597c7a131e2c7102f5356fa421f9fa063a691e18 WatchSource:0}: Error finding container efed09c67a13ce2e772b97b4597c7a131e2c7102f5356fa421f9fa063a691e18: Status 404 returned error can't find the container with id efed09c67a13ce2e772b97b4597c7a131e2c7102f5356fa421f9fa063a691e18
Sep 05 06:28:05.328283 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:05.328248    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hsthh"
Sep 05 06:28:05.434221 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:05.434185    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-h6j9h" event={"ID":"04da8f82-4950-40f1-b128-5c1fe193f655","Type":"ContainerStarted","Data":"d9710dfd8ebe317dad64ae0a51d33b36909c3df00278c112bcc3f4b7a58c62ec"}
Sep 05 06:28:05.434382 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:05.434257    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-h6j9h" event={"ID":"04da8f82-4950-40f1-b128-5c1fe193f655","Type":"ContainerStarted","Data":"efed09c67a13ce2e772b97b4597c7a131e2c7102f5356fa421f9fa063a691e18"}
Sep 05 06:28:06.439070 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:06.439025    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s2wh5" event={"ID":"3df3774c-273c-4b5b-9c63-785f0f683d72","Type":"ContainerStarted","Data":"737095b3512e323d2dfc82d496dea31a1850c51f9234793d1982dc823ba5fb35"}
Sep 05 06:28:06.439463 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:06.439100    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s2wh5" event={"ID":"3df3774c-273c-4b5b-9c63-785f0f683d72","Type":"ContainerStarted","Data":"447faca7fd76ab0f438d62411464c378b551b9f47208694d78032a4a172b7a7d"}
Sep 05 06:28:06.439463 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:06.439130    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:28:06.460601 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:06.460553    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-s2wh5" podStartSLOduration=65.244130574 podStartE2EDuration="1m7.460537635s" totalImagesPullingTime="2.216407061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:26:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:06.460184472 +0000 UTC m=+103.050743426" watchObservedRunningTime="2026-09-05 06:28:06.460537635 +0000 UTC m=+103.051096587"
Sep 05 06:28:07.445303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:07.445264    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-h6j9h" event={"ID":"04da8f82-4950-40f1-b128-5c1fe193f655","Type":"ContainerStarted","Data":"b32dd3a29aad3e8bed19f0394d4fd0e0f2da05db7aa5390f70ef025017f207c7"}
Sep 05 06:28:08.450992 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:08.450953    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-h6j9h" event={"ID":"04da8f82-4950-40f1-b128-5c1fe193f655","Type":"ContainerStarted","Data":"14f20512e64c692539dde6bbb409d2c32da87bc6b360ca4e51465cc8be16854d"}
Sep 05 06:28:08.472190 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:08.472136    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-h6j9h" podStartSLOduration=33.617432585 podStartE2EDuration="36.472119503s" totalImagesPullingTime="2.854686918s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 06:27:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:08.471770508 +0000 UTC m=+105.062329462" watchObservedRunningTime="2026-09-05 06:28:08.472119503 +0000 UTC m=+105.062678456"
Sep 05 06:28:10.353936 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.353898    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-576l9"]
Sep 05 06:28:10.356161 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.356143    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.359962 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.359918    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 05 06:28:10.360236 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.360133    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 05 06:28:10.361378 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.361076    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 05 06:28:10.361378 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.361186    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 05 06:28:10.361651 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.361441    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bcsr2\""
Sep 05 06:28:10.361651 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.361631    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 05 06:28:10.362924 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.362895    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 05 06:28:10.532132 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.532091    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/443c621a-8883-486c-9fcf-98a0a117d462-sys\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.532303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.532148    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/443c621a-8883-486c-9fcf-98a0a117d462-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.532303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.532219    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/443c621a-8883-486c-9fcf-98a0a117d462-node-exporter-accelerators-collector-config\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.532303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.532267    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/443c621a-8883-486c-9fcf-98a0a117d462-node-exporter-wtmp\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.532303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.532298    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/443c621a-8883-486c-9fcf-98a0a117d462-node-exporter-tls\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.532445 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.532315    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct4zj\" (UniqueName: \"kubernetes.io/projected/443c621a-8883-486c-9fcf-98a0a117d462-kube-api-access-ct4zj\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.532445 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.532394    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/443c621a-8883-486c-9fcf-98a0a117d462-root\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.532445 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.532410    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/443c621a-8883-486c-9fcf-98a0a117d462-node-exporter-textfile\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.532445 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.532431    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/443c621a-8883-486c-9fcf-98a0a117d462-metrics-client-ca\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.633588 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.633450    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/443c621a-8883-486c-9fcf-98a0a117d462-root\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.633798 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.633578    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/443c621a-8883-486c-9fcf-98a0a117d462-root\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.633798 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.633606    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/443c621a-8883-486c-9fcf-98a0a117d462-node-exporter-textfile\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.633798 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.633668    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/443c621a-8883-486c-9fcf-98a0a117d462-metrics-client-ca\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.633798 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.633696    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/443c621a-8883-486c-9fcf-98a0a117d462-sys\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.633798 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.633757    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/443c621a-8883-486c-9fcf-98a0a117d462-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.634088 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.633810    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/443c621a-8883-486c-9fcf-98a0a117d462-node-exporter-accelerators-collector-config\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.634088 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.633846    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/443c621a-8883-486c-9fcf-98a0a117d462-node-exporter-wtmp\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.634088 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.633907    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/443c621a-8883-486c-9fcf-98a0a117d462-node-exporter-tls\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.634088 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.633877    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/443c621a-8883-486c-9fcf-98a0a117d462-sys\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.634088 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.633933    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct4zj\" (UniqueName: \"kubernetes.io/projected/443c621a-8883-486c-9fcf-98a0a117d462-kube-api-access-ct4zj\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.634088 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.634068    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/443c621a-8883-486c-9fcf-98a0a117d462-node-exporter-textfile\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.634399 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.634237    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/443c621a-8883-486c-9fcf-98a0a117d462-node-exporter-wtmp\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.634566 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.634527    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/443c621a-8883-486c-9fcf-98a0a117d462-metrics-client-ca\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.634732 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.634707    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/443c621a-8883-486c-9fcf-98a0a117d462-node-exporter-accelerators-collector-config\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.637272 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.637239    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/443c621a-8883-486c-9fcf-98a0a117d462-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.637417 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.637367    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/443c621a-8883-486c-9fcf-98a0a117d462-node-exporter-tls\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.643728 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.643701    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct4zj\" (UniqueName: \"kubernetes.io/projected/443c621a-8883-486c-9fcf-98a0a117d462-kube-api-access-ct4zj\") pod \"node-exporter-576l9\" (UID: \"443c621a-8883-486c-9fcf-98a0a117d462\") " pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.666718 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:10.666684    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-576l9"
Sep 05 06:28:10.677950 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:28:10.677923    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod443c621a_8883_486c_9fcf_98a0a117d462.slice/crio-904fed38c4be44b200a23d61b37431c703979b7875400e3dfcec4702d4acb5a1 WatchSource:0}: Error finding container 904fed38c4be44b200a23d61b37431c703979b7875400e3dfcec4702d4acb5a1: Status 404 returned error can't find the container with id 904fed38c4be44b200a23d61b37431c703979b7875400e3dfcec4702d4acb5a1
Sep 05 06:28:11.464685 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:11.464637    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-576l9" event={"ID":"443c621a-8883-486c-9fcf-98a0a117d462","Type":"ContainerStarted","Data":"904fed38c4be44b200a23d61b37431c703979b7875400e3dfcec4702d4acb5a1"}
Sep 05 06:28:12.469439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:12.469402    2559 generic.go:356] "Generic (PLEG): container finished" podID="443c621a-8883-486c-9fcf-98a0a117d462" containerID="2cf00c2a14b0a0ae6ff31dcd5812d5af64f129fb5d2a83c556e63399580a3c04" exitCode=0
Sep 05 06:28:12.469978 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:12.469459    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-576l9" event={"ID":"443c621a-8883-486c-9fcf-98a0a117d462","Type":"ContainerDied","Data":"2cf00c2a14b0a0ae6ff31dcd5812d5af64f129fb5d2a83c556e63399580a3c04"}
Sep 05 06:28:13.474956 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:13.474920    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-576l9" event={"ID":"443c621a-8883-486c-9fcf-98a0a117d462","Type":"ContainerStarted","Data":"1854eb3e642d2d21b926d71a4e94001d37e28d4fc75e59cf1e7e2439f09498e9"}
Sep 05 06:28:13.475347 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:13.474989    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-576l9" event={"ID":"443c621a-8883-486c-9fcf-98a0a117d462","Type":"ContainerStarted","Data":"5bd6c9fe33ab171db96177c56d87556a01e0a227df05f2c981f153464034377f"}
Sep 05 06:28:13.497922 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:13.497856    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-576l9" podStartSLOduration=2.547525039 podStartE2EDuration="3.497838628s" totalImagesPullingTime="887.749964ms" totalInitContainerRuntime="62.563625ms" isStatefulPod=true podCreationTimestamp="2026-09-05 06:28:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:13.496209198 +0000 UTC m=+110.086768161" watchObservedRunningTime="2026-09-05 06:28:13.497838628 +0000 UTC m=+110.088397579"
Sep 05 06:28:16.448063 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:16.448019    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-s2wh5"
Sep 05 06:28:31.573663 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.573630    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b5fdf546f-mqdvg"]
Sep 05 06:28:31.576302 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.576277    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.579296 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.579256    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 05 06:28:31.579296 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.579256    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 05 06:28:31.580499 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.580440    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 05 06:28:31.580499 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.580440    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 05 06:28:31.580499 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.580490    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 05 06:28:31.580779 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.580448    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 05 06:28:31.580779 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.580447    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 05 06:28:31.580779 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.580447    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7r4f6\""
Sep 05 06:28:31.584138 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.584111    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-oauth-serving-cert\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.584283 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.584148    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07974e55-64db-40e2-be8a-d36898d60e9c-console-serving-cert\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.584283 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.584192    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rtwx\" (UniqueName: \"kubernetes.io/projected/07974e55-64db-40e2-be8a-d36898d60e9c-kube-api-access-7rtwx\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.584393 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.584273    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-trusted-ca-bundle\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.584393 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.584313    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-console-config\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.584393 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.584345    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07974e55-64db-40e2-be8a-d36898d60e9c-console-oauth-config\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.584858 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.584432    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-service-ca\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.590128 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.590103    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 05 06:28:31.591515 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.591490    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b5fdf546f-mqdvg"]
Sep 05 06:28:31.685896 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.685849    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rtwx\" (UniqueName: \"kubernetes.io/projected/07974e55-64db-40e2-be8a-d36898d60e9c-kube-api-access-7rtwx\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.686077 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.685913    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-trusted-ca-bundle\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.686077 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.685934    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-console-config\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.686229 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.686103    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07974e55-64db-40e2-be8a-d36898d60e9c-console-oauth-config\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.686229 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.686210    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-service-ca\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.686332 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.686315    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-oauth-serving-cert\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.686385 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.686354    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07974e55-64db-40e2-be8a-d36898d60e9c-console-serving-cert\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.686659 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.686630    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-console-config\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.686815 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.686792    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-trusted-ca-bundle\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.686908 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.686805    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-service-ca\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.687054 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.687027    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-oauth-serving-cert\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.688762 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.688738    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07974e55-64db-40e2-be8a-d36898d60e9c-console-serving-cert\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.688762 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.688746    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07974e55-64db-40e2-be8a-d36898d60e9c-console-oauth-config\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.695973 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.695953    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rtwx\" (UniqueName: \"kubernetes.io/projected/07974e55-64db-40e2-be8a-d36898d60e9c-kube-api-access-7rtwx\") pod \"console-7b5fdf546f-mqdvg\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") " pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:31.888107 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:31.888002    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:32.162938 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:32.162838    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b5fdf546f-mqdvg"]
Sep 05 06:28:32.166824 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:28:32.166794    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07974e55_64db_40e2_be8a_d36898d60e9c.slice/crio-2aa6c62cfb234469488e032b2156f097aae0b5b8cd445843e9816ae7ed2d41d6 WatchSource:0}: Error finding container 2aa6c62cfb234469488e032b2156f097aae0b5b8cd445843e9816ae7ed2d41d6: Status 404 returned error can't find the container with id 2aa6c62cfb234469488e032b2156f097aae0b5b8cd445843e9816ae7ed2d41d6
Sep 05 06:28:32.539201 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:32.539161    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b5fdf546f-mqdvg" event={"ID":"07974e55-64db-40e2-be8a-d36898d60e9c","Type":"ContainerStarted","Data":"2aa6c62cfb234469488e032b2156f097aae0b5b8cd445843e9816ae7ed2d41d6"}
Sep 05 06:28:32.692349 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:32.692297    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs\") pod \"network-metrics-daemon-jslwg\" (UID: \"99bf56fa-eafd-489b-9d5d-75f31b076c45\") " pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:28:32.695082 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:32.695056    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99bf56fa-eafd-489b-9d5d-75f31b076c45-metrics-certs\") pod \"network-metrics-daemon-jslwg\" (UID: \"99bf56fa-eafd-489b-9d5d-75f31b076c45\") " pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:28:32.900086 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:32.899998    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4m5kh\""
Sep 05 06:28:32.907493 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:32.907461    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jslwg"
Sep 05 06:28:33.202402 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:33.202362    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jslwg"]
Sep 05 06:28:33.206550 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:28:33.206518    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99bf56fa_eafd_489b_9d5d_75f31b076c45.slice/crio-8ee9364d0a2d50e79f5e7f43ff1946d74a1d4fb2a7b85cd717ab19584c6ca17c WatchSource:0}: Error finding container 8ee9364d0a2d50e79f5e7f43ff1946d74a1d4fb2a7b85cd717ab19584c6ca17c: Status 404 returned error can't find the container with id 8ee9364d0a2d50e79f5e7f43ff1946d74a1d4fb2a7b85cd717ab19584c6ca17c
Sep 05 06:28:33.545123 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:33.545031    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jslwg" event={"ID":"99bf56fa-eafd-489b-9d5d-75f31b076c45","Type":"ContainerStarted","Data":"8ee9364d0a2d50e79f5e7f43ff1946d74a1d4fb2a7b85cd717ab19584c6ca17c"}
Sep 05 06:28:35.554159 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:35.554063    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jslwg" event={"ID":"99bf56fa-eafd-489b-9d5d-75f31b076c45","Type":"ContainerStarted","Data":"5f9a9e0c95b12725bb707b458000c4a6c2b17b0225884904c6960618c31d75c6"}
Sep 05 06:28:35.554159 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:35.554128    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jslwg" event={"ID":"99bf56fa-eafd-489b-9d5d-75f31b076c45","Type":"ContainerStarted","Data":"fd8b23548a2210352935595adf1389c7fff5f38467f0dd7cec4dfc3efb65f89d"}
Sep 05 06:28:35.555704 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:35.555683    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b5fdf546f-mqdvg" event={"ID":"07974e55-64db-40e2-be8a-d36898d60e9c","Type":"ContainerStarted","Data":"68a25c5c5ccdc9ea1df504c39951dcc932a30d2a06b45a01dd257d13534a8c67"}
Sep 05 06:28:35.571489 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:35.571427    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jslwg" podStartSLOduration=128.587152913 podStartE2EDuration="2m10.571407339s" totalImagesPullingTime="1.984254426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:35.570472813 +0000 UTC m=+132.161031765" watchObservedRunningTime="2026-09-05 06:28:35.571407339 +0000 UTC m=+132.161966292"
Sep 05 06:28:35.588977 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:35.588923    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b5fdf546f-mqdvg" podStartSLOduration=1.559996285 podStartE2EDuration="4.588906082s" totalImagesPullingTime="3.028909797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:28:35.587559171 +0000 UTC m=+132.178118123" watchObservedRunningTime="2026-09-05 06:28:35.588906082 +0000 UTC m=+132.179465039"
Sep 05 06:28:41.890573 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:41.890537    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:41.891097 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:41.890600    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:41.893677 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:41.893658    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:42.583896 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:42.583856    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:28:47.597869 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:47.597830    2559 generic.go:356] "Generic (PLEG): container finished" podID="f8057dec-2f84-4037-9fc9-b7e0d68c5cc2" containerID="4f1fcb0e041a07f0445d4531ba5c3e1bbe0679d480e6b536b424d4ac5ae40872" exitCode=0
Sep 05 06:28:47.598284 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:47.597911    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg" event={"ID":"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2","Type":"ContainerDied","Data":"4f1fcb0e041a07f0445d4531ba5c3e1bbe0679d480e6b536b424d4ac5ae40872"}
Sep 05 06:28:47.598344 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:47.598329    2559 scope.go:117] "RemoveContainer" containerID="4f1fcb0e041a07f0445d4531ba5c3e1bbe0679d480e6b536b424d4ac5ae40872"
Sep 05 06:28:48.605565 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:48.605530    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-gjwjg" event={"ID":"f8057dec-2f84-4037-9fc9-b7e0d68c5cc2","Type":"ContainerStarted","Data":"b8df9fcfa12d852dcf105d47b60213eecd5149139e6fa2c606c59d529b693ae4"}
Sep 05 06:28:48.748467 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:48.748439    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74c88b6858-trsfj_f9b3185c-0e0e-43aa-8e47-1090bfb5f016/router/0.log"
Sep 05 06:28:48.755011 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:28:48.754984    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-74htm_543b8b05-067c-49ff-81da-00f05d7e4454/serve-healthcheck-canary/0.log"
Sep 05 06:29:40.186125 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:29:40.186036    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b5fdf546f-mqdvg"]
Sep 05 06:30:05.206135 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.206051    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b5fdf546f-mqdvg" podUID="07974e55-64db-40e2-be8a-d36898d60e9c" containerName="console" containerID="cri-o://68a25c5c5ccdc9ea1df504c39951dcc932a30d2a06b45a01dd257d13534a8c67" gracePeriod=15
Sep 05 06:30:05.452800 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.452768    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:30:05.501274 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.501187    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-console-config\") pod \"07974e55-64db-40e2-be8a-d36898d60e9c\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") "
Sep 05 06:30:05.501274 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.501270    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rtwx\" (UniqueName: \"kubernetes.io/projected/07974e55-64db-40e2-be8a-d36898d60e9c-kube-api-access-7rtwx\") pod \"07974e55-64db-40e2-be8a-d36898d60e9c\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") "
Sep 05 06:30:05.501502 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.501294    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-trusted-ca-bundle\") pod \"07974e55-64db-40e2-be8a-d36898d60e9c\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") "
Sep 05 06:30:05.501502 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.501312    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-service-ca\") pod \"07974e55-64db-40e2-be8a-d36898d60e9c\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") "
Sep 05 06:30:05.501502 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.501348    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-oauth-serving-cert\") pod \"07974e55-64db-40e2-be8a-d36898d60e9c\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") "
Sep 05 06:30:05.501502 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.501404    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07974e55-64db-40e2-be8a-d36898d60e9c-console-oauth-config\") pod \"07974e55-64db-40e2-be8a-d36898d60e9c\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") "
Sep 05 06:30:05.501502 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.501438    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07974e55-64db-40e2-be8a-d36898d60e9c-console-serving-cert\") pod \"07974e55-64db-40e2-be8a-d36898d60e9c\" (UID: \"07974e55-64db-40e2-be8a-d36898d60e9c\") "
Sep 05 06:30:05.501788 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.501748    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-console-config" (OuterVolumeSpecName: "console-config") pod "07974e55-64db-40e2-be8a-d36898d60e9c" (UID: "07974e55-64db-40e2-be8a-d36898d60e9c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:30:05.501788 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.501784    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "07974e55-64db-40e2-be8a-d36898d60e9c" (UID: "07974e55-64db-40e2-be8a-d36898d60e9c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:30:05.501999 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.501857    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-service-ca" (OuterVolumeSpecName: "service-ca") pod "07974e55-64db-40e2-be8a-d36898d60e9c" (UID: "07974e55-64db-40e2-be8a-d36898d60e9c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:30:05.501999 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.501900    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "07974e55-64db-40e2-be8a-d36898d60e9c" (UID: "07974e55-64db-40e2-be8a-d36898d60e9c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:30:05.503766 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.503735    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07974e55-64db-40e2-be8a-d36898d60e9c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "07974e55-64db-40e2-be8a-d36898d60e9c" (UID: "07974e55-64db-40e2-be8a-d36898d60e9c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:30:05.503845 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.503754    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07974e55-64db-40e2-be8a-d36898d60e9c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "07974e55-64db-40e2-be8a-d36898d60e9c" (UID: "07974e55-64db-40e2-be8a-d36898d60e9c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:30:05.503845 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.503810    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07974e55-64db-40e2-be8a-d36898d60e9c-kube-api-access-7rtwx" (OuterVolumeSpecName: "kube-api-access-7rtwx") pod "07974e55-64db-40e2-be8a-d36898d60e9c" (UID: "07974e55-64db-40e2-be8a-d36898d60e9c"). InnerVolumeSpecName "kube-api-access-7rtwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:30:05.603005 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.602969    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rtwx\" (UniqueName: \"kubernetes.io/projected/07974e55-64db-40e2-be8a-d36898d60e9c-kube-api-access-7rtwx\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:30:05.603005 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.602999    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-trusted-ca-bundle\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:30:05.603005 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.603009    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-service-ca\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:30:05.603005 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.603018    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-oauth-serving-cert\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:30:05.603278 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.603026    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07974e55-64db-40e2-be8a-d36898d60e9c-console-oauth-config\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:30:05.603278 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.603035    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07974e55-64db-40e2-be8a-d36898d60e9c-console-serving-cert\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:30:05.603278 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.603043    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07974e55-64db-40e2-be8a-d36898d60e9c-console-config\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:30:05.889976 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.889874    2559 generic.go:356] "Generic (PLEG): container finished" podID="07974e55-64db-40e2-be8a-d36898d60e9c" containerID="68a25c5c5ccdc9ea1df504c39951dcc932a30d2a06b45a01dd257d13534a8c67" exitCode=0
Sep 05 06:30:05.890111 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.889979    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b5fdf546f-mqdvg"
Sep 05 06:30:05.890325 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.890307    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b5fdf546f-mqdvg" event={"ID":"07974e55-64db-40e2-be8a-d36898d60e9c","Type":"ContainerDied","Data":"68a25c5c5ccdc9ea1df504c39951dcc932a30d2a06b45a01dd257d13534a8c67"}
Sep 05 06:30:05.890393 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.890349    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b5fdf546f-mqdvg" event={"ID":"07974e55-64db-40e2-be8a-d36898d60e9c","Type":"ContainerDied","Data":"2aa6c62cfb234469488e032b2156f097aae0b5b8cd445843e9816ae7ed2d41d6"}
Sep 05 06:30:05.890447 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.890392    2559 scope.go:117] "RemoveContainer" containerID="68a25c5c5ccdc9ea1df504c39951dcc932a30d2a06b45a01dd257d13534a8c67"
Sep 05 06:30:05.901760 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.901737    2559 scope.go:117] "RemoveContainer" containerID="68a25c5c5ccdc9ea1df504c39951dcc932a30d2a06b45a01dd257d13534a8c67"
Sep 05 06:30:05.902163 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:30:05.902139    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68a25c5c5ccdc9ea1df504c39951dcc932a30d2a06b45a01dd257d13534a8c67\": container with ID starting with 68a25c5c5ccdc9ea1df504c39951dcc932a30d2a06b45a01dd257d13534a8c67 not found: ID does not exist" containerID="68a25c5c5ccdc9ea1df504c39951dcc932a30d2a06b45a01dd257d13534a8c67"
Sep 05 06:30:05.902252 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.902171    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68a25c5c5ccdc9ea1df504c39951dcc932a30d2a06b45a01dd257d13534a8c67"} err="failed to get container status \"68a25c5c5ccdc9ea1df504c39951dcc932a30d2a06b45a01dd257d13534a8c67\": rpc error: code = NotFound desc = could not find container \"68a25c5c5ccdc9ea1df504c39951dcc932a30d2a06b45a01dd257d13534a8c67\": container with ID starting with 68a25c5c5ccdc9ea1df504c39951dcc932a30d2a06b45a01dd257d13534a8c67 not found: ID does not exist"
Sep 05 06:30:05.921154 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.921120    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b5fdf546f-mqdvg"]
Sep 05 06:30:05.925500 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:05.925474    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b5fdf546f-mqdvg"]
Sep 05 06:30:07.030605 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.030567    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq"]
Sep 05 06:30:07.031028 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.030828    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07974e55-64db-40e2-be8a-d36898d60e9c" containerName="console"
Sep 05 06:30:07.031028 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.030841    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="07974e55-64db-40e2-be8a-d36898d60e9c" containerName="console"
Sep 05 06:30:07.031028 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.030909    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="07974e55-64db-40e2-be8a-d36898d60e9c" containerName="console"
Sep 05 06:30:07.035133 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.035113    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq"
Sep 05 06:30:07.038001 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.037980    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 05 06:30:07.038114 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.037980    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 05 06:30:07.038949 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.038933    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g76sk\""
Sep 05 06:30:07.043338 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.043311    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq"]
Sep 05 06:30:07.113366 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.113317    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zwr5\" (UniqueName: \"kubernetes.io/projected/022aa623-e479-4722-9b16-969312890c87-kube-api-access-4zwr5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq\" (UID: \"022aa623-e479-4722-9b16-969312890c87\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq"
Sep 05 06:30:07.113545 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.113380    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/022aa623-e479-4722-9b16-969312890c87-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq\" (UID: \"022aa623-e479-4722-9b16-969312890c87\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq"
Sep 05 06:30:07.113545 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.113440    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/022aa623-e479-4722-9b16-969312890c87-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq\" (UID: \"022aa623-e479-4722-9b16-969312890c87\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq"
Sep 05 06:30:07.214376 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.214328    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/022aa623-e479-4722-9b16-969312890c87-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq\" (UID: \"022aa623-e479-4722-9b16-969312890c87\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq"
Sep 05 06:30:07.214477 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.214392    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/022aa623-e479-4722-9b16-969312890c87-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq\" (UID: \"022aa623-e479-4722-9b16-969312890c87\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq"
Sep 05 06:30:07.214477 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.214459    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zwr5\" (UniqueName: \"kubernetes.io/projected/022aa623-e479-4722-9b16-969312890c87-kube-api-access-4zwr5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq\" (UID: \"022aa623-e479-4722-9b16-969312890c87\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq"
Sep 05 06:30:07.214808 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.214779    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/022aa623-e479-4722-9b16-969312890c87-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq\" (UID: \"022aa623-e479-4722-9b16-969312890c87\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq"
Sep 05 06:30:07.214808 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.214809    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/022aa623-e479-4722-9b16-969312890c87-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq\" (UID: \"022aa623-e479-4722-9b16-969312890c87\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq"
Sep 05 06:30:07.224901 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.224859    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zwr5\" (UniqueName: \"kubernetes.io/projected/022aa623-e479-4722-9b16-969312890c87-kube-api-access-4zwr5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq\" (UID: \"022aa623-e479-4722-9b16-969312890c87\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq"
Sep 05 06:30:07.345226 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.345117    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq"
Sep 05 06:30:07.621076 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.621000    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq"]
Sep 05 06:30:07.623323 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:30:07.623294    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod022aa623_e479_4722_9b16_969312890c87.slice/crio-e4d8326f1065ac1fecfde56ab5d8a1f40694acab7d853aafaeb85c650bdada37 WatchSource:0}: Error finding container e4d8326f1065ac1fecfde56ab5d8a1f40694acab7d853aafaeb85c650bdada37: Status 404 returned error can't find the container with id e4d8326f1065ac1fecfde56ab5d8a1f40694acab7d853aafaeb85c650bdada37
Sep 05 06:30:07.890403 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.890310    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07974e55-64db-40e2-be8a-d36898d60e9c" path="/var/lib/kubelet/pods/07974e55-64db-40e2-be8a-d36898d60e9c/volumes"
Sep 05 06:30:07.898374 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:07.898334    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq" event={"ID":"022aa623-e479-4722-9b16-969312890c87","Type":"ContainerStarted","Data":"e4d8326f1065ac1fecfde56ab5d8a1f40694acab7d853aafaeb85c650bdada37"}
Sep 05 06:30:13.925689 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:13.925649    2559 generic.go:356] "Generic (PLEG): container finished" podID="022aa623-e479-4722-9b16-969312890c87" containerID="1ae4da98653a8b9a8ab67e9ec3287f04fdabf1282ecc091a9dd8175ab015b503" exitCode=0
Sep 05 06:30:13.926152 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:13.925733    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq" event={"ID":"022aa623-e479-4722-9b16-969312890c87","Type":"ContainerDied","Data":"1ae4da98653a8b9a8ab67e9ec3287f04fdabf1282ecc091a9dd8175ab015b503"}
Sep 05 06:30:16.938700 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:16.938665    2559 generic.go:356] "Generic (PLEG): container finished" podID="022aa623-e479-4722-9b16-969312890c87" containerID="549117bd149ab632cba797d811a4ebffb445de3b80a8dc1a5510a41587737442" exitCode=0
Sep 05 06:30:16.939181 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:16.938742    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq" event={"ID":"022aa623-e479-4722-9b16-969312890c87","Type":"ContainerDied","Data":"549117bd149ab632cba797d811a4ebffb445de3b80a8dc1a5510a41587737442"}
Sep 05 06:30:23.967573 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:23.967531    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq" event={"ID":"022aa623-e479-4722-9b16-969312890c87","Type":"ContainerStarted","Data":"944d989d510bad65e656403f385d5b603c96d1f3a1db6e4b89deb2a84401b71a"}
Sep 05 06:30:23.987757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:23.987712    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq" podStartSLOduration=2.116399963 podStartE2EDuration="16.987697135s" totalImagesPullingTime="14.871297172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:30:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:30:23.987346158 +0000 UTC m=+240.577905123" watchObservedRunningTime="2026-09-05 06:30:23.987697135 +0000 UTC m=+240.578256086"
Sep 05 06:30:24.972524 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:24.972488    2559 generic.go:356] "Generic (PLEG): container finished" podID="022aa623-e479-4722-9b16-969312890c87" containerID="944d989d510bad65e656403f385d5b603c96d1f3a1db6e4b89deb2a84401b71a" exitCode=0
Sep 05 06:30:24.972931 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:24.972586    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq" event={"ID":"022aa623-e479-4722-9b16-969312890c87","Type":"ContainerDied","Data":"944d989d510bad65e656403f385d5b603c96d1f3a1db6e4b89deb2a84401b71a"}
Sep 05 06:30:26.103456 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:26.103433    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq"
Sep 05 06:30:26.173468 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:26.173442    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/022aa623-e479-4722-9b16-969312890c87-bundle\") pod \"022aa623-e479-4722-9b16-969312890c87\" (UID: \"022aa623-e479-4722-9b16-969312890c87\") "
Sep 05 06:30:26.173603 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:26.173525    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zwr5\" (UniqueName: \"kubernetes.io/projected/022aa623-e479-4722-9b16-969312890c87-kube-api-access-4zwr5\") pod \"022aa623-e479-4722-9b16-969312890c87\" (UID: \"022aa623-e479-4722-9b16-969312890c87\") "
Sep 05 06:30:26.173603 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:26.173580    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/022aa623-e479-4722-9b16-969312890c87-util\") pod \"022aa623-e479-4722-9b16-969312890c87\" (UID: \"022aa623-e479-4722-9b16-969312890c87\") "
Sep 05 06:30:26.174165 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:26.174141    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/022aa623-e479-4722-9b16-969312890c87-bundle" (OuterVolumeSpecName: "bundle") pod "022aa623-e479-4722-9b16-969312890c87" (UID: "022aa623-e479-4722-9b16-969312890c87"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:30:26.175596 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:26.175560    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/022aa623-e479-4722-9b16-969312890c87-kube-api-access-4zwr5" (OuterVolumeSpecName: "kube-api-access-4zwr5") pod "022aa623-e479-4722-9b16-969312890c87" (UID: "022aa623-e479-4722-9b16-969312890c87"). InnerVolumeSpecName "kube-api-access-4zwr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:30:26.177626 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:26.177603    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/022aa623-e479-4722-9b16-969312890c87-util" (OuterVolumeSpecName: "util") pod "022aa623-e479-4722-9b16-969312890c87" (UID: "022aa623-e479-4722-9b16-969312890c87"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:30:26.274865 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:26.274810    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zwr5\" (UniqueName: \"kubernetes.io/projected/022aa623-e479-4722-9b16-969312890c87-kube-api-access-4zwr5\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:30:26.274865 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:26.274834    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/022aa623-e479-4722-9b16-969312890c87-util\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:30:26.274865 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:26.274844    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/022aa623-e479-4722-9b16-969312890c87-bundle\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:30:26.983041 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:26.983010    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq"
Sep 05 06:30:26.983183 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:26.983012    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540wqlcq" event={"ID":"022aa623-e479-4722-9b16-969312890c87","Type":"ContainerDied","Data":"e4d8326f1065ac1fecfde56ab5d8a1f40694acab7d853aafaeb85c650bdada37"}
Sep 05 06:30:26.983183 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:26.983126    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4d8326f1065ac1fecfde56ab5d8a1f40694acab7d853aafaeb85c650bdada37"
Sep 05 06:30:29.787870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.787837    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5cn4s"]
Sep 05 06:30:29.788268 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.788084    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="022aa623-e479-4722-9b16-969312890c87" containerName="pull"
Sep 05 06:30:29.788268 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.788095    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="022aa623-e479-4722-9b16-969312890c87" containerName="pull"
Sep 05 06:30:29.788268 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.788108    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="022aa623-e479-4722-9b16-969312890c87" containerName="util"
Sep 05 06:30:29.788268 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.788114    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="022aa623-e479-4722-9b16-969312890c87" containerName="util"
Sep 05 06:30:29.788268 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.788122    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="022aa623-e479-4722-9b16-969312890c87" containerName="extract"
Sep 05 06:30:29.788268 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.788127    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="022aa623-e479-4722-9b16-969312890c87" containerName="extract"
Sep 05 06:30:29.788268 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.788182    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="022aa623-e479-4722-9b16-969312890c87" containerName="extract"
Sep 05 06:30:29.791282 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.791265    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5cn4s"
Sep 05 06:30:29.794043 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.794020    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 05 06:30:29.794184 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.794088    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 05 06:30:29.794237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.794189    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-8g8jj\""
Sep 05 06:30:29.803623 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.803601    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5cn4s"]
Sep 05 06:30:29.895842 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.895816    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xqc8\" (UniqueName: \"kubernetes.io/projected/f9e49511-c9ef-4b4e-ad41-c5e506a8632d-kube-api-access-9xqc8\") pod \"cert-manager-operator-controller-manager-55c444bf86-5cn4s\" (UID: \"f9e49511-c9ef-4b4e-ad41-c5e506a8632d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5cn4s"
Sep 05 06:30:29.895966 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.895846    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f9e49511-c9ef-4b4e-ad41-c5e506a8632d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5cn4s\" (UID: \"f9e49511-c9ef-4b4e-ad41-c5e506a8632d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5cn4s"
Sep 05 06:30:29.996654 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.996629    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f9e49511-c9ef-4b4e-ad41-c5e506a8632d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5cn4s\" (UID: \"f9e49511-c9ef-4b4e-ad41-c5e506a8632d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5cn4s"
Sep 05 06:30:29.996757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.996685    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xqc8\" (UniqueName: \"kubernetes.io/projected/f9e49511-c9ef-4b4e-ad41-c5e506a8632d-kube-api-access-9xqc8\") pod \"cert-manager-operator-controller-manager-55c444bf86-5cn4s\" (UID: \"f9e49511-c9ef-4b4e-ad41-c5e506a8632d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5cn4s"
Sep 05 06:30:29.997034 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:29.997016    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f9e49511-c9ef-4b4e-ad41-c5e506a8632d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5cn4s\" (UID: \"f9e49511-c9ef-4b4e-ad41-c5e506a8632d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5cn4s"
Sep 05 06:30:30.006518 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:30.006500    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xqc8\" (UniqueName: \"kubernetes.io/projected/f9e49511-c9ef-4b4e-ad41-c5e506a8632d-kube-api-access-9xqc8\") pod \"cert-manager-operator-controller-manager-55c444bf86-5cn4s\" (UID: \"f9e49511-c9ef-4b4e-ad41-c5e506a8632d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5cn4s"
Sep 05 06:30:30.100092 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:30.100041    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5cn4s"
Sep 05 06:30:30.372499 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:30.372420    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5cn4s"]
Sep 05 06:30:30.376376 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:30:30.376350    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9e49511_c9ef_4b4e_ad41_c5e506a8632d.slice/crio-b6060a4aea687f3797bcb3b3f2f04a86824f2953aadcf14fe47e3d52935bb995 WatchSource:0}: Error finding container b6060a4aea687f3797bcb3b3f2f04a86824f2953aadcf14fe47e3d52935bb995: Status 404 returned error can't find the container with id b6060a4aea687f3797bcb3b3f2f04a86824f2953aadcf14fe47e3d52935bb995
Sep 05 06:30:30.997596 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:30.997564    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5cn4s" event={"ID":"f9e49511-c9ef-4b4e-ad41-c5e506a8632d","Type":"ContainerStarted","Data":"b6060a4aea687f3797bcb3b3f2f04a86824f2953aadcf14fe47e3d52935bb995"}
Sep 05 06:30:33.007354 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:33.007320    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5cn4s" event={"ID":"f9e49511-c9ef-4b4e-ad41-c5e506a8632d","Type":"ContainerStarted","Data":"2c850ce3a049aebb11c67f7d783bfe209152c7faa24f705560950dddcd66eb7d"}
Sep 05 06:30:33.033464 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:33.033402    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5cn4s" podStartSLOduration=2.283467408 podStartE2EDuration="4.033382982s" totalImagesPullingTime="1.749915574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:30:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:30:33.032052816 +0000 UTC m=+249.622611767" watchObservedRunningTime="2026-09-05 06:30:33.033382982 +0000 UTC m=+249.623941955"
Sep 05 06:30:34.154476 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:34.154441    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh"]
Sep 05 06:30:34.156465 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:34.156449    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh"
Sep 05 06:30:34.159316 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:34.159287    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g76sk\""
Sep 05 06:30:34.159316 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:34.159312    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 05 06:30:34.159498 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:34.159311    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 05 06:30:34.168264 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:34.168234    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh"]
Sep 05 06:30:34.323497 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:34.323459    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d419fca-d019-428b-95d7-3e303277e513-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh\" (UID: \"6d419fca-d019-428b-95d7-3e303277e513\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh"
Sep 05 06:30:34.323744 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:34.323507    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26c74\" (UniqueName: \"kubernetes.io/projected/6d419fca-d019-428b-95d7-3e303277e513-kube-api-access-26c74\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh\" (UID: \"6d419fca-d019-428b-95d7-3e303277e513\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh"
Sep 05 06:30:34.323744 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:34.323575    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d419fca-d019-428b-95d7-3e303277e513-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh\" (UID: \"6d419fca-d019-428b-95d7-3e303277e513\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh"
Sep 05 06:30:34.424995 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:34.424919    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d419fca-d019-428b-95d7-3e303277e513-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh\" (UID: \"6d419fca-d019-428b-95d7-3e303277e513\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh"
Sep 05 06:30:34.424995 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:34.424961    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26c74\" (UniqueName: \"kubernetes.io/projected/6d419fca-d019-428b-95d7-3e303277e513-kube-api-access-26c74\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh\" (UID: \"6d419fca-d019-428b-95d7-3e303277e513\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh"
Sep 05 06:30:34.424995 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:34.424992    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d419fca-d019-428b-95d7-3e303277e513-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh\" (UID: \"6d419fca-d019-428b-95d7-3e303277e513\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh"
Sep 05 06:30:34.425259 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:34.425237    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d419fca-d019-428b-95d7-3e303277e513-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh\" (UID: \"6d419fca-d019-428b-95d7-3e303277e513\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh"
Sep 05 06:30:34.425318 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:34.425267    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d419fca-d019-428b-95d7-3e303277e513-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh\" (UID: \"6d419fca-d019-428b-95d7-3e303277e513\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh"
Sep 05 06:30:34.434995 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:34.434966    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26c74\" (UniqueName: \"kubernetes.io/projected/6d419fca-d019-428b-95d7-3e303277e513-kube-api-access-26c74\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh\" (UID: \"6d419fca-d019-428b-95d7-3e303277e513\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh"
Sep 05 06:30:34.470096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:34.470073    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh"
Sep 05 06:30:34.739403 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:34.739379    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh"]
Sep 05 06:30:34.741758 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:30:34.741724    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d419fca_d019_428b_95d7_3e303277e513.slice/crio-9fdcea77f1a20d205f40bef37a8adc750c2ca03fe4013b3453506e420171a99a WatchSource:0}: Error finding container 9fdcea77f1a20d205f40bef37a8adc750c2ca03fe4013b3453506e420171a99a: Status 404 returned error can't find the container with id 9fdcea77f1a20d205f40bef37a8adc750c2ca03fe4013b3453506e420171a99a
Sep 05 06:30:35.016408 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:35.016340    2559 generic.go:356] "Generic (PLEG): container finished" podID="6d419fca-d019-428b-95d7-3e303277e513" containerID="a643295a9249be36ad2e1b33e2528fea1735eb51b2903fa7d53941683027c668" exitCode=0
Sep 05 06:30:35.016534 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:35.016406    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh" event={"ID":"6d419fca-d019-428b-95d7-3e303277e513","Type":"ContainerDied","Data":"a643295a9249be36ad2e1b33e2528fea1735eb51b2903fa7d53941683027c668"}
Sep 05 06:30:35.016534 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:35.016440    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh" event={"ID":"6d419fca-d019-428b-95d7-3e303277e513","Type":"ContainerStarted","Data":"9fdcea77f1a20d205f40bef37a8adc750c2ca03fe4013b3453506e420171a99a"}
Sep 05 06:30:36.671640 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:36.671600    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9t7wp"]
Sep 05 06:30:36.678475 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:36.678444    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9t7wp"
Sep 05 06:30:36.681588 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:36.681544    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 05 06:30:36.681588 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:36.681568    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 05 06:30:36.682268 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:36.682250    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-r79np\""
Sep 05 06:30:36.691072 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:36.691050    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9t7wp"]
Sep 05 06:30:36.844634 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:36.844584    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvxq9\" (UniqueName: \"kubernetes.io/projected/994a4f23-514f-4aad-a77a-165b75ceb4f5-kube-api-access-nvxq9\") pod \"cert-manager-cainjector-559cdd8c87-9t7wp\" (UID: \"994a4f23-514f-4aad-a77a-165b75ceb4f5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9t7wp"
Sep 05 06:30:36.844785 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:36.844688    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/994a4f23-514f-4aad-a77a-165b75ceb4f5-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9t7wp\" (UID: \"994a4f23-514f-4aad-a77a-165b75ceb4f5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9t7wp"
Sep 05 06:30:36.945915 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:36.945872    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvxq9\" (UniqueName: \"kubernetes.io/projected/994a4f23-514f-4aad-a77a-165b75ceb4f5-kube-api-access-nvxq9\") pod \"cert-manager-cainjector-559cdd8c87-9t7wp\" (UID: \"994a4f23-514f-4aad-a77a-165b75ceb4f5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9t7wp"
Sep 05 06:30:36.946008 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:36.945935    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/994a4f23-514f-4aad-a77a-165b75ceb4f5-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9t7wp\" (UID: \"994a4f23-514f-4aad-a77a-165b75ceb4f5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9t7wp"
Sep 05 06:30:36.960100 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:36.960071    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/994a4f23-514f-4aad-a77a-165b75ceb4f5-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-9t7wp\" (UID: \"994a4f23-514f-4aad-a77a-165b75ceb4f5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9t7wp"
Sep 05 06:30:36.960277 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:36.960257    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvxq9\" (UniqueName: \"kubernetes.io/projected/994a4f23-514f-4aad-a77a-165b75ceb4f5-kube-api-access-nvxq9\") pod \"cert-manager-cainjector-559cdd8c87-9t7wp\" (UID: \"994a4f23-514f-4aad-a77a-165b75ceb4f5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-9t7wp"
Sep 05 06:30:36.989338 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:36.989315    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9t7wp"
Sep 05 06:30:37.026400 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:37.026346    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh" event={"ID":"6d419fca-d019-428b-95d7-3e303277e513","Type":"ContainerStarted","Data":"d8016855ff5d25b1aa9f9b8fda9603d673ca6d96e4a87c49a6bb21572cc54be1"}
Sep 05 06:30:37.286083 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:37.285938    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-9t7wp"]
Sep 05 06:30:37.288554 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:30:37.288524    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod994a4f23_514f_4aad_a77a_165b75ceb4f5.slice/crio-addd3c208fb51be17696762219c76aec2fa8e8b12378771b90f47d4d8612b759 WatchSource:0}: Error finding container addd3c208fb51be17696762219c76aec2fa8e8b12378771b90f47d4d8612b759: Status 404 returned error can't find the container with id addd3c208fb51be17696762219c76aec2fa8e8b12378771b90f47d4d8612b759
Sep 05 06:30:38.032287 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:38.032256    2559 generic.go:356] "Generic (PLEG): container finished" podID="6d419fca-d019-428b-95d7-3e303277e513" containerID="d8016855ff5d25b1aa9f9b8fda9603d673ca6d96e4a87c49a6bb21572cc54be1" exitCode=0
Sep 05 06:30:38.032648 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:38.032335    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh" event={"ID":"6d419fca-d019-428b-95d7-3e303277e513","Type":"ContainerDied","Data":"d8016855ff5d25b1aa9f9b8fda9603d673ca6d96e4a87c49a6bb21572cc54be1"}
Sep 05 06:30:38.033582 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:38.033559    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9t7wp" event={"ID":"994a4f23-514f-4aad-a77a-165b75ceb4f5","Type":"ContainerStarted","Data":"addd3c208fb51be17696762219c76aec2fa8e8b12378771b90f47d4d8612b759"}
Sep 05 06:30:39.040530 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:39.040493    2559 generic.go:356] "Generic (PLEG): container finished" podID="6d419fca-d019-428b-95d7-3e303277e513" containerID="0b526f99f022aab336cefa866424dbab66009dc24b6ff10ffcfdd5fcaeb52fe9" exitCode=0
Sep 05 06:30:39.040960 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:39.040583    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh" event={"ID":"6d419fca-d019-428b-95d7-3e303277e513","Type":"ContainerDied","Data":"0b526f99f022aab336cefa866424dbab66009dc24b6ff10ffcfdd5fcaeb52fe9"}
Sep 05 06:30:40.167975 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:40.167945    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh"
Sep 05 06:30:40.273876 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:40.273843    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d419fca-d019-428b-95d7-3e303277e513-bundle\") pod \"6d419fca-d019-428b-95d7-3e303277e513\" (UID: \"6d419fca-d019-428b-95d7-3e303277e513\") "
Sep 05 06:30:40.273876 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:40.273896    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26c74\" (UniqueName: \"kubernetes.io/projected/6d419fca-d019-428b-95d7-3e303277e513-kube-api-access-26c74\") pod \"6d419fca-d019-428b-95d7-3e303277e513\" (UID: \"6d419fca-d019-428b-95d7-3e303277e513\") "
Sep 05 06:30:40.274150 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:40.273925    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d419fca-d019-428b-95d7-3e303277e513-util\") pod \"6d419fca-d019-428b-95d7-3e303277e513\" (UID: \"6d419fca-d019-428b-95d7-3e303277e513\") "
Sep 05 06:30:40.274228 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:40.274201    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d419fca-d019-428b-95d7-3e303277e513-bundle" (OuterVolumeSpecName: "bundle") pod "6d419fca-d019-428b-95d7-3e303277e513" (UID: "6d419fca-d019-428b-95d7-3e303277e513"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:30:40.276096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:40.276070    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d419fca-d019-428b-95d7-3e303277e513-kube-api-access-26c74" (OuterVolumeSpecName: "kube-api-access-26c74") pod "6d419fca-d019-428b-95d7-3e303277e513" (UID: "6d419fca-d019-428b-95d7-3e303277e513"). InnerVolumeSpecName "kube-api-access-26c74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:30:40.279355 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:40.279325    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d419fca-d019-428b-95d7-3e303277e513-util" (OuterVolumeSpecName: "util") pod "6d419fca-d019-428b-95d7-3e303277e513" (UID: "6d419fca-d019-428b-95d7-3e303277e513"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:30:40.374652 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:40.374573    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6d419fca-d019-428b-95d7-3e303277e513-util\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:30:40.374652 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:40.374599    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6d419fca-d019-428b-95d7-3e303277e513-bundle\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:30:40.374652 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:40.374610    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26c74\" (UniqueName: \"kubernetes.io/projected/6d419fca-d019-428b-95d7-3e303277e513-kube-api-access-26c74\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:30:41.049625 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:41.049585    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh"
Sep 05 06:30:41.049800 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:41.049594    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjx9bh" event={"ID":"6d419fca-d019-428b-95d7-3e303277e513","Type":"ContainerDied","Data":"9fdcea77f1a20d205f40bef37a8adc750c2ca03fe4013b3453506e420171a99a"}
Sep 05 06:30:41.049800 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:41.049713    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9fdcea77f1a20d205f40bef37a8adc750c2ca03fe4013b3453506e420171a99a"
Sep 05 06:30:41.051270 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:41.051241    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9t7wp" event={"ID":"994a4f23-514f-4aad-a77a-165b75ceb4f5","Type":"ContainerStarted","Data":"4f04900072ad6d77c5fbd18aefc2f1d99732cd0021bb1ebdfde3e1d9d2dadaec"}
Sep 05 06:30:41.091704 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:41.091668    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-9t7wp" podStartSLOduration=2.3209774100000002 podStartE2EDuration="5.091654914s" totalImagesPullingTime="2.770677504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:30:41.090418815 +0000 UTC m=+257.680977768" watchObservedRunningTime="2026-09-05 06:30:41.091654914 +0000 UTC m=+257.682213866"
Sep 05 06:30:53.297084 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.296992    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg"]
Sep 05 06:30:53.297682 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.297346    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d419fca-d019-428b-95d7-3e303277e513" containerName="pull"
Sep 05 06:30:53.297682 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.297369    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d419fca-d019-428b-95d7-3e303277e513" containerName="pull"
Sep 05 06:30:53.297682 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.297383    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d419fca-d019-428b-95d7-3e303277e513" containerName="extract"
Sep 05 06:30:53.297682 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.297392    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d419fca-d019-428b-95d7-3e303277e513" containerName="extract"
Sep 05 06:30:53.297682 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.297423    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d419fca-d019-428b-95d7-3e303277e513" containerName="util"
Sep 05 06:30:53.297682 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.297432    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d419fca-d019-428b-95d7-3e303277e513" containerName="util"
Sep 05 06:30:53.297682 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.297499    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d419fca-d019-428b-95d7-3e303277e513" containerName="extract"
Sep 05 06:30:53.300648 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.300624    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg"
Sep 05 06:30:53.303894 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.303861    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 05 06:30:53.304364 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.304347    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g76sk\""
Sep 05 06:30:53.304981 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.304965    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 05 06:30:53.311847 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.311828    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg"]
Sep 05 06:30:53.465773 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.465745    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjcrr\" (UniqueName: \"kubernetes.io/projected/d89267ba-d0bd-4b18-a456-76c967110547-kube-api-access-tjcrr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg\" (UID: \"d89267ba-d0bd-4b18-a456-76c967110547\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg"
Sep 05 06:30:53.465914 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.465794    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d89267ba-d0bd-4b18-a456-76c967110547-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg\" (UID: \"d89267ba-d0bd-4b18-a456-76c967110547\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg"
Sep 05 06:30:53.465914 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.465819    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d89267ba-d0bd-4b18-a456-76c967110547-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg\" (UID: \"d89267ba-d0bd-4b18-a456-76c967110547\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg"
Sep 05 06:30:53.566683 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.566617    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjcrr\" (UniqueName: \"kubernetes.io/projected/d89267ba-d0bd-4b18-a456-76c967110547-kube-api-access-tjcrr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg\" (UID: \"d89267ba-d0bd-4b18-a456-76c967110547\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg"
Sep 05 06:30:53.566683 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.566664    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d89267ba-d0bd-4b18-a456-76c967110547-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg\" (UID: \"d89267ba-d0bd-4b18-a456-76c967110547\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg"
Sep 05 06:30:53.566821 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.566690    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d89267ba-d0bd-4b18-a456-76c967110547-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg\" (UID: \"d89267ba-d0bd-4b18-a456-76c967110547\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg"
Sep 05 06:30:53.567016 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.567001    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d89267ba-d0bd-4b18-a456-76c967110547-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg\" (UID: \"d89267ba-d0bd-4b18-a456-76c967110547\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg"
Sep 05 06:30:53.567098 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.567078    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d89267ba-d0bd-4b18-a456-76c967110547-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg\" (UID: \"d89267ba-d0bd-4b18-a456-76c967110547\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg"
Sep 05 06:30:53.585247 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.585224    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjcrr\" (UniqueName: \"kubernetes.io/projected/d89267ba-d0bd-4b18-a456-76c967110547-kube-api-access-tjcrr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg\" (UID: \"d89267ba-d0bd-4b18-a456-76c967110547\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg"
Sep 05 06:30:53.609998 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.609978    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg"
Sep 05 06:30:53.884452 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:53.884344    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg"]
Sep 05 06:30:53.886929 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:30:53.886899    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd89267ba_d0bd_4b18_a456_76c967110547.slice/crio-37500e763b964ab6ac11345d5a3287459313335bcdb1a4565d2b0acfc00f298e WatchSource:0}: Error finding container 37500e763b964ab6ac11345d5a3287459313335bcdb1a4565d2b0acfc00f298e: Status 404 returned error can't find the container with id 37500e763b964ab6ac11345d5a3287459313335bcdb1a4565d2b0acfc00f298e
Sep 05 06:30:54.102131 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:54.102098    2559 generic.go:356] "Generic (PLEG): container finished" podID="d89267ba-d0bd-4b18-a456-76c967110547" containerID="b41cf8d4927bb572d03e05436345cfb99a206f9e1b63c56a02bd2e7b3a05dc81" exitCode=0
Sep 05 06:30:54.102260 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:54.102154    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg" event={"ID":"d89267ba-d0bd-4b18-a456-76c967110547","Type":"ContainerDied","Data":"b41cf8d4927bb572d03e05436345cfb99a206f9e1b63c56a02bd2e7b3a05dc81"}
Sep 05 06:30:54.102260 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:54.102189    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg" event={"ID":"d89267ba-d0bd-4b18-a456-76c967110547","Type":"ContainerStarted","Data":"37500e763b964ab6ac11345d5a3287459313335bcdb1a4565d2b0acfc00f298e"}
Sep 05 06:30:55.109217 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:55.109188    2559 generic.go:356] "Generic (PLEG): container finished" podID="d89267ba-d0bd-4b18-a456-76c967110547" containerID="6716c688b0f71ef3dd40e3ef69a9d0d8c02eb928e372715f8778f1aa4f8e7a83" exitCode=0
Sep 05 06:30:55.109549 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:55.109264    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg" event={"ID":"d89267ba-d0bd-4b18-a456-76c967110547","Type":"ContainerDied","Data":"6716c688b0f71ef3dd40e3ef69a9d0d8c02eb928e372715f8778f1aa4f8e7a83"}
Sep 05 06:30:56.115392 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:56.115355    2559 generic.go:356] "Generic (PLEG): container finished" podID="d89267ba-d0bd-4b18-a456-76c967110547" containerID="f3d1cf13076514e111d1e1c1ad59dc437350aefc9280f5828c7eb27d36a7fd90" exitCode=0
Sep 05 06:30:56.115777 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:56.115445    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg" event={"ID":"d89267ba-d0bd-4b18-a456-76c967110547","Type":"ContainerDied","Data":"f3d1cf13076514e111d1e1c1ad59dc437350aefc9280f5828c7eb27d36a7fd90"}
Sep 05 06:30:57.261284 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:57.261258    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg"
Sep 05 06:30:57.293897 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:57.293861    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d89267ba-d0bd-4b18-a456-76c967110547-bundle\") pod \"d89267ba-d0bd-4b18-a456-76c967110547\" (UID: \"d89267ba-d0bd-4b18-a456-76c967110547\") "
Sep 05 06:30:57.294018 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:57.293926    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjcrr\" (UniqueName: \"kubernetes.io/projected/d89267ba-d0bd-4b18-a456-76c967110547-kube-api-access-tjcrr\") pod \"d89267ba-d0bd-4b18-a456-76c967110547\" (UID: \"d89267ba-d0bd-4b18-a456-76c967110547\") "
Sep 05 06:30:57.294018 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:57.293976    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d89267ba-d0bd-4b18-a456-76c967110547-util\") pod \"d89267ba-d0bd-4b18-a456-76c967110547\" (UID: \"d89267ba-d0bd-4b18-a456-76c967110547\") "
Sep 05 06:30:57.294516 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:57.294489    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d89267ba-d0bd-4b18-a456-76c967110547-bundle" (OuterVolumeSpecName: "bundle") pod "d89267ba-d0bd-4b18-a456-76c967110547" (UID: "d89267ba-d0bd-4b18-a456-76c967110547"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:30:57.295995 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:57.295966    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d89267ba-d0bd-4b18-a456-76c967110547-kube-api-access-tjcrr" (OuterVolumeSpecName: "kube-api-access-tjcrr") pod "d89267ba-d0bd-4b18-a456-76c967110547" (UID: "d89267ba-d0bd-4b18-a456-76c967110547"). InnerVolumeSpecName "kube-api-access-tjcrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:30:57.301765 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:57.301741    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d89267ba-d0bd-4b18-a456-76c967110547-util" (OuterVolumeSpecName: "util") pod "d89267ba-d0bd-4b18-a456-76c967110547" (UID: "d89267ba-d0bd-4b18-a456-76c967110547"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:30:57.394815 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:57.394768    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d89267ba-d0bd-4b18-a456-76c967110547-bundle\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:30:57.394815 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:57.394791    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjcrr\" (UniqueName: \"kubernetes.io/projected/d89267ba-d0bd-4b18-a456-76c967110547-kube-api-access-tjcrr\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:30:57.394815 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:57.394802    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d89267ba-d0bd-4b18-a456-76c967110547-util\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:30:58.126757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:58.126725    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg" event={"ID":"d89267ba-d0bd-4b18-a456-76c967110547","Type":"ContainerDied","Data":"37500e763b964ab6ac11345d5a3287459313335bcdb1a4565d2b0acfc00f298e"}
Sep 05 06:30:58.126944 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:58.126783    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37500e763b964ab6ac11345d5a3287459313335bcdb1a4565d2b0acfc00f298e"
Sep 05 06:30:58.126944 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:30:58.126735    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpnhwg"
Sep 05 06:31:06.902710 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:06.902671    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn"]
Sep 05 06:31:06.903288 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:06.903059    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d89267ba-d0bd-4b18-a456-76c967110547" containerName="util"
Sep 05 06:31:06.903288 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:06.903078    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d89267ba-d0bd-4b18-a456-76c967110547" containerName="util"
Sep 05 06:31:06.903288 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:06.903092    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d89267ba-d0bd-4b18-a456-76c967110547" containerName="pull"
Sep 05 06:31:06.903288 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:06.903101    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d89267ba-d0bd-4b18-a456-76c967110547" containerName="pull"
Sep 05 06:31:06.903288 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:06.903127    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d89267ba-d0bd-4b18-a456-76c967110547" containerName="extract"
Sep 05 06:31:06.903288 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:06.903137    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d89267ba-d0bd-4b18-a456-76c967110547" containerName="extract"
Sep 05 06:31:06.903288 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:06.903200    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d89267ba-d0bd-4b18-a456-76c967110547" containerName="extract"
Sep 05 06:31:06.909124 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:06.909101    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn"
Sep 05 06:31:06.911625 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:06.911600    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 05 06:31:06.911727 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:06.911658    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g76sk\""
Sep 05 06:31:06.911727 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:06.911701    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 05 06:31:06.918506 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:06.918482    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn"]
Sep 05 06:31:06.955101 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:06.955075    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eaba80ec-bff8-4efa-b572-e28328014656-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn\" (UID: \"eaba80ec-bff8-4efa-b572-e28328014656\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn"
Sep 05 06:31:06.955201 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:06.955119    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eaba80ec-bff8-4efa-b572-e28328014656-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn\" (UID: \"eaba80ec-bff8-4efa-b572-e28328014656\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn"
Sep 05 06:31:06.955201 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:06.955138    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vztt\" (UniqueName: \"kubernetes.io/projected/eaba80ec-bff8-4efa-b572-e28328014656-kube-api-access-2vztt\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn\" (UID: \"eaba80ec-bff8-4efa-b572-e28328014656\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn"
Sep 05 06:31:07.055923 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:07.055896    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eaba80ec-bff8-4efa-b572-e28328014656-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn\" (UID: \"eaba80ec-bff8-4efa-b572-e28328014656\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn"
Sep 05 06:31:07.056054 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:07.055931    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eaba80ec-bff8-4efa-b572-e28328014656-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn\" (UID: \"eaba80ec-bff8-4efa-b572-e28328014656\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn"
Sep 05 06:31:07.056054 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:07.055953    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vztt\" (UniqueName: \"kubernetes.io/projected/eaba80ec-bff8-4efa-b572-e28328014656-kube-api-access-2vztt\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn\" (UID: \"eaba80ec-bff8-4efa-b572-e28328014656\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn"
Sep 05 06:31:07.056276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:07.056256    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eaba80ec-bff8-4efa-b572-e28328014656-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn\" (UID: \"eaba80ec-bff8-4efa-b572-e28328014656\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn"
Sep 05 06:31:07.056345 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:07.056316    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eaba80ec-bff8-4efa-b572-e28328014656-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn\" (UID: \"eaba80ec-bff8-4efa-b572-e28328014656\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn"
Sep 05 06:31:07.065411 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:07.065389    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vztt\" (UniqueName: \"kubernetes.io/projected/eaba80ec-bff8-4efa-b572-e28328014656-kube-api-access-2vztt\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn\" (UID: \"eaba80ec-bff8-4efa-b572-e28328014656\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn"
Sep 05 06:31:07.217895 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:07.217862    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn"
Sep 05 06:31:07.490039 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:07.490011    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn"]
Sep 05 06:31:07.491543 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:31:07.491518    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaba80ec_bff8_4efa_b572_e28328014656.slice/crio-3a08c1351ba6a1d3756660dd2aa7f40141973830dd966d0c2258dd8a1c9d214b WatchSource:0}: Error finding container 3a08c1351ba6a1d3756660dd2aa7f40141973830dd966d0c2258dd8a1c9d214b: Status 404 returned error can't find the container with id 3a08c1351ba6a1d3756660dd2aa7f40141973830dd966d0c2258dd8a1c9d214b
Sep 05 06:31:08.168827 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.168793    2559 generic.go:356] "Generic (PLEG): container finished" podID="eaba80ec-bff8-4efa-b572-e28328014656" containerID="489fd63eadfc6f1863373852427b6d9fd84126f379de3c04c05fc939f2f39ddf" exitCode=0
Sep 05 06:31:08.169248 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.168832    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn" event={"ID":"eaba80ec-bff8-4efa-b572-e28328014656","Type":"ContainerDied","Data":"489fd63eadfc6f1863373852427b6d9fd84126f379de3c04c05fc939f2f39ddf"}
Sep 05 06:31:08.169248 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.168937    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn" event={"ID":"eaba80ec-bff8-4efa-b572-e28328014656","Type":"ContainerStarted","Data":"3a08c1351ba6a1d3756660dd2aa7f40141973830dd966d0c2258dd8a1c9d214b"}
Sep 05 06:31:08.284227 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.284199    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"]
Sep 05 06:31:08.287744 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.287726    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:08.291239 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.291216    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 05 06:31:08.291239 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.291230    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 05 06:31:08.291377 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.291299    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 05 06:31:08.291425 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.291403    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2f8p8\""
Sep 05 06:31:08.291728 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.291710    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 05 06:31:08.303056 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.303032    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"]
Sep 05 06:31:08.365434 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.365385    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4b7d6079-e03b-4bd4-99a9-982adf37a1da-webhook-cert\") pod \"opendatahub-operator-controller-manager-555dc54ddd-zgvqq\" (UID: \"4b7d6079-e03b-4bd4-99a9-982adf37a1da\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:08.365434 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.365425    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4b7d6079-e03b-4bd4-99a9-982adf37a1da-manifests\") pod \"opendatahub-operator-controller-manager-555dc54ddd-zgvqq\" (UID: \"4b7d6079-e03b-4bd4-99a9-982adf37a1da\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:08.365553 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.365483    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4b7d6079-e03b-4bd4-99a9-982adf37a1da-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555dc54ddd-zgvqq\" (UID: \"4b7d6079-e03b-4bd4-99a9-982adf37a1da\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:08.365553 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.365508    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncwjt\" (UniqueName: \"kubernetes.io/projected/4b7d6079-e03b-4bd4-99a9-982adf37a1da-kube-api-access-ncwjt\") pod \"opendatahub-operator-controller-manager-555dc54ddd-zgvqq\" (UID: \"4b7d6079-e03b-4bd4-99a9-982adf37a1da\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:08.365553 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.365527    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b7d6079-e03b-4bd4-99a9-982adf37a1da-tmp\") pod \"opendatahub-operator-controller-manager-555dc54ddd-zgvqq\" (UID: \"4b7d6079-e03b-4bd4-99a9-982adf37a1da\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:08.465773 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.465751    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4b7d6079-e03b-4bd4-99a9-982adf37a1da-webhook-cert\") pod \"opendatahub-operator-controller-manager-555dc54ddd-zgvqq\" (UID: \"4b7d6079-e03b-4bd4-99a9-982adf37a1da\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:08.465854 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.465788    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4b7d6079-e03b-4bd4-99a9-982adf37a1da-manifests\") pod \"opendatahub-operator-controller-manager-555dc54ddd-zgvqq\" (UID: \"4b7d6079-e03b-4bd4-99a9-982adf37a1da\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:08.465854 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.465823    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4b7d6079-e03b-4bd4-99a9-982adf37a1da-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555dc54ddd-zgvqq\" (UID: \"4b7d6079-e03b-4bd4-99a9-982adf37a1da\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:08.466067 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.466047    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncwjt\" (UniqueName: \"kubernetes.io/projected/4b7d6079-e03b-4bd4-99a9-982adf37a1da-kube-api-access-ncwjt\") pod \"opendatahub-operator-controller-manager-555dc54ddd-zgvqq\" (UID: \"4b7d6079-e03b-4bd4-99a9-982adf37a1da\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:08.466105 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.466082    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b7d6079-e03b-4bd4-99a9-982adf37a1da-tmp\") pod \"opendatahub-operator-controller-manager-555dc54ddd-zgvqq\" (UID: \"4b7d6079-e03b-4bd4-99a9-982adf37a1da\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:08.466207 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.466188    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4b7d6079-e03b-4bd4-99a9-982adf37a1da-manifests\") pod \"opendatahub-operator-controller-manager-555dc54ddd-zgvqq\" (UID: \"4b7d6079-e03b-4bd4-99a9-982adf37a1da\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:08.468129 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.468104    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b7d6079-e03b-4bd4-99a9-982adf37a1da-tmp\") pod \"opendatahub-operator-controller-manager-555dc54ddd-zgvqq\" (UID: \"4b7d6079-e03b-4bd4-99a9-982adf37a1da\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:08.468224 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.468206    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4b7d6079-e03b-4bd4-99a9-982adf37a1da-apiservice-cert\") pod \"opendatahub-operator-controller-manager-555dc54ddd-zgvqq\" (UID: \"4b7d6079-e03b-4bd4-99a9-982adf37a1da\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:08.468321 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.468305    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4b7d6079-e03b-4bd4-99a9-982adf37a1da-webhook-cert\") pod \"opendatahub-operator-controller-manager-555dc54ddd-zgvqq\" (UID: \"4b7d6079-e03b-4bd4-99a9-982adf37a1da\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:08.479930 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.479905    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncwjt\" (UniqueName: \"kubernetes.io/projected/4b7d6079-e03b-4bd4-99a9-982adf37a1da-kube-api-access-ncwjt\") pod \"opendatahub-operator-controller-manager-555dc54ddd-zgvqq\" (UID: \"4b7d6079-e03b-4bd4-99a9-982adf37a1da\") " pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:08.596551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.596530    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:08.878984 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:08.878962    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"]
Sep 05 06:31:08.927646 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:31:08.927585    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b7d6079_e03b_4bd4_99a9_982adf37a1da.slice/crio-50e1dfa15f1aa065a7d103df8e62307d29092d08097e3de15f0529bc83e13842 WatchSource:0}: Error finding container 50e1dfa15f1aa065a7d103df8e62307d29092d08097e3de15f0529bc83e13842: Status 404 returned error can't find the container with id 50e1dfa15f1aa065a7d103df8e62307d29092d08097e3de15f0529bc83e13842
Sep 05 06:31:09.175313 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:09.175282    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq" event={"ID":"4b7d6079-e03b-4bd4-99a9-982adf37a1da","Type":"ContainerStarted","Data":"50e1dfa15f1aa065a7d103df8e62307d29092d08097e3de15f0529bc83e13842"}
Sep 05 06:31:09.177150 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:09.177125    2559 generic.go:356] "Generic (PLEG): container finished" podID="eaba80ec-bff8-4efa-b572-e28328014656" containerID="c0a1a27393eae665fbc1c7002de60445acde07524305395736ebecad4fa61c85" exitCode=0
Sep 05 06:31:09.177274 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:09.177185    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn" event={"ID":"eaba80ec-bff8-4efa-b572-e28328014656","Type":"ContainerDied","Data":"c0a1a27393eae665fbc1c7002de60445acde07524305395736ebecad4fa61c85"}
Sep 05 06:31:10.185041 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:10.185005    2559 generic.go:356] "Generic (PLEG): container finished" podID="eaba80ec-bff8-4efa-b572-e28328014656" containerID="fd3d12eab3caf9f69efaef327bc6402fc094ece957804ac85106d2a9a3852e68" exitCode=0
Sep 05 06:31:10.185470 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:10.185097    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn" event={"ID":"eaba80ec-bff8-4efa-b572-e28328014656","Type":"ContainerDied","Data":"fd3d12eab3caf9f69efaef327bc6402fc094ece957804ac85106d2a9a3852e68"}
Sep 05 06:31:11.862534 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:11.862511    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn"
Sep 05 06:31:11.894833 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:11.894810    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eaba80ec-bff8-4efa-b572-e28328014656-bundle\") pod \"eaba80ec-bff8-4efa-b572-e28328014656\" (UID: \"eaba80ec-bff8-4efa-b572-e28328014656\") "
Sep 05 06:31:11.894965 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:11.894858    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eaba80ec-bff8-4efa-b572-e28328014656-util\") pod \"eaba80ec-bff8-4efa-b572-e28328014656\" (UID: \"eaba80ec-bff8-4efa-b572-e28328014656\") "
Sep 05 06:31:11.894965 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:11.894913    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vztt\" (UniqueName: \"kubernetes.io/projected/eaba80ec-bff8-4efa-b572-e28328014656-kube-api-access-2vztt\") pod \"eaba80ec-bff8-4efa-b572-e28328014656\" (UID: \"eaba80ec-bff8-4efa-b572-e28328014656\") "
Sep 05 06:31:11.896141 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:11.896006    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaba80ec-bff8-4efa-b572-e28328014656-bundle" (OuterVolumeSpecName: "bundle") pod "eaba80ec-bff8-4efa-b572-e28328014656" (UID: "eaba80ec-bff8-4efa-b572-e28328014656"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:31:11.897216 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:11.897191    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaba80ec-bff8-4efa-b572-e28328014656-kube-api-access-2vztt" (OuterVolumeSpecName: "kube-api-access-2vztt") pod "eaba80ec-bff8-4efa-b572-e28328014656" (UID: "eaba80ec-bff8-4efa-b572-e28328014656"). InnerVolumeSpecName "kube-api-access-2vztt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:31:11.900925 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:11.900899    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eaba80ec-bff8-4efa-b572-e28328014656-util" (OuterVolumeSpecName: "util") pod "eaba80ec-bff8-4efa-b572-e28328014656" (UID: "eaba80ec-bff8-4efa-b572-e28328014656"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:31:11.996077 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:11.996054    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eaba80ec-bff8-4efa-b572-e28328014656-bundle\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:31:11.996077 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:11.996073    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eaba80ec-bff8-4efa-b572-e28328014656-util\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:31:11.996191 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:11.996082    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vztt\" (UniqueName: \"kubernetes.io/projected/eaba80ec-bff8-4efa-b572-e28328014656-kube-api-access-2vztt\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:31:12.194950 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:12.194916    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq" event={"ID":"4b7d6079-e03b-4bd4-99a9-982adf37a1da","Type":"ContainerStarted","Data":"06b5977b0b4bf9578a3c4b70f205bf5f5e55d2663d0cd77366a06eb7450d820f"}
Sep 05 06:31:12.196648 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:12.196631    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn"
Sep 05 06:31:12.196726 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:12.196631    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0z57nn" event={"ID":"eaba80ec-bff8-4efa-b572-e28328014656","Type":"ContainerDied","Data":"3a08c1351ba6a1d3756660dd2aa7f40141973830dd966d0c2258dd8a1c9d214b"}
Sep 05 06:31:12.196779 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:12.196733    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a08c1351ba6a1d3756660dd2aa7f40141973830dd966d0c2258dd8a1c9d214b"
Sep 05 06:31:13.202799 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:13.202763    2559 generic.go:356] "Generic (PLEG): container finished" podID="4b7d6079-e03b-4bd4-99a9-982adf37a1da" containerID="06b5977b0b4bf9578a3c4b70f205bf5f5e55d2663d0cd77366a06eb7450d820f" exitCode=0
Sep 05 06:31:13.203127 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:13.202832    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq" event={"ID":"4b7d6079-e03b-4bd4-99a9-982adf37a1da","Type":"ContainerDied","Data":"06b5977b0b4bf9578a3c4b70f205bf5f5e55d2663d0cd77366a06eb7450d820f"}
Sep 05 06:31:14.211145 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:14.211108    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq" event={"ID":"4b7d6079-e03b-4bd4-99a9-982adf37a1da","Type":"ContainerStarted","Data":"262173a0f13ac3f10b770b5480b93deeb3e5cd67ca6dc75d9e903efaf04b8494"}
Sep 05 06:31:14.211657 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:14.211261    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:14.239495 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:14.239429    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq" podStartSLOduration=1.8840763680000001 podStartE2EDuration="6.239396176s" totalImagesPullingTime="3.273262281s" totalInitContainerRuntime="1.082057527s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:31:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:31:14.23807622 +0000 UTC m=+290.828635174" watchObservedRunningTime="2026-09-05 06:31:14.239396176 +0000 UTC m=+290.829955129"
Sep 05 06:31:23.795809 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:23.795781    2559 kubelet.go:1648] "Image garbage collection succeeded"
Sep 05 06:31:25.217683 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:25.217652    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-555dc54ddd-zgvqq"
Sep 05 06:31:27.860946 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:27.860905    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8"]
Sep 05 06:31:27.861380 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:27.861317    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaba80ec-bff8-4efa-b572-e28328014656" containerName="extract"
Sep 05 06:31:27.861380 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:27.861337    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaba80ec-bff8-4efa-b572-e28328014656" containerName="extract"
Sep 05 06:31:27.861380 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:27.861347    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaba80ec-bff8-4efa-b572-e28328014656" containerName="pull"
Sep 05 06:31:27.861380 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:27.861354    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaba80ec-bff8-4efa-b572-e28328014656" containerName="pull"
Sep 05 06:31:27.861380 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:27.861375    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaba80ec-bff8-4efa-b572-e28328014656" containerName="util"
Sep 05 06:31:27.861380 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:27.861384    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaba80ec-bff8-4efa-b572-e28328014656" containerName="util"
Sep 05 06:31:27.861671 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:27.861469    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaba80ec-bff8-4efa-b572-e28328014656" containerName="extract"
Sep 05 06:31:27.864482 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:27.864461    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8"
Sep 05 06:31:27.868405 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:27.868384    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 05 06:31:27.868742 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:27.868719    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-sxfnq\""
Sep 05 06:31:27.873385 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:27.873365    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 05 06:31:27.878334 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:27.878313    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8"]
Sep 05 06:31:27.903719 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:27.903692    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d783a45f-8564-44f1-ace1-906ef791ee4c-tls-certs\") pod \"kube-auth-proxy-6bd96764bd-6zzt8\" (UID: \"d783a45f-8564-44f1-ace1-906ef791ee4c\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8"
Sep 05 06:31:27.903821 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:27.903735    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xzs4\" (UniqueName: \"kubernetes.io/projected/d783a45f-8564-44f1-ace1-906ef791ee4c-kube-api-access-4xzs4\") pod \"kube-auth-proxy-6bd96764bd-6zzt8\" (UID: \"d783a45f-8564-44f1-ace1-906ef791ee4c\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8"
Sep 05 06:31:27.903821 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:27.903768    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d783a45f-8564-44f1-ace1-906ef791ee4c-tmp\") pod \"kube-auth-proxy-6bd96764bd-6zzt8\" (UID: \"d783a45f-8564-44f1-ace1-906ef791ee4c\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8"
Sep 05 06:31:28.004661 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:28.004636    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d783a45f-8564-44f1-ace1-906ef791ee4c-tls-certs\") pod \"kube-auth-proxy-6bd96764bd-6zzt8\" (UID: \"d783a45f-8564-44f1-ace1-906ef791ee4c\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8"
Sep 05 06:31:28.004774 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:28.004669    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xzs4\" (UniqueName: \"kubernetes.io/projected/d783a45f-8564-44f1-ace1-906ef791ee4c-kube-api-access-4xzs4\") pod \"kube-auth-proxy-6bd96764bd-6zzt8\" (UID: \"d783a45f-8564-44f1-ace1-906ef791ee4c\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8"
Sep 05 06:31:28.004774 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:28.004689    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d783a45f-8564-44f1-ace1-906ef791ee4c-tmp\") pod \"kube-auth-proxy-6bd96764bd-6zzt8\" (UID: \"d783a45f-8564-44f1-ace1-906ef791ee4c\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8"
Sep 05 06:31:28.004843 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:31:28.004805    2559 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 05 06:31:28.004922 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:31:28.004907    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d783a45f-8564-44f1-ace1-906ef791ee4c-tls-certs podName:d783a45f-8564-44f1-ace1-906ef791ee4c nodeName:}" failed. No retries permitted until 2026-09-05 06:31:28.504864564 +0000 UTC m=+305.095423498 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d783a45f-8564-44f1-ace1-906ef791ee4c-tls-certs") pod "kube-auth-proxy-6bd96764bd-6zzt8" (UID: "d783a45f-8564-44f1-ace1-906ef791ee4c") : secret "kube-auth-proxy-tls" not found
Sep 05 06:31:28.006935 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:28.006915    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d783a45f-8564-44f1-ace1-906ef791ee4c-tmp\") pod \"kube-auth-proxy-6bd96764bd-6zzt8\" (UID: \"d783a45f-8564-44f1-ace1-906ef791ee4c\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8"
Sep 05 06:31:28.020307 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:28.020286    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xzs4\" (UniqueName: \"kubernetes.io/projected/d783a45f-8564-44f1-ace1-906ef791ee4c-kube-api-access-4xzs4\") pod \"kube-auth-proxy-6bd96764bd-6zzt8\" (UID: \"d783a45f-8564-44f1-ace1-906ef791ee4c\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8"
Sep 05 06:31:28.508170 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:28.508129    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d783a45f-8564-44f1-ace1-906ef791ee4c-tls-certs\") pod \"kube-auth-proxy-6bd96764bd-6zzt8\" (UID: \"d783a45f-8564-44f1-ace1-906ef791ee4c\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8"
Sep 05 06:31:28.510720 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:28.510692    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d783a45f-8564-44f1-ace1-906ef791ee4c-tls-certs\") pod \"kube-auth-proxy-6bd96764bd-6zzt8\" (UID: \"d783a45f-8564-44f1-ace1-906ef791ee4c\") " pod="openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8"
Sep 05 06:31:28.774541 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:28.774450    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8"
Sep 05 06:31:29.055562 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.055532    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8"]
Sep 05 06:31:29.057799 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:31:29.057770    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd783a45f_8564_44f1_ace1_906ef791ee4c.slice/crio-9ff024fa860b2675ca1bf841205a168b12e87f811feb7aa5dbbef549c59e8a2d WatchSource:0}: Error finding container 9ff024fa860b2675ca1bf841205a168b12e87f811feb7aa5dbbef549c59e8a2d: Status 404 returned error can't find the container with id 9ff024fa860b2675ca1bf841205a168b12e87f811feb7aa5dbbef549c59e8a2d
Sep 05 06:31:29.060206 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.060188    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 06:31:29.275194 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.275158    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8" event={"ID":"d783a45f-8564-44f1-ace1-906ef791ee4c","Type":"ContainerStarted","Data":"9ff024fa860b2675ca1bf841205a168b12e87f811feb7aa5dbbef549c59e8a2d"}
Sep 05 06:31:29.437402 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.437376    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"]
Sep 05 06:31:29.441637 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.441619    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.444308 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.444281    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 05 06:31:29.444599 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.444583    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 05 06:31:29.444675 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.444588    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wqmcp\""
Sep 05 06:31:29.444799 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.444786    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 05 06:31:29.445022 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.445007    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 05 06:31:29.456860 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.456800    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"]
Sep 05 06:31:29.516095 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.516053    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.516095 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.516102    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.516310 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.516144    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.516310 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.516255    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.516310 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.516300    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssh79\" (UniqueName: \"kubernetes.io/projected/9846707f-1ab7-4ac8-b0be-3025811c4ee9-kube-api-access-ssh79\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.516420 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.516340    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9846707f-1ab7-4ac8-b0be-3025811c4ee9-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.516420 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.516367    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.617160 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.617120    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.617160 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.617168    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.617405 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.617186    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.617405 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.617208    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.617405 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.617229    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssh79\" (UniqueName: \"kubernetes.io/projected/9846707f-1ab7-4ac8-b0be-3025811c4ee9-kube-api-access-ssh79\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.617405 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.617258    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9846707f-1ab7-4ac8-b0be-3025811c4ee9-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.617405 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.617273    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.618143 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.618091    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.619735 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.619703    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9846707f-1ab7-4ac8-b0be-3025811c4ee9-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.619898 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.619833    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.619979 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.619956    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.620056 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.620034    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.630798 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.630771    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssh79\" (UniqueName: \"kubernetes.io/projected/9846707f-1ab7-4ac8-b0be-3025811c4ee9-kube-api-access-ssh79\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.633304 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.633282    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-nqr7q\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:29.751269 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:29.750908    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:30.051948 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:30.051846    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"]
Sep 05 06:31:30.055652 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:31:30.055613    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9846707f_1ab7_4ac8_b0be_3025811c4ee9.slice/crio-3f696738be3eb15187d28ddbc0900dd2fb694ff00e255b6f05856a160b3dcfc5 WatchSource:0}: Error finding container 3f696738be3eb15187d28ddbc0900dd2fb694ff00e255b6f05856a160b3dcfc5: Status 404 returned error can't find the container with id 3f696738be3eb15187d28ddbc0900dd2fb694ff00e255b6f05856a160b3dcfc5
Sep 05 06:31:30.283258 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:30.283198    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q" event={"ID":"9846707f-1ab7-4ac8-b0be-3025811c4ee9","Type":"ContainerStarted","Data":"3f696738be3eb15187d28ddbc0900dd2fb694ff00e255b6f05856a160b3dcfc5"}
Sep 05 06:31:32.297740 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:32.297694    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8" event={"ID":"d783a45f-8564-44f1-ace1-906ef791ee4c","Type":"ContainerStarted","Data":"7135dbd6042a34f7d20fc428fe9c697c1e14b7104e3a74d5b78bbf5559e7f8ff"}
Sep 05 06:31:32.321410 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:32.321327    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6bd96764bd-6zzt8" podStartSLOduration=2.465211186 podStartE2EDuration="5.32129957s" totalImagesPullingTime="2.856088384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:31:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:31:32.320621617 +0000 UTC m=+308.911180570" watchObservedRunningTime="2026-09-05 06:31:32.32129957 +0000 UTC m=+308.911858523"
Sep 05 06:31:33.726501 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:33.726457    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:31:33.726777 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:33.726564    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:31:34.310133 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:34.310092    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q" event={"ID":"9846707f-1ab7-4ac8-b0be-3025811c4ee9","Type":"ContainerStarted","Data":"deefe0fff18997511bb61ab5979a18d7fea21fc231e1290df3fa88b638ac448b"}
Sep 05 06:31:34.310354 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:34.310165    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:34.311900 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:34.311856    2559 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-nqr7q container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 05 06:31:34.312022 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:34.311927    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q" podUID="9846707f-1ab7-4ac8-b0be-3025811c4ee9" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 05 06:31:34.334713 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:34.334643    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q" podStartSLOduration=1.667381968 podStartE2EDuration="5.334623022s" totalImagesPullingTime="3.667241054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:31:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:31:34.333459017 +0000 UTC m=+310.924017969" watchObservedRunningTime="2026-09-05 06:31:34.334623022 +0000 UTC m=+310.925181974"
Sep 05 06:31:35.316027 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:35.315992    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:39.310374 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:39.310335    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gzgqb"]
Sep 05 06:31:39.315087 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:39.315066    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gzgqb"
Sep 05 06:31:39.317709 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:39.317680    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 05 06:31:39.327307 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:39.327274    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gzgqb"]
Sep 05 06:31:39.408057 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:39.408012    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99fcb1d8-584f-4b9a-beab-d0c1636b398a-tls-certs\") pod \"model-serving-api-54d89bd5fc-gzgqb\" (UID: \"99fcb1d8-584f-4b9a-beab-d0c1636b398a\") " pod="opendatahub/model-serving-api-54d89bd5fc-gzgqb"
Sep 05 06:31:39.408226 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:39.408067    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pkgr\" (UniqueName: \"kubernetes.io/projected/99fcb1d8-584f-4b9a-beab-d0c1636b398a-kube-api-access-9pkgr\") pod \"model-serving-api-54d89bd5fc-gzgqb\" (UID: \"99fcb1d8-584f-4b9a-beab-d0c1636b398a\") " pod="opendatahub/model-serving-api-54d89bd5fc-gzgqb"
Sep 05 06:31:39.509412 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:39.509369    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99fcb1d8-584f-4b9a-beab-d0c1636b398a-tls-certs\") pod \"model-serving-api-54d89bd5fc-gzgqb\" (UID: \"99fcb1d8-584f-4b9a-beab-d0c1636b398a\") " pod="opendatahub/model-serving-api-54d89bd5fc-gzgqb"
Sep 05 06:31:39.509412 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:39.509419    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pkgr\" (UniqueName: \"kubernetes.io/projected/99fcb1d8-584f-4b9a-beab-d0c1636b398a-kube-api-access-9pkgr\") pod \"model-serving-api-54d89bd5fc-gzgqb\" (UID: \"99fcb1d8-584f-4b9a-beab-d0c1636b398a\") " pod="opendatahub/model-serving-api-54d89bd5fc-gzgqb"
Sep 05 06:31:39.509636 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:31:39.509549    2559 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 05 06:31:39.509636 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:31:39.509634    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fcb1d8-584f-4b9a-beab-d0c1636b398a-tls-certs podName:99fcb1d8-584f-4b9a-beab-d0c1636b398a nodeName:}" failed. No retries permitted until 2026-09-05 06:31:40.009616728 +0000 UTC m=+316.600175662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/99fcb1d8-584f-4b9a-beab-d0c1636b398a-tls-certs") pod "model-serving-api-54d89bd5fc-gzgqb" (UID: "99fcb1d8-584f-4b9a-beab-d0c1636b398a") : secret "model-serving-api-tls" not found
Sep 05 06:31:39.521562 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:39.521518    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pkgr\" (UniqueName: \"kubernetes.io/projected/99fcb1d8-584f-4b9a-beab-d0c1636b398a-kube-api-access-9pkgr\") pod \"model-serving-api-54d89bd5fc-gzgqb\" (UID: \"99fcb1d8-584f-4b9a-beab-d0c1636b398a\") " pod="opendatahub/model-serving-api-54d89bd5fc-gzgqb"
Sep 05 06:31:40.014129 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:40.014045    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99fcb1d8-584f-4b9a-beab-d0c1636b398a-tls-certs\") pod \"model-serving-api-54d89bd5fc-gzgqb\" (UID: \"99fcb1d8-584f-4b9a-beab-d0c1636b398a\") " pod="opendatahub/model-serving-api-54d89bd5fc-gzgqb"
Sep 05 06:31:40.014413 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:31:40.014224    2559 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 05 06:31:40.014413 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:31:40.014313    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99fcb1d8-584f-4b9a-beab-d0c1636b398a-tls-certs podName:99fcb1d8-584f-4b9a-beab-d0c1636b398a nodeName:}" failed. No retries permitted until 2026-09-05 06:31:41.014295245 +0000 UTC m=+317.604854179 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/99fcb1d8-584f-4b9a-beab-d0c1636b398a-tls-certs") pod "model-serving-api-54d89bd5fc-gzgqb" (UID: "99fcb1d8-584f-4b9a-beab-d0c1636b398a") : secret "model-serving-api-tls" not found
Sep 05 06:31:41.024067 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:41.024006    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99fcb1d8-584f-4b9a-beab-d0c1636b398a-tls-certs\") pod \"model-serving-api-54d89bd5fc-gzgqb\" (UID: \"99fcb1d8-584f-4b9a-beab-d0c1636b398a\") " pod="opendatahub/model-serving-api-54d89bd5fc-gzgqb"
Sep 05 06:31:41.026684 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:41.026641    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99fcb1d8-584f-4b9a-beab-d0c1636b398a-tls-certs\") pod \"model-serving-api-54d89bd5fc-gzgqb\" (UID: \"99fcb1d8-584f-4b9a-beab-d0c1636b398a\") " pod="opendatahub/model-serving-api-54d89bd5fc-gzgqb"
Sep 05 06:31:41.126617 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:41.126564    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gzgqb"
Sep 05 06:31:41.416119 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:41.416088    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gzgqb"]
Sep 05 06:31:41.419037 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:31:41.419003    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99fcb1d8_584f_4b9a_beab_d0c1636b398a.slice/crio-24fd153eafb1ba09184bd61e79d48cb1f071b8f48602252c4b5e9ab0b2bb077e WatchSource:0}: Error finding container 24fd153eafb1ba09184bd61e79d48cb1f071b8f48602252c4b5e9ab0b2bb077e: Status 404 returned error can't find the container with id 24fd153eafb1ba09184bd61e79d48cb1f071b8f48602252c4b5e9ab0b2bb077e
Sep 05 06:31:42.353543 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:42.353492    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gzgqb" event={"ID":"99fcb1d8-584f-4b9a-beab-d0c1636b398a","Type":"ContainerStarted","Data":"24fd153eafb1ba09184bd61e79d48cb1f071b8f48602252c4b5e9ab0b2bb077e"}
Sep 05 06:31:43.360503 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:43.360383    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gzgqb" event={"ID":"99fcb1d8-584f-4b9a-beab-d0c1636b398a","Type":"ContainerStarted","Data":"98801dcb4e30d3bad3461d4b7255921073e6ee65d1cc61e8d32eb5c0ee911f04"}
Sep 05 06:31:43.361027 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:43.360549    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-gzgqb"
Sep 05 06:31:43.380455 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:43.380380    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-gzgqb" podStartSLOduration=2.751591391 podStartE2EDuration="4.380354169s" totalImagesPullingTime="1.628762778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:31:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:31:43.380140734 +0000 UTC m=+319.970699687" watchObservedRunningTime="2026-09-05 06:31:43.380354169 +0000 UTC m=+319.970913122"
Sep 05 06:31:49.291105 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.291056    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-59mrk"]
Sep 05 06:31:49.293349 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.293321    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-59mrk"
Sep 05 06:31:49.301522 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.301485    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 05 06:31:49.301673 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.301596    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-5s2fl\""
Sep 05 06:31:49.311718 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.311688    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-59mrk"]
Sep 05 06:31:49.319876 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.319839    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5rvzx"]
Sep 05 06:31:49.322601 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.322578    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5rvzx"
Sep 05 06:31:49.326218 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.326195    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-9g2vk\""
Sep 05 06:31:49.326218 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.326200    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 05 06:31:49.352727 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.352686    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5rvzx"]
Sep 05 06:31:49.402796 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.402748    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blznx\" (UniqueName: \"kubernetes.io/projected/7dacb06e-b358-46b8-8508-f27c61f168eb-kube-api-access-blznx\") pod \"llmisvc-controller-manager-777c6bd7d5-5rvzx\" (UID: \"7dacb06e-b358-46b8-8508-f27c61f168eb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5rvzx"
Sep 05 06:31:49.403038 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.402807    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7dacb06e-b358-46b8-8508-f27c61f168eb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5rvzx\" (UID: \"7dacb06e-b358-46b8-8508-f27c61f168eb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5rvzx"
Sep 05 06:31:49.403038 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.402920    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c5aeb09-5dd2-49fd-9114-6791fbf0d742-cert\") pod \"kserve-controller-manager-d6f4b65b8-59mrk\" (UID: \"9c5aeb09-5dd2-49fd-9114-6791fbf0d742\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-59mrk"
Sep 05 06:31:49.403038 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.402950    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2twmf\" (UniqueName: \"kubernetes.io/projected/9c5aeb09-5dd2-49fd-9114-6791fbf0d742-kube-api-access-2twmf\") pod \"kserve-controller-manager-d6f4b65b8-59mrk\" (UID: \"9c5aeb09-5dd2-49fd-9114-6791fbf0d742\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-59mrk"
Sep 05 06:31:49.503939 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.503892    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blznx\" (UniqueName: \"kubernetes.io/projected/7dacb06e-b358-46b8-8508-f27c61f168eb-kube-api-access-blznx\") pod \"llmisvc-controller-manager-777c6bd7d5-5rvzx\" (UID: \"7dacb06e-b358-46b8-8508-f27c61f168eb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5rvzx"
Sep 05 06:31:49.504162 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.503960    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7dacb06e-b358-46b8-8508-f27c61f168eb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5rvzx\" (UID: \"7dacb06e-b358-46b8-8508-f27c61f168eb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5rvzx"
Sep 05 06:31:49.504162 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.503993    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c5aeb09-5dd2-49fd-9114-6791fbf0d742-cert\") pod \"kserve-controller-manager-d6f4b65b8-59mrk\" (UID: \"9c5aeb09-5dd2-49fd-9114-6791fbf0d742\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-59mrk"
Sep 05 06:31:49.504162 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.504012    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2twmf\" (UniqueName: \"kubernetes.io/projected/9c5aeb09-5dd2-49fd-9114-6791fbf0d742-kube-api-access-2twmf\") pod \"kserve-controller-manager-d6f4b65b8-59mrk\" (UID: \"9c5aeb09-5dd2-49fd-9114-6791fbf0d742\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-59mrk"
Sep 05 06:31:49.504162 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:31:49.504140    2559 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 05 06:31:49.504371 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:31:49.504215    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dacb06e-b358-46b8-8508-f27c61f168eb-cert podName:7dacb06e-b358-46b8-8508-f27c61f168eb nodeName:}" failed. No retries permitted until 2026-09-05 06:31:50.004193383 +0000 UTC m=+326.594752331 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7dacb06e-b358-46b8-8508-f27c61f168eb-cert") pod "llmisvc-controller-manager-777c6bd7d5-5rvzx" (UID: "7dacb06e-b358-46b8-8508-f27c61f168eb") : secret "llmisvc-webhook-server-cert" not found
Sep 05 06:31:49.506728 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.506700    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c5aeb09-5dd2-49fd-9114-6791fbf0d742-cert\") pod \"kserve-controller-manager-d6f4b65b8-59mrk\" (UID: \"9c5aeb09-5dd2-49fd-9114-6791fbf0d742\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-59mrk"
Sep 05 06:31:49.515712 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.515675    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2twmf\" (UniqueName: \"kubernetes.io/projected/9c5aeb09-5dd2-49fd-9114-6791fbf0d742-kube-api-access-2twmf\") pod \"kserve-controller-manager-d6f4b65b8-59mrk\" (UID: \"9c5aeb09-5dd2-49fd-9114-6791fbf0d742\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-59mrk"
Sep 05 06:31:49.517574 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.517549    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blznx\" (UniqueName: \"kubernetes.io/projected/7dacb06e-b358-46b8-8508-f27c61f168eb-kube-api-access-blznx\") pod \"llmisvc-controller-manager-777c6bd7d5-5rvzx\" (UID: \"7dacb06e-b358-46b8-8508-f27c61f168eb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5rvzx"
Sep 05 06:31:49.604085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.603984    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-59mrk"
Sep 05 06:31:49.834991 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.834940    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"]
Sep 05 06:31:49.835343 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.835311    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q" podUID="9846707f-1ab7-4ac8-b0be-3025811c4ee9" containerName="discovery" containerID="cri-o://deefe0fff18997511bb61ab5979a18d7fea21fc231e1290df3fa88b638ac448b" gracePeriod=30
Sep 05 06:31:49.890792 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:49.890758    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-59mrk"]
Sep 05 06:31:49.914988 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:31:49.914945    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c5aeb09_5dd2_49fd_9114_6791fbf0d742.slice/crio-8d79309243a1cdaf986088de92e2cb06c2a7e82dca95e961ce9d85ce035e5375 WatchSource:0}: Error finding container 8d79309243a1cdaf986088de92e2cb06c2a7e82dca95e961ce9d85ce035e5375: Status 404 returned error can't find the container with id 8d79309243a1cdaf986088de92e2cb06c2a7e82dca95e961ce9d85ce035e5375
Sep 05 06:31:50.010073 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.009489    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7dacb06e-b358-46b8-8508-f27c61f168eb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5rvzx\" (UID: \"7dacb06e-b358-46b8-8508-f27c61f168eb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5rvzx"
Sep 05 06:31:50.012393 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.012354    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7dacb06e-b358-46b8-8508-f27c61f168eb-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5rvzx\" (UID: \"7dacb06e-b358-46b8-8508-f27c61f168eb\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5rvzx"
Sep 05 06:31:50.077894 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.077854    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:50.212489 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.212452    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-cacerts\") pod \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") "
Sep 05 06:31:50.212489 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.212495    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-csr-ca-configmap\") pod \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") "
Sep 05 06:31:50.212772 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.212525    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssh79\" (UniqueName: \"kubernetes.io/projected/9846707f-1ab7-4ac8-b0be-3025811c4ee9-kube-api-access-ssh79\") pod \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") "
Sep 05 06:31:50.212772 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.212551    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-token\") pod \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") "
Sep 05 06:31:50.212772 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.212568    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-kubeconfig\") pod \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") "
Sep 05 06:31:50.212772 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.212613    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-csr-dns-cert\") pod \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") "
Sep 05 06:31:50.212772 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.212634    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9846707f-1ab7-4ac8-b0be-3025811c4ee9-local-certs\") pod \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\" (UID: \"9846707f-1ab7-4ac8-b0be-3025811c4ee9\") "
Sep 05 06:31:50.213288 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.213222    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "9846707f-1ab7-4ac8-b0be-3025811c4ee9" (UID: "9846707f-1ab7-4ac8-b0be-3025811c4ee9"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:31:50.215420 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.215381    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "9846707f-1ab7-4ac8-b0be-3025811c4ee9" (UID: "9846707f-1ab7-4ac8-b0be-3025811c4ee9"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:31:50.215556 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.215482    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "9846707f-1ab7-4ac8-b0be-3025811c4ee9" (UID: "9846707f-1ab7-4ac8-b0be-3025811c4ee9"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:31:50.215631 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.215559    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-cacerts" (OuterVolumeSpecName: "cacerts") pod "9846707f-1ab7-4ac8-b0be-3025811c4ee9" (UID: "9846707f-1ab7-4ac8-b0be-3025811c4ee9"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:31:50.215631 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.215565    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9846707f-1ab7-4ac8-b0be-3025811c4ee9-local-certs" (OuterVolumeSpecName: "local-certs") pod "9846707f-1ab7-4ac8-b0be-3025811c4ee9" (UID: "9846707f-1ab7-4ac8-b0be-3025811c4ee9"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:31:50.215631 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.215602    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9846707f-1ab7-4ac8-b0be-3025811c4ee9-kube-api-access-ssh79" (OuterVolumeSpecName: "kube-api-access-ssh79") pod "9846707f-1ab7-4ac8-b0be-3025811c4ee9" (UID: "9846707f-1ab7-4ac8-b0be-3025811c4ee9"). InnerVolumeSpecName "kube-api-access-ssh79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:31:50.215840 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.215817    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-token" (OuterVolumeSpecName: "istio-token") pod "9846707f-1ab7-4ac8-b0be-3025811c4ee9" (UID: "9846707f-1ab7-4ac8-b0be-3025811c4ee9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:31:50.234092 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.234048    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5rvzx"
Sep 05 06:31:50.313518 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.313483    2559 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-cacerts\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:31:50.313518 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.313515    2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-csr-ca-configmap\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:31:50.313518 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.313527    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssh79\" (UniqueName: \"kubernetes.io/projected/9846707f-1ab7-4ac8-b0be-3025811c4ee9-kube-api-access-ssh79\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:31:50.314046 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.313538    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-token\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:31:50.314046 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.313548    2559 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-kubeconfig\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:31:50.314046 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.313555    2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9846707f-1ab7-4ac8-b0be-3025811c4ee9-istio-csr-dns-cert\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:31:50.314046 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.313565    2559 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9846707f-1ab7-4ac8-b0be-3025811c4ee9-local-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:31:50.394255 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.394207    2559 generic.go:356] "Generic (PLEG): container finished" podID="9846707f-1ab7-4ac8-b0be-3025811c4ee9" containerID="deefe0fff18997511bb61ab5979a18d7fea21fc231e1290df3fa88b638ac448b" exitCode=0
Sep 05 06:31:50.394255 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.394253    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"
Sep 05 06:31:50.394525 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.394288    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q" event={"ID":"9846707f-1ab7-4ac8-b0be-3025811c4ee9","Type":"ContainerDied","Data":"deefe0fff18997511bb61ab5979a18d7fea21fc231e1290df3fa88b638ac448b"}
Sep 05 06:31:50.394525 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.394354    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q" event={"ID":"9846707f-1ab7-4ac8-b0be-3025811c4ee9","Type":"ContainerDied","Data":"3f696738be3eb15187d28ddbc0900dd2fb694ff00e255b6f05856a160b3dcfc5"}
Sep 05 06:31:50.394525 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.394383    2559 scope.go:117] "RemoveContainer" containerID="deefe0fff18997511bb61ab5979a18d7fea21fc231e1290df3fa88b638ac448b"
Sep 05 06:31:50.395873 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.395849    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-59mrk" event={"ID":"9c5aeb09-5dd2-49fd-9114-6791fbf0d742","Type":"ContainerStarted","Data":"8d79309243a1cdaf986088de92e2cb06c2a7e82dca95e961ce9d85ce035e5375"}
Sep 05 06:31:50.404097 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.404074    2559 scope.go:117] "RemoveContainer" containerID="deefe0fff18997511bb61ab5979a18d7fea21fc231e1290df3fa88b638ac448b"
Sep 05 06:31:50.404389 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:31:50.404369    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deefe0fff18997511bb61ab5979a18d7fea21fc231e1290df3fa88b638ac448b\": container with ID starting with deefe0fff18997511bb61ab5979a18d7fea21fc231e1290df3fa88b638ac448b not found: ID does not exist" containerID="deefe0fff18997511bb61ab5979a18d7fea21fc231e1290df3fa88b638ac448b"
Sep 05 06:31:50.404441 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.404400    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deefe0fff18997511bb61ab5979a18d7fea21fc231e1290df3fa88b638ac448b"} err="failed to get container status \"deefe0fff18997511bb61ab5979a18d7fea21fc231e1290df3fa88b638ac448b\": rpc error: code = NotFound desc = could not find container \"deefe0fff18997511bb61ab5979a18d7fea21fc231e1290df3fa88b638ac448b\": container with ID starting with deefe0fff18997511bb61ab5979a18d7fea21fc231e1290df3fa88b638ac448b not found: ID does not exist"
Sep 05 06:31:50.421967 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.421924    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"]
Sep 05 06:31:50.424074 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.424042    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-nqr7q"]
Sep 05 06:31:50.555173 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:50.555137    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5rvzx"]
Sep 05 06:31:50.557990 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:31:50.557944    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7dacb06e_b358_46b8_8508_f27c61f168eb.slice/crio-6113d66803f339c8abe7b6285c354372e387e282b9fd662ffbccaaec2f495572 WatchSource:0}: Error finding container 6113d66803f339c8abe7b6285c354372e387e282b9fd662ffbccaaec2f495572: Status 404 returned error can't find the container with id 6113d66803f339c8abe7b6285c354372e387e282b9fd662ffbccaaec2f495572
Sep 05 06:31:51.405953 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:51.405874    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5rvzx" event={"ID":"7dacb06e-b358-46b8-8508-f27c61f168eb","Type":"ContainerStarted","Data":"6113d66803f339c8abe7b6285c354372e387e282b9fd662ffbccaaec2f495572"}
Sep 05 06:31:51.893560 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:51.893510    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9846707f-1ab7-4ac8-b0be-3025811c4ee9" path="/var/lib/kubelet/pods/9846707f-1ab7-4ac8-b0be-3025811c4ee9/volumes"
Sep 05 06:31:53.416464 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:53.416429    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-59mrk" event={"ID":"9c5aeb09-5dd2-49fd-9114-6791fbf0d742","Type":"ContainerStarted","Data":"7d9c321d70cbaabb39c9e384ee07ce1c1907ab82ea0540a0c3c7fdde2ee0b0b9"}
Sep 05 06:31:53.416869 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:53.416510    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-59mrk"
Sep 05 06:31:53.442017 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:53.441956    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-59mrk" podStartSLOduration=1.977328833 podStartE2EDuration="4.441874173s" totalImagesPullingTime="2.46454534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:31:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:31:53.439419413 +0000 UTC m=+330.029978364" watchObservedRunningTime="2026-09-05 06:31:53.441874173 +0000 UTC m=+330.032433124"
Sep 05 06:31:54.368995 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:54.368953    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-gzgqb"
Sep 05 06:31:54.423938 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:54.423868    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5rvzx" event={"ID":"7dacb06e-b358-46b8-8508-f27c61f168eb","Type":"ContainerStarted","Data":"ac4871d24b71921c5a5c2d79dece5fa05c09a075761aedd770eee95beb46a112"}
Sep 05 06:31:54.424525 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:54.424498    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5rvzx"
Sep 05 06:31:54.442807 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:31:54.442745    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5rvzx" podStartSLOduration=2.647463029 podStartE2EDuration="5.442725193s" totalImagesPullingTime="2.795262164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:31:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:31:54.442560975 +0000 UTC m=+331.033119926" watchObservedRunningTime="2026-09-05 06:31:54.442725193 +0000 UTC m=+331.033284286"
Sep 05 06:32:24.430530 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:24.430488    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-59mrk"
Sep 05 06:32:25.431779 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:25.431738    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5rvzx"
Sep 05 06:32:43.057000 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.056958    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9"]
Sep 05 06:32:43.057450 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.057326    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9846707f-1ab7-4ac8-b0be-3025811c4ee9" containerName="discovery"
Sep 05 06:32:43.057450 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.057341    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9846707f-1ab7-4ac8-b0be-3025811c4ee9" containerName="discovery"
Sep 05 06:32:43.057450 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.057410    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9846707f-1ab7-4ac8-b0be-3025811c4ee9" containerName="discovery"
Sep 05 06:32:43.062666 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.062641    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9"
Sep 05 06:32:43.065745 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.065720    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 05 06:32:43.066456 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.066436    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-g76sk\""
Sep 05 06:32:43.066456 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.066445    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 05 06:32:43.071775 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.071748    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9"]
Sep 05 06:32:43.158029 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.157988    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr"]
Sep 05 06:32:43.160480 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.160456    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr"
Sep 05 06:32:43.170716 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.170684    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr"]
Sep 05 06:32:43.192191 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.192156    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/00396ed4-1a15-4edf-88ce-3a4d9be3b532-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9\" (UID: \"00396ed4-1a15-4edf-88ce-3a4d9be3b532\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9"
Sep 05 06:32:43.192381 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.192233    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/00396ed4-1a15-4edf-88ce-3a4d9be3b532-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9\" (UID: \"00396ed4-1a15-4edf-88ce-3a4d9be3b532\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9"
Sep 05 06:32:43.192381 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.192275    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfjxv\" (UniqueName: \"kubernetes.io/projected/00396ed4-1a15-4edf-88ce-3a4d9be3b532-kube-api-access-cfjxv\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9\" (UID: \"00396ed4-1a15-4edf-88ce-3a4d9be3b532\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9"
Sep 05 06:32:43.259301 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.255656    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh"]
Sep 05 06:32:43.260951 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.260396    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh"
Sep 05 06:32:43.272218 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.272186    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh"]
Sep 05 06:32:43.293116 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.293049    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfjxv\" (UniqueName: \"kubernetes.io/projected/00396ed4-1a15-4edf-88ce-3a4d9be3b532-kube-api-access-cfjxv\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9\" (UID: \"00396ed4-1a15-4edf-88ce-3a4d9be3b532\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9"
Sep 05 06:32:43.293338 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.293122    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f0f883e-2600-4114-a7d2-f339b1678ad5-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr\" (UID: \"3f0f883e-2600-4114-a7d2-f339b1678ad5\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr"
Sep 05 06:32:43.293338 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.293182    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/00396ed4-1a15-4edf-88ce-3a4d9be3b532-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9\" (UID: \"00396ed4-1a15-4edf-88ce-3a4d9be3b532\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9"
Sep 05 06:32:43.293338 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.293220    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gqk7\" (UniqueName: \"kubernetes.io/projected/3f0f883e-2600-4114-a7d2-f339b1678ad5-kube-api-access-6gqk7\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr\" (UID: \"3f0f883e-2600-4114-a7d2-f339b1678ad5\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr"
Sep 05 06:32:43.293338 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.293282    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f0f883e-2600-4114-a7d2-f339b1678ad5-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr\" (UID: \"3f0f883e-2600-4114-a7d2-f339b1678ad5\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr"
Sep 05 06:32:43.293338 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.293329    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/00396ed4-1a15-4edf-88ce-3a4d9be3b532-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9\" (UID: \"00396ed4-1a15-4edf-88ce-3a4d9be3b532\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9"
Sep 05 06:32:43.293801 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.293774    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/00396ed4-1a15-4edf-88ce-3a4d9be3b532-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9\" (UID: \"00396ed4-1a15-4edf-88ce-3a4d9be3b532\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9"
Sep 05 06:32:43.293952 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.293915    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/00396ed4-1a15-4edf-88ce-3a4d9be3b532-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9\" (UID: \"00396ed4-1a15-4edf-88ce-3a4d9be3b532\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9"
Sep 05 06:32:43.303771 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.303744    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfjxv\" (UniqueName: \"kubernetes.io/projected/00396ed4-1a15-4edf-88ce-3a4d9be3b532-kube-api-access-cfjxv\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9\" (UID: \"00396ed4-1a15-4edf-88ce-3a4d9be3b532\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9"
Sep 05 06:32:43.357420 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.357327    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd"]
Sep 05 06:32:43.359934 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.359916    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd"
Sep 05 06:32:43.370055 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.370027    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd"]
Sep 05 06:32:43.373635 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.373610    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9"
Sep 05 06:32:43.394046 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.394001    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gqk7\" (UniqueName: \"kubernetes.io/projected/3f0f883e-2600-4114-a7d2-f339b1678ad5-kube-api-access-6gqk7\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr\" (UID: \"3f0f883e-2600-4114-a7d2-f339b1678ad5\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr"
Sep 05 06:32:43.394191 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.394107    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f0f883e-2600-4114-a7d2-f339b1678ad5-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr\" (UID: \"3f0f883e-2600-4114-a7d2-f339b1678ad5\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr"
Sep 05 06:32:43.394191 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.394153    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mqn9\" (UniqueName: \"kubernetes.io/projected/9fefd764-c745-416a-97c6-0fdf5d436da7-kube-api-access-6mqn9\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh\" (UID: \"9fefd764-c745-416a-97c6-0fdf5d436da7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh"
Sep 05 06:32:43.394273 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.394201    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fefd764-c745-416a-97c6-0fdf5d436da7-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh\" (UID: \"9fefd764-c745-416a-97c6-0fdf5d436da7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh"
Sep 05 06:32:43.394273 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.394267    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f0f883e-2600-4114-a7d2-f339b1678ad5-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr\" (UID: \"3f0f883e-2600-4114-a7d2-f339b1678ad5\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr"
Sep 05 06:32:43.394355 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.394335    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fefd764-c745-416a-97c6-0fdf5d436da7-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh\" (UID: \"9fefd764-c745-416a-97c6-0fdf5d436da7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh"
Sep 05 06:32:43.394543 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.394524    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f0f883e-2600-4114-a7d2-f339b1678ad5-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr\" (UID: \"3f0f883e-2600-4114-a7d2-f339b1678ad5\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr"
Sep 05 06:32:43.394631 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.394609    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f0f883e-2600-4114-a7d2-f339b1678ad5-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr\" (UID: \"3f0f883e-2600-4114-a7d2-f339b1678ad5\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr"
Sep 05 06:32:43.404055 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.404017    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gqk7\" (UniqueName: \"kubernetes.io/projected/3f0f883e-2600-4114-a7d2-f339b1678ad5-kube-api-access-6gqk7\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr\" (UID: \"3f0f883e-2600-4114-a7d2-f339b1678ad5\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr"
Sep 05 06:32:43.470322 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.470284    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr"
Sep 05 06:32:43.495619 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.495562    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mqn9\" (UniqueName: \"kubernetes.io/projected/9fefd764-c745-416a-97c6-0fdf5d436da7-kube-api-access-6mqn9\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh\" (UID: \"9fefd764-c745-416a-97c6-0fdf5d436da7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh"
Sep 05 06:32:43.495803 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.495646    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fefd764-c745-416a-97c6-0fdf5d436da7-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh\" (UID: \"9fefd764-c745-416a-97c6-0fdf5d436da7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh"
Sep 05 06:32:43.495803 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.495689    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd\" (UID: \"23f95159-8e26-44c8-8e0d-bc8d4fbf925f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd"
Sep 05 06:32:43.495803 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.495733    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd\" (UID: \"23f95159-8e26-44c8-8e0d-bc8d4fbf925f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd"
Sep 05 06:32:43.495803 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.495794    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fefd764-c745-416a-97c6-0fdf5d436da7-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh\" (UID: \"9fefd764-c745-416a-97c6-0fdf5d436da7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh"
Sep 05 06:32:43.496031 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.495822    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k42cf\" (UniqueName: \"kubernetes.io/projected/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-kube-api-access-k42cf\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd\" (UID: \"23f95159-8e26-44c8-8e0d-bc8d4fbf925f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd"
Sep 05 06:32:43.496264 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.496237    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fefd764-c745-416a-97c6-0fdf5d436da7-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh\" (UID: \"9fefd764-c745-416a-97c6-0fdf5d436da7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh"
Sep 05 06:32:43.496371 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.496308    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fefd764-c745-416a-97c6-0fdf5d436da7-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh\" (UID: \"9fefd764-c745-416a-97c6-0fdf5d436da7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh"
Sep 05 06:32:43.506723 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.506684    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mqn9\" (UniqueName: \"kubernetes.io/projected/9fefd764-c745-416a-97c6-0fdf5d436da7-kube-api-access-6mqn9\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh\" (UID: \"9fefd764-c745-416a-97c6-0fdf5d436da7\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh"
Sep 05 06:32:43.570736 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.570697    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh"
Sep 05 06:32:43.596389 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.596351    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k42cf\" (UniqueName: \"kubernetes.io/projected/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-kube-api-access-k42cf\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd\" (UID: \"23f95159-8e26-44c8-8e0d-bc8d4fbf925f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd"
Sep 05 06:32:43.596562 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.596501    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd\" (UID: \"23f95159-8e26-44c8-8e0d-bc8d4fbf925f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd"
Sep 05 06:32:43.596562 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.596530    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd\" (UID: \"23f95159-8e26-44c8-8e0d-bc8d4fbf925f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd"
Sep 05 06:32:43.596913 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.596876    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd\" (UID: \"23f95159-8e26-44c8-8e0d-bc8d4fbf925f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd"
Sep 05 06:32:43.597053 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.596931    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd\" (UID: \"23f95159-8e26-44c8-8e0d-bc8d4fbf925f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd"
Sep 05 06:32:43.611223 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.611139    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k42cf\" (UniqueName: \"kubernetes.io/projected/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-kube-api-access-k42cf\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd\" (UID: \"23f95159-8e26-44c8-8e0d-bc8d4fbf925f\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd"
Sep 05 06:32:43.670769 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.670731    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd"
Sep 05 06:32:43.675651 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.675625    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9"]
Sep 05 06:32:43.679982 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:32:43.679947    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00396ed4_1a15_4edf_88ce_3a4d9be3b532.slice/crio-4a18ba6a0ea6afec1ccffd724e9c109cbfdcd0edaeff76a24c1504252c12a514 WatchSource:0}: Error finding container 4a18ba6a0ea6afec1ccffd724e9c109cbfdcd0edaeff76a24c1504252c12a514: Status 404 returned error can't find the container with id 4a18ba6a0ea6afec1ccffd724e9c109cbfdcd0edaeff76a24c1504252c12a514
Sep 05 06:32:43.759570 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:32:43.759538    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f0f883e_2600_4114_a7d2_f339b1678ad5.slice/crio-7e5ce224b53784d9576ec7b414ccaaee4ad5a47bdd8c4b35643c3809037e13ea WatchSource:0}: Error finding container 7e5ce224b53784d9576ec7b414ccaaee4ad5a47bdd8c4b35643c3809037e13ea: Status 404 returned error can't find the container with id 7e5ce224b53784d9576ec7b414ccaaee4ad5a47bdd8c4b35643c3809037e13ea
Sep 05 06:32:43.759570 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.759541    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr"]
Sep 05 06:32:43.846704 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.846672    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh"]
Sep 05 06:32:43.859131 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:32:43.859101    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fefd764_c745_416a_97c6_0fdf5d436da7.slice/crio-7927b33fa69628c5050c7193f2cd5fb7f86ed2e53c82ec4ade7ccbe8c70dd973 WatchSource:0}: Error finding container 7927b33fa69628c5050c7193f2cd5fb7f86ed2e53c82ec4ade7ccbe8c70dd973: Status 404 returned error can't find the container with id 7927b33fa69628c5050c7193f2cd5fb7f86ed2e53c82ec4ade7ccbe8c70dd973
Sep 05 06:32:43.956671 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:43.956637    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd"]
Sep 05 06:32:44.017852 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:32:44.017813    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23f95159_8e26_44c8_8e0d_bc8d4fbf925f.slice/crio-280b5e380965d836f78e3c7d9eb280c6cb97b3f16d60bf8d76b0be8d69fc364a WatchSource:0}: Error finding container 280b5e380965d836f78e3c7d9eb280c6cb97b3f16d60bf8d76b0be8d69fc364a: Status 404 returned error can't find the container with id 280b5e380965d836f78e3c7d9eb280c6cb97b3f16d60bf8d76b0be8d69fc364a
Sep 05 06:32:44.666714 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:44.666670    2559 generic.go:356] "Generic (PLEG): container finished" podID="23f95159-8e26-44c8-8e0d-bc8d4fbf925f" containerID="3b456ef2f2b77fe2722f8b3c4d720d5aa2a2131b5fbc7125fa488c3848b2ca6f" exitCode=0
Sep 05 06:32:44.667222 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:44.666763    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd" event={"ID":"23f95159-8e26-44c8-8e0d-bc8d4fbf925f","Type":"ContainerDied","Data":"3b456ef2f2b77fe2722f8b3c4d720d5aa2a2131b5fbc7125fa488c3848b2ca6f"}
Sep 05 06:32:44.667222 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:44.666829    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd" event={"ID":"23f95159-8e26-44c8-8e0d-bc8d4fbf925f","Type":"ContainerStarted","Data":"280b5e380965d836f78e3c7d9eb280c6cb97b3f16d60bf8d76b0be8d69fc364a"}
Sep 05 06:32:44.668575 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:44.668554    2559 generic.go:356] "Generic (PLEG): container finished" podID="3f0f883e-2600-4114-a7d2-f339b1678ad5" containerID="c8020bb36b63b47046c62e2a94e62d7f5351bf59fbbb339cc89aab474f1ed224" exitCode=0
Sep 05 06:32:44.668668 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:44.668640    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr" event={"ID":"3f0f883e-2600-4114-a7d2-f339b1678ad5","Type":"ContainerDied","Data":"c8020bb36b63b47046c62e2a94e62d7f5351bf59fbbb339cc89aab474f1ed224"}
Sep 05 06:32:44.668725 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:44.668697    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr" event={"ID":"3f0f883e-2600-4114-a7d2-f339b1678ad5","Type":"ContainerStarted","Data":"7e5ce224b53784d9576ec7b414ccaaee4ad5a47bdd8c4b35643c3809037e13ea"}
Sep 05 06:32:44.670512 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:44.670485    2559 generic.go:356] "Generic (PLEG): container finished" podID="00396ed4-1a15-4edf-88ce-3a4d9be3b532" containerID="c2f75a7184a897e405cf5754e4b8bd0f3dcfa7af490e925328c5216896169130" exitCode=0
Sep 05 06:32:44.670658 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:44.670568    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9" event={"ID":"00396ed4-1a15-4edf-88ce-3a4d9be3b532","Type":"ContainerDied","Data":"c2f75a7184a897e405cf5754e4b8bd0f3dcfa7af490e925328c5216896169130"}
Sep 05 06:32:44.670658 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:44.670617    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9" event={"ID":"00396ed4-1a15-4edf-88ce-3a4d9be3b532","Type":"ContainerStarted","Data":"4a18ba6a0ea6afec1ccffd724e9c109cbfdcd0edaeff76a24c1504252c12a514"}
Sep 05 06:32:44.672490 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:44.672467    2559 generic.go:356] "Generic (PLEG): container finished" podID="9fefd764-c745-416a-97c6-0fdf5d436da7" containerID="d5b6eb89437923dc0e1ed07e391fdef7b87a5009ef148fe44ac3f56e6e3526b6" exitCode=0
Sep 05 06:32:44.672589 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:44.672514    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh" event={"ID":"9fefd764-c745-416a-97c6-0fdf5d436da7","Type":"ContainerDied","Data":"d5b6eb89437923dc0e1ed07e391fdef7b87a5009ef148fe44ac3f56e6e3526b6"}
Sep 05 06:32:44.672589 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:44.672551    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh" event={"ID":"9fefd764-c745-416a-97c6-0fdf5d436da7","Type":"ContainerStarted","Data":"7927b33fa69628c5050c7193f2cd5fb7f86ed2e53c82ec4ade7ccbe8c70dd973"}
Sep 05 06:32:45.682225 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:45.682172    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd" event={"ID":"23f95159-8e26-44c8-8e0d-bc8d4fbf925f","Type":"ContainerStarted","Data":"4e178fcdd4e938780eb5487453b1e2030094db5afe6a3e4655933f48d40f467c"}
Sep 05 06:32:45.684628 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:45.684589    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr" event={"ID":"3f0f883e-2600-4114-a7d2-f339b1678ad5","Type":"ContainerStarted","Data":"743b9d0e408730399e3cdb3ee3a6a186d4fb2c6e35cabd2b9d4791635b4efbd5"}
Sep 05 06:32:45.687033 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:45.686991    2559 generic.go:356] "Generic (PLEG): container finished" podID="9fefd764-c745-416a-97c6-0fdf5d436da7" containerID="40aa6ee7c8d33f5d8d6d64545b6b0da202bae9308aed8a25a7cd04e163fd77c5" exitCode=0
Sep 05 06:32:45.687148 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:45.687071    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh" event={"ID":"9fefd764-c745-416a-97c6-0fdf5d436da7","Type":"ContainerDied","Data":"40aa6ee7c8d33f5d8d6d64545b6b0da202bae9308aed8a25a7cd04e163fd77c5"}
Sep 05 06:32:46.694428 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:46.694375    2559 generic.go:356] "Generic (PLEG): container finished" podID="00396ed4-1a15-4edf-88ce-3a4d9be3b532" containerID="b9036fea0e78e5f7ff059d6d0d341cd40db2aa3af32e1e04d2c7f5a781a7ec8b" exitCode=0
Sep 05 06:32:46.694961 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:46.694464    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9" event={"ID":"00396ed4-1a15-4edf-88ce-3a4d9be3b532","Type":"ContainerDied","Data":"b9036fea0e78e5f7ff059d6d0d341cd40db2aa3af32e1e04d2c7f5a781a7ec8b"}
Sep 05 06:32:46.697172 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:46.697144    2559 generic.go:356] "Generic (PLEG): container finished" podID="9fefd764-c745-416a-97c6-0fdf5d436da7" containerID="4dcd738668b060ce5d4d6c7a495a334838b698e1173c020432bd0be8d045bdbf" exitCode=0
Sep 05 06:32:46.697267 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:46.697241    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh" event={"ID":"9fefd764-c745-416a-97c6-0fdf5d436da7","Type":"ContainerDied","Data":"4dcd738668b060ce5d4d6c7a495a334838b698e1173c020432bd0be8d045bdbf"}
Sep 05 06:32:46.699462 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:46.699288    2559 generic.go:356] "Generic (PLEG): container finished" podID="23f95159-8e26-44c8-8e0d-bc8d4fbf925f" containerID="4e178fcdd4e938780eb5487453b1e2030094db5afe6a3e4655933f48d40f467c" exitCode=0
Sep 05 06:32:46.699462 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:46.699342    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd" event={"ID":"23f95159-8e26-44c8-8e0d-bc8d4fbf925f","Type":"ContainerDied","Data":"4e178fcdd4e938780eb5487453b1e2030094db5afe6a3e4655933f48d40f467c"}
Sep 05 06:32:46.701520 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:46.701488    2559 generic.go:356] "Generic (PLEG): container finished" podID="3f0f883e-2600-4114-a7d2-f339b1678ad5" containerID="743b9d0e408730399e3cdb3ee3a6a186d4fb2c6e35cabd2b9d4791635b4efbd5" exitCode=0
Sep 05 06:32:46.701586 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:46.701550    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr" event={"ID":"3f0f883e-2600-4114-a7d2-f339b1678ad5","Type":"ContainerDied","Data":"743b9d0e408730399e3cdb3ee3a6a186d4fb2c6e35cabd2b9d4791635b4efbd5"}
Sep 05 06:32:47.708251 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:47.708210    2559 generic.go:356] "Generic (PLEG): container finished" podID="00396ed4-1a15-4edf-88ce-3a4d9be3b532" containerID="d9d4de77eae037153e587a4fbe10db69b86f72e7792d0f01a63a69ec6355da42" exitCode=0
Sep 05 06:32:47.708689 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:47.708294    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9" event={"ID":"00396ed4-1a15-4edf-88ce-3a4d9be3b532","Type":"ContainerDied","Data":"d9d4de77eae037153e587a4fbe10db69b86f72e7792d0f01a63a69ec6355da42"}
Sep 05 06:32:47.710582 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:47.710553    2559 generic.go:356] "Generic (PLEG): container finished" podID="23f95159-8e26-44c8-8e0d-bc8d4fbf925f" containerID="5705c56337c90447445ce667da13280b95fd50f496f9f4773b0c74bf348d28ac" exitCode=0
Sep 05 06:32:47.710748 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:47.710640    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd" event={"ID":"23f95159-8e26-44c8-8e0d-bc8d4fbf925f","Type":"ContainerDied","Data":"5705c56337c90447445ce667da13280b95fd50f496f9f4773b0c74bf348d28ac"}
Sep 05 06:32:47.712825 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:47.712800    2559 generic.go:356] "Generic (PLEG): container finished" podID="3f0f883e-2600-4114-a7d2-f339b1678ad5" containerID="1dfa081594c14813dc343e531a14267e53027ab0525dd3f214eedf0acf869e0d" exitCode=0
Sep 05 06:32:47.712968 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:47.712873    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr" event={"ID":"3f0f883e-2600-4114-a7d2-f339b1678ad5","Type":"ContainerDied","Data":"1dfa081594c14813dc343e531a14267e53027ab0525dd3f214eedf0acf869e0d"}
Sep 05 06:32:47.854823 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:47.854788    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh"
Sep 05 06:32:47.938815 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:47.938769    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mqn9\" (UniqueName: \"kubernetes.io/projected/9fefd764-c745-416a-97c6-0fdf5d436da7-kube-api-access-6mqn9\") pod \"9fefd764-c745-416a-97c6-0fdf5d436da7\" (UID: \"9fefd764-c745-416a-97c6-0fdf5d436da7\") "
Sep 05 06:32:47.939059 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:47.938849    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fefd764-c745-416a-97c6-0fdf5d436da7-util\") pod \"9fefd764-c745-416a-97c6-0fdf5d436da7\" (UID: \"9fefd764-c745-416a-97c6-0fdf5d436da7\") "
Sep 05 06:32:47.939059 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:47.938965    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fefd764-c745-416a-97c6-0fdf5d436da7-bundle\") pod \"9fefd764-c745-416a-97c6-0fdf5d436da7\" (UID: \"9fefd764-c745-416a-97c6-0fdf5d436da7\") "
Sep 05 06:32:47.939456 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:47.939428    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fefd764-c745-416a-97c6-0fdf5d436da7-bundle" (OuterVolumeSpecName: "bundle") pod "9fefd764-c745-416a-97c6-0fdf5d436da7" (UID: "9fefd764-c745-416a-97c6-0fdf5d436da7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:32:47.941254 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:47.941225    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fefd764-c745-416a-97c6-0fdf5d436da7-kube-api-access-6mqn9" (OuterVolumeSpecName: "kube-api-access-6mqn9") pod "9fefd764-c745-416a-97c6-0fdf5d436da7" (UID: "9fefd764-c745-416a-97c6-0fdf5d436da7"). InnerVolumeSpecName "kube-api-access-6mqn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:32:47.946661 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:47.946625    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fefd764-c745-416a-97c6-0fdf5d436da7-util" (OuterVolumeSpecName: "util") pod "9fefd764-c745-416a-97c6-0fdf5d436da7" (UID: "9fefd764-c745-416a-97c6-0fdf5d436da7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:32:48.039980 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:48.039826    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fefd764-c745-416a-97c6-0fdf5d436da7-util\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:32:48.039980 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:48.039877    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fefd764-c745-416a-97c6-0fdf5d436da7-bundle\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:32:48.039980 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:48.039909    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mqn9\" (UniqueName: \"kubernetes.io/projected/9fefd764-c745-416a-97c6-0fdf5d436da7-kube-api-access-6mqn9\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:32:48.719627 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:48.719579    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh"
Sep 05 06:32:48.719627 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:48.719601    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bb62nqh" event={"ID":"9fefd764-c745-416a-97c6-0fdf5d436da7","Type":"ContainerDied","Data":"7927b33fa69628c5050c7193f2cd5fb7f86ed2e53c82ec4ade7ccbe8c70dd973"}
Sep 05 06:32:48.720180 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:48.719661    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7927b33fa69628c5050c7193f2cd5fb7f86ed2e53c82ec4ade7ccbe8c70dd973"
Sep 05 06:32:48.907656 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:48.907618    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9"
Sep 05 06:32:48.911022 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:48.910992    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr"
Sep 05 06:32:48.914731 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:48.914704    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd"
Sep 05 06:32:49.049463 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.049339    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gqk7\" (UniqueName: \"kubernetes.io/projected/3f0f883e-2600-4114-a7d2-f339b1678ad5-kube-api-access-6gqk7\") pod \"3f0f883e-2600-4114-a7d2-f339b1678ad5\" (UID: \"3f0f883e-2600-4114-a7d2-f339b1678ad5\") "
Sep 05 06:32:49.049463 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.049399    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f0f883e-2600-4114-a7d2-f339b1678ad5-bundle\") pod \"3f0f883e-2600-4114-a7d2-f339b1678ad5\" (UID: \"3f0f883e-2600-4114-a7d2-f339b1678ad5\") "
Sep 05 06:32:49.049463 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.049437    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f0f883e-2600-4114-a7d2-f339b1678ad5-util\") pod \"3f0f883e-2600-4114-a7d2-f339b1678ad5\" (UID: \"3f0f883e-2600-4114-a7d2-f339b1678ad5\") "
Sep 05 06:32:49.049463 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.049465    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-util\") pod \"23f95159-8e26-44c8-8e0d-bc8d4fbf925f\" (UID: \"23f95159-8e26-44c8-8e0d-bc8d4fbf925f\") "
Sep 05 06:32:49.049836 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.049496    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/00396ed4-1a15-4edf-88ce-3a4d9be3b532-util\") pod \"00396ed4-1a15-4edf-88ce-3a4d9be3b532\" (UID: \"00396ed4-1a15-4edf-88ce-3a4d9be3b532\") "
Sep 05 06:32:49.049836 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.049546    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfjxv\" (UniqueName: \"kubernetes.io/projected/00396ed4-1a15-4edf-88ce-3a4d9be3b532-kube-api-access-cfjxv\") pod \"00396ed4-1a15-4edf-88ce-3a4d9be3b532\" (UID: \"00396ed4-1a15-4edf-88ce-3a4d9be3b532\") "
Sep 05 06:32:49.049836 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.049574    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-bundle\") pod \"23f95159-8e26-44c8-8e0d-bc8d4fbf925f\" (UID: \"23f95159-8e26-44c8-8e0d-bc8d4fbf925f\") "
Sep 05 06:32:49.049836 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.049600    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/00396ed4-1a15-4edf-88ce-3a4d9be3b532-bundle\") pod \"00396ed4-1a15-4edf-88ce-3a4d9be3b532\" (UID: \"00396ed4-1a15-4edf-88ce-3a4d9be3b532\") "
Sep 05 06:32:49.049836 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.049648    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k42cf\" (UniqueName: \"kubernetes.io/projected/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-kube-api-access-k42cf\") pod \"23f95159-8e26-44c8-8e0d-bc8d4fbf925f\" (UID: \"23f95159-8e26-44c8-8e0d-bc8d4fbf925f\") "
Sep 05 06:32:49.050340 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.050256    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f0f883e-2600-4114-a7d2-f339b1678ad5-bundle" (OuterVolumeSpecName: "bundle") pod "3f0f883e-2600-4114-a7d2-f339b1678ad5" (UID: "3f0f883e-2600-4114-a7d2-f339b1678ad5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:32:49.050673 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.050570    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-bundle" (OuterVolumeSpecName: "bundle") pod "23f95159-8e26-44c8-8e0d-bc8d4fbf925f" (UID: "23f95159-8e26-44c8-8e0d-bc8d4fbf925f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:32:49.051167 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.051136    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00396ed4-1a15-4edf-88ce-3a4d9be3b532-bundle" (OuterVolumeSpecName: "bundle") pod "00396ed4-1a15-4edf-88ce-3a4d9be3b532" (UID: "00396ed4-1a15-4edf-88ce-3a4d9be3b532"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:32:49.052246 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.052216    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f0f883e-2600-4114-a7d2-f339b1678ad5-kube-api-access-6gqk7" (OuterVolumeSpecName: "kube-api-access-6gqk7") pod "3f0f883e-2600-4114-a7d2-f339b1678ad5" (UID: "3f0f883e-2600-4114-a7d2-f339b1678ad5"). InnerVolumeSpecName "kube-api-access-6gqk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:32:49.052532 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.052493    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00396ed4-1a15-4edf-88ce-3a4d9be3b532-kube-api-access-cfjxv" (OuterVolumeSpecName: "kube-api-access-cfjxv") pod "00396ed4-1a15-4edf-88ce-3a4d9be3b532" (UID: "00396ed4-1a15-4edf-88ce-3a4d9be3b532"). InnerVolumeSpecName "kube-api-access-cfjxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:32:49.052964 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.052938    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-kube-api-access-k42cf" (OuterVolumeSpecName: "kube-api-access-k42cf") pod "23f95159-8e26-44c8-8e0d-bc8d4fbf925f" (UID: "23f95159-8e26-44c8-8e0d-bc8d4fbf925f"). InnerVolumeSpecName "kube-api-access-k42cf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:32:49.056174 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.056143    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-util" (OuterVolumeSpecName: "util") pod "23f95159-8e26-44c8-8e0d-bc8d4fbf925f" (UID: "23f95159-8e26-44c8-8e0d-bc8d4fbf925f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:32:49.057525 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.057484    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00396ed4-1a15-4edf-88ce-3a4d9be3b532-util" (OuterVolumeSpecName: "util") pod "00396ed4-1a15-4edf-88ce-3a4d9be3b532" (UID: "00396ed4-1a15-4edf-88ce-3a4d9be3b532"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:32:49.058430 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.058398    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f0f883e-2600-4114-a7d2-f339b1678ad5-util" (OuterVolumeSpecName: "util") pod "3f0f883e-2600-4114-a7d2-f339b1678ad5" (UID: "3f0f883e-2600-4114-a7d2-f339b1678ad5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:32:49.151143 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.151084    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f0f883e-2600-4114-a7d2-f339b1678ad5-util\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:32:49.151143 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.151133    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-util\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:32:49.151143 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.151145    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/00396ed4-1a15-4edf-88ce-3a4d9be3b532-util\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:32:49.151418 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.151157    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfjxv\" (UniqueName: \"kubernetes.io/projected/00396ed4-1a15-4edf-88ce-3a4d9be3b532-kube-api-access-cfjxv\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:32:49.151418 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.151176    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-bundle\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:32:49.151418 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.151189    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/00396ed4-1a15-4edf-88ce-3a4d9be3b532-bundle\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:32:49.151418 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.151201    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k42cf\" (UniqueName: \"kubernetes.io/projected/23f95159-8e26-44c8-8e0d-bc8d4fbf925f-kube-api-access-k42cf\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:32:49.151418 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.151212    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gqk7\" (UniqueName: \"kubernetes.io/projected/3f0f883e-2600-4114-a7d2-f339b1678ad5-kube-api-access-6gqk7\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:32:49.151418 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.151227    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f0f883e-2600-4114-a7d2-f339b1678ad5-bundle\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:32:49.727439 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.727396    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd" event={"ID":"23f95159-8e26-44c8-8e0d-bc8d4fbf925f","Type":"ContainerDied","Data":"280b5e380965d836f78e3c7d9eb280c6cb97b3f16d60bf8d76b0be8d69fc364a"}
Sep 05 06:32:49.727964 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.727452    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cl5pkd"
Sep 05 06:32:49.727964 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.727473    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="280b5e380965d836f78e3c7d9eb280c6cb97b3f16d60bf8d76b0be8d69fc364a"
Sep 05 06:32:49.729724 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.729705    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr"
Sep 05 06:32:49.729848 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.729729    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0t5dmr" event={"ID":"3f0f883e-2600-4114-a7d2-f339b1678ad5","Type":"ContainerDied","Data":"7e5ce224b53784d9576ec7b414ccaaee4ad5a47bdd8c4b35643c3809037e13ea"}
Sep 05 06:32:49.729848 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.729767    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e5ce224b53784d9576ec7b414ccaaee4ad5a47bdd8c4b35643c3809037e13ea"
Sep 05 06:32:49.732068 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.732045    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9" event={"ID":"00396ed4-1a15-4edf-88ce-3a4d9be3b532","Type":"ContainerDied","Data":"4a18ba6a0ea6afec1ccffd724e9c109cbfdcd0edaeff76a24c1504252c12a514"}
Sep 05 06:32:49.732163 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.732076    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053629n9"
Sep 05 06:32:49.732163 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:49.732083    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a18ba6a0ea6afec1ccffd724e9c109cbfdcd0edaeff76a24c1504252c12a514"
Sep 05 06:32:54.843021 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.842975    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"]
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843328    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00396ed4-1a15-4edf-88ce-3a4d9be3b532" containerName="extract"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843340    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="00396ed4-1a15-4edf-88ce-3a4d9be3b532" containerName="extract"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843353    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f0f883e-2600-4114-a7d2-f339b1678ad5" containerName="pull"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843360    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f0f883e-2600-4114-a7d2-f339b1678ad5" containerName="pull"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843371    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23f95159-8e26-44c8-8e0d-bc8d4fbf925f" containerName="extract"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843376    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="23f95159-8e26-44c8-8e0d-bc8d4fbf925f" containerName="extract"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843385    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23f95159-8e26-44c8-8e0d-bc8d4fbf925f" containerName="util"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843390    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="23f95159-8e26-44c8-8e0d-bc8d4fbf925f" containerName="util"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843396    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23f95159-8e26-44c8-8e0d-bc8d4fbf925f" containerName="pull"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843401    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="23f95159-8e26-44c8-8e0d-bc8d4fbf925f" containerName="pull"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843409    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f0f883e-2600-4114-a7d2-f339b1678ad5" containerName="extract"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843415    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f0f883e-2600-4114-a7d2-f339b1678ad5" containerName="extract"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843425    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fefd764-c745-416a-97c6-0fdf5d436da7" containerName="pull"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843430    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fefd764-c745-416a-97c6-0fdf5d436da7" containerName="pull"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843436    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00396ed4-1a15-4edf-88ce-3a4d9be3b532" containerName="util"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843441    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="00396ed4-1a15-4edf-88ce-3a4d9be3b532" containerName="util"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843448    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f0f883e-2600-4114-a7d2-f339b1678ad5" containerName="util"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843455    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f0f883e-2600-4114-a7d2-f339b1678ad5" containerName="util"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843463    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fefd764-c745-416a-97c6-0fdf5d436da7" containerName="util"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843471    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fefd764-c745-416a-97c6-0fdf5d436da7" containerName="util"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843481    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00396ed4-1a15-4edf-88ce-3a4d9be3b532" containerName="pull"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843488    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="00396ed4-1a15-4edf-88ce-3a4d9be3b532" containerName="pull"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843507    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fefd764-c745-416a-97c6-0fdf5d436da7" containerName="extract"
Sep 05 06:32:54.843563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843515    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fefd764-c745-416a-97c6-0fdf5d436da7" containerName="extract"
Sep 05 06:32:54.844457 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843608    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f0f883e-2600-4114-a7d2-f339b1678ad5" containerName="extract"
Sep 05 06:32:54.844457 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843624    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="23f95159-8e26-44c8-8e0d-bc8d4fbf925f" containerName="extract"
Sep 05 06:32:54.844457 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843633    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fefd764-c745-416a-97c6-0fdf5d436da7" containerName="extract"
Sep 05 06:32:54.844457 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.843643    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="00396ed4-1a15-4edf-88ce-3a4d9be3b532" containerName="extract"
Sep 05 06:32:54.848343 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.848322    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"
Sep 05 06:32:54.851575 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.851550    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-rvx8p\""
Sep 05 06:32:54.851720 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.851580    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 05 06:32:54.852606 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.852585    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 05 06:32:54.863823 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:54.863787    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"]
Sep 05 06:32:55.006346 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:55.006289    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8s6s\" (UniqueName: \"kubernetes.io/projected/6bc982ec-936c-4930-85ee-83504b0903f3-kube-api-access-l8s6s\") pod \"limitador-operator-controller-manager-6c549b6d85-ct7x5\" (UID: \"6bc982ec-936c-4930-85ee-83504b0903f3\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"
Sep 05 06:32:55.107445 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:55.107325    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8s6s\" (UniqueName: \"kubernetes.io/projected/6bc982ec-936c-4930-85ee-83504b0903f3-kube-api-access-l8s6s\") pod \"limitador-operator-controller-manager-6c549b6d85-ct7x5\" (UID: \"6bc982ec-936c-4930-85ee-83504b0903f3\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"
Sep 05 06:32:55.117358 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:55.117328    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8s6s\" (UniqueName: \"kubernetes.io/projected/6bc982ec-936c-4930-85ee-83504b0903f3-kube-api-access-l8s6s\") pod \"limitador-operator-controller-manager-6c549b6d85-ct7x5\" (UID: \"6bc982ec-936c-4930-85ee-83504b0903f3\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"
Sep 05 06:32:55.158731 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:55.158684    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"
Sep 05 06:32:55.452612 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:55.452576    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"]
Sep 05 06:32:55.454434 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:32:55.454401    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bc982ec_936c_4930_85ee_83504b0903f3.slice/crio-1c8b55355b78701abafe192668998a7703b39e3d0d39c8dd2fe1f2bfa7e7c09d WatchSource:0}: Error finding container 1c8b55355b78701abafe192668998a7703b39e3d0d39c8dd2fe1f2bfa7e7c09d: Status 404 returned error can't find the container with id 1c8b55355b78701abafe192668998a7703b39e3d0d39c8dd2fe1f2bfa7e7c09d
Sep 05 06:32:55.763531 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:55.763428    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5" event={"ID":"6bc982ec-936c-4930-85ee-83504b0903f3","Type":"ContainerStarted","Data":"1c8b55355b78701abafe192668998a7703b39e3d0d39c8dd2fe1f2bfa7e7c09d"}
Sep 05 06:32:56.804663 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:56.804620    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-vjkb7"]
Sep 05 06:32:56.808255 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:56.808224    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-vjkb7"
Sep 05 06:32:56.811221 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:56.811189    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 05 06:32:56.811453 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:56.811280    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-tt2mv\""
Sep 05 06:32:56.821809 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:56.821681    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-vjkb7"]
Sep 05 06:32:56.822236 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:56.822101    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh68x\" (UniqueName: \"kubernetes.io/projected/0ae3f591-8cde-466c-99f3-5cbf47a5fbcf-kube-api-access-gh68x\") pod \"dns-operator-controller-manager-5985564776-vjkb7\" (UID: \"0ae3f591-8cde-466c-99f3-5cbf47a5fbcf\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-vjkb7"
Sep 05 06:32:56.923333 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:56.923277    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh68x\" (UniqueName: \"kubernetes.io/projected/0ae3f591-8cde-466c-99f3-5cbf47a5fbcf-kube-api-access-gh68x\") pod \"dns-operator-controller-manager-5985564776-vjkb7\" (UID: \"0ae3f591-8cde-466c-99f3-5cbf47a5fbcf\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-vjkb7"
Sep 05 06:32:56.947414 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:56.947371    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh68x\" (UniqueName: \"kubernetes.io/projected/0ae3f591-8cde-466c-99f3-5cbf47a5fbcf-kube-api-access-gh68x\") pod \"dns-operator-controller-manager-5985564776-vjkb7\" (UID: \"0ae3f591-8cde-466c-99f3-5cbf47a5fbcf\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-vjkb7"
Sep 05 06:32:57.123390 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:57.123287    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-vjkb7"
Sep 05 06:32:58.297670 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:58.297566    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-vjkb7"]
Sep 05 06:32:58.309982 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:32:58.309947    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae3f591_8cde_466c_99f3_5cbf47a5fbcf.slice/crio-b117047bd8ed492104567e0ee39ac2ac1de2088bbabd5ac8d31a319be84a221e WatchSource:0}: Error finding container b117047bd8ed492104567e0ee39ac2ac1de2088bbabd5ac8d31a319be84a221e: Status 404 returned error can't find the container with id b117047bd8ed492104567e0ee39ac2ac1de2088bbabd5ac8d31a319be84a221e
Sep 05 06:32:58.790284 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:58.790234    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5" event={"ID":"6bc982ec-936c-4930-85ee-83504b0903f3","Type":"ContainerStarted","Data":"68a024743af1f92851e9f85538d93b7f6c16f4861177cf3c17637de51757555f"}
Sep 05 06:32:58.790522 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:58.790324    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"
Sep 05 06:32:58.791820 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:58.791788    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-vjkb7" event={"ID":"0ae3f591-8cde-466c-99f3-5cbf47a5fbcf","Type":"ContainerStarted","Data":"b117047bd8ed492104567e0ee39ac2ac1de2088bbabd5ac8d31a319be84a221e"}
Sep 05 06:32:58.817995 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:32:58.817935    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5" podStartSLOduration=2.219108022 podStartE2EDuration="4.817914693s" totalImagesPullingTime="2.598806671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:32:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:32:58.815798515 +0000 UTC m=+395.406357552" watchObservedRunningTime="2026-09-05 06:32:58.817914693 +0000 UTC m=+395.408473636"
Sep 05 06:33:00.804380 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:00.804327    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-vjkb7" event={"ID":"0ae3f591-8cde-466c-99f3-5cbf47a5fbcf","Type":"ContainerStarted","Data":"4dc8c9e0eb8d893d03233d781cf9f1f8d370e65eb5e36897ba3e4694071dc586"}
Sep 05 06:33:00.804848 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:00.804545    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-vjkb7"
Sep 05 06:33:00.834369 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:00.834301    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-vjkb7" podStartSLOduration=3.287631139 podStartE2EDuration="4.834281094s" totalImagesPullingTime="1.546649955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:33:00.833021528 +0000 UTC m=+397.423580507" watchObservedRunningTime="2026-09-05 06:33:00.834281094 +0000 UTC m=+397.424840046"
Sep 05 06:33:01.479217 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:01.479176    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"]
Sep 05 06:33:01.482771 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:01.482751    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:01.486461 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:01.486438    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gkp9h\""
Sep 05 06:33:01.502293 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:01.502256    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"]
Sep 05 06:33:01.564627 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:01.564581    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzztd\" (UniqueName: \"kubernetes.io/projected/acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89-kube-api-access-bzztd\") pod \"kuadrant-operator-controller-manager-58fc56757c-mx2s2\" (UID: \"acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:01.564627 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:01.564625    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-mx2s2\" (UID: \"acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:01.665433 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:01.665379    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzztd\" (UniqueName: \"kubernetes.io/projected/acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89-kube-api-access-bzztd\") pod \"kuadrant-operator-controller-manager-58fc56757c-mx2s2\" (UID: \"acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:01.665433 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:01.665426    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-mx2s2\" (UID: \"acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:01.665936 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:01.665910    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-mx2s2\" (UID: \"acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:01.680494 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:01.680453    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzztd\" (UniqueName: \"kubernetes.io/projected/acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89-kube-api-access-bzztd\") pod \"kuadrant-operator-controller-manager-58fc56757c-mx2s2\" (UID: \"acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:01.793134 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:01.793025    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:02.089745 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:02.089695    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"]
Sep 05 06:33:02.091703 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:33:02.091668    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacac11fa_9ae6_4fb8_b0cd_bc55df3e5b89.slice/crio-e70a6a351b7607953d4e5d20f4850f06d2f515be935f8e39cca1a43778339483 WatchSource:0}: Error finding container e70a6a351b7607953d4e5d20f4850f06d2f515be935f8e39cca1a43778339483: Status 404 returned error can't find the container with id e70a6a351b7607953d4e5d20f4850f06d2f515be935f8e39cca1a43778339483
Sep 05 06:33:02.816924 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:02.816859    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2" event={"ID":"acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89","Type":"ContainerStarted","Data":"e70a6a351b7607953d4e5d20f4850f06d2f515be935f8e39cca1a43778339483"}
Sep 05 06:33:03.902835 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:03.902794    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-szsxn"]
Sep 05 06:33:03.906738 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:03.906705    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-szsxn"
Sep 05 06:33:03.909401 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:03.909373    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-2qrg5\""
Sep 05 06:33:03.910915 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:03.910874    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-szsxn"]
Sep 05 06:33:03.985819 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:03.985782    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mccb\" (UniqueName: \"kubernetes.io/projected/c546b6c3-6b40-4eea-bbbb-66859b113974-kube-api-access-8mccb\") pod \"authorino-operator-68d88948d8-szsxn\" (UID: \"c546b6c3-6b40-4eea-bbbb-66859b113974\") " pod="kuadrant-system/authorino-operator-68d88948d8-szsxn"
Sep 05 06:33:04.087316 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:04.087268    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mccb\" (UniqueName: \"kubernetes.io/projected/c546b6c3-6b40-4eea-bbbb-66859b113974-kube-api-access-8mccb\") pod \"authorino-operator-68d88948d8-szsxn\" (UID: \"c546b6c3-6b40-4eea-bbbb-66859b113974\") " pod="kuadrant-system/authorino-operator-68d88948d8-szsxn"
Sep 05 06:33:04.101956 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:04.101911    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mccb\" (UniqueName: \"kubernetes.io/projected/c546b6c3-6b40-4eea-bbbb-66859b113974-kube-api-access-8mccb\") pod \"authorino-operator-68d88948d8-szsxn\" (UID: \"c546b6c3-6b40-4eea-bbbb-66859b113974\") " pod="kuadrant-system/authorino-operator-68d88948d8-szsxn"
Sep 05 06:33:04.222681 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:04.222642    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-szsxn"
Sep 05 06:33:04.517364 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:04.516498    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-szsxn"]
Sep 05 06:33:04.520086 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:33:04.519682    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc546b6c3_6b40_4eea_bbbb_66859b113974.slice/crio-40b96093b15fe78db36d07105edaa8115fbf1acad76906c1cef3443b24ca2dd7 WatchSource:0}: Error finding container 40b96093b15fe78db36d07105edaa8115fbf1acad76906c1cef3443b24ca2dd7: Status 404 returned error can't find the container with id 40b96093b15fe78db36d07105edaa8115fbf1acad76906c1cef3443b24ca2dd7
Sep 05 06:33:04.832996 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:04.832869    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-szsxn" event={"ID":"c546b6c3-6b40-4eea-bbbb-66859b113974","Type":"ContainerStarted","Data":"40b96093b15fe78db36d07105edaa8115fbf1acad76906c1cef3443b24ca2dd7"}
Sep 05 06:33:06.847677 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:06.847628    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2" event={"ID":"acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89","Type":"ContainerStarted","Data":"2bd4c88e5deefb33f842cc043c58d7b4f4792a2a7455bdc8b1e2cb34637b657e"}
Sep 05 06:33:06.848230 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:06.847851    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:06.870489 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:06.870254    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2" podStartSLOduration=1.702665893 podStartE2EDuration="5.870229517s" totalImagesPullingTime="4.167563624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:33:06.868323005 +0000 UTC m=+403.458881968" watchObservedRunningTime="2026-09-05 06:33:06.870229517 +0000 UTC m=+403.460788472"
Sep 05 06:33:07.855226 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:07.855174    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-szsxn" event={"ID":"c546b6c3-6b40-4eea-bbbb-66859b113974","Type":"ContainerStarted","Data":"3e52b34c2abf5ee196b4d4c0a37a46bad6031ee31da0e62e95010cdafe45045a"}
Sep 05 06:33:07.855804 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:07.855372    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-szsxn"
Sep 05 06:33:07.875629 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:07.875554    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-szsxn" podStartSLOduration=2.422256146 podStartE2EDuration="4.87553072s" totalImagesPullingTime="2.453274574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:33:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:33:07.873141833 +0000 UTC m=+404.463700786" watchObservedRunningTime="2026-09-05 06:33:07.87553072 +0000 UTC m=+404.466089671"
Sep 05 06:33:09.800949 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:09.800908    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"
Sep 05 06:33:11.813567 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:11.813528    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-vjkb7"
Sep 05 06:33:17.857927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:17.857862    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:18.864297 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:18.864258    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-szsxn"
Sep 05 06:33:19.228869 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:19.228823    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"]
Sep 05 06:33:19.232642 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:19.232613    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"
Sep 05 06:33:19.248363 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:19.248324    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"]
Sep 05 06:33:19.327421 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:19.327378    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kktmf\" (UniqueName: \"kubernetes.io/projected/d00c82d9-79da-498e-9070-9d1ab1eccf0c-kube-api-access-kktmf\") pod \"kuadrant-operator-controller-manager-96db98c47-pwv9l\" (UID: \"d00c82d9-79da-498e-9070-9d1ab1eccf0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"
Sep 05 06:33:19.327421 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:19.327419    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d00c82d9-79da-498e-9070-9d1ab1eccf0c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-pwv9l\" (UID: \"d00c82d9-79da-498e-9070-9d1ab1eccf0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"
Sep 05 06:33:19.343182 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:19.343141    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"]
Sep 05 06:33:19.343436 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:19.343417    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-kktmf], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l" podUID="d00c82d9-79da-498e-9070-9d1ab1eccf0c"
Sep 05 06:33:19.428778 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:19.428728    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kktmf\" (UniqueName: \"kubernetes.io/projected/d00c82d9-79da-498e-9070-9d1ab1eccf0c-kube-api-access-kktmf\") pod \"kuadrant-operator-controller-manager-96db98c47-pwv9l\" (UID: \"d00c82d9-79da-498e-9070-9d1ab1eccf0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"
Sep 05 06:33:19.429004 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:19.428788    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d00c82d9-79da-498e-9070-9d1ab1eccf0c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-pwv9l\" (UID: \"d00c82d9-79da-498e-9070-9d1ab1eccf0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"
Sep 05 06:33:19.429244 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:19.429221    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d00c82d9-79da-498e-9070-9d1ab1eccf0c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-pwv9l\" (UID: \"d00c82d9-79da-498e-9070-9d1ab1eccf0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"
Sep 05 06:33:19.445140 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:19.445079    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kktmf\" (UniqueName: \"kubernetes.io/projected/d00c82d9-79da-498e-9070-9d1ab1eccf0c-kube-api-access-kktmf\") pod \"kuadrant-operator-controller-manager-96db98c47-pwv9l\" (UID: \"d00c82d9-79da-498e-9070-9d1ab1eccf0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"
Sep 05 06:33:19.922911 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:19.922859    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"
Sep 05 06:33:19.932055 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:19.932026    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"
Sep 05 06:33:20.033693 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.033649    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d00c82d9-79da-498e-9070-9d1ab1eccf0c-extensions-socket-volume\") pod \"d00c82d9-79da-498e-9070-9d1ab1eccf0c\" (UID: \"d00c82d9-79da-498e-9070-9d1ab1eccf0c\") "
Sep 05 06:33:20.033693 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.033696    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kktmf\" (UniqueName: \"kubernetes.io/projected/d00c82d9-79da-498e-9070-9d1ab1eccf0c-kube-api-access-kktmf\") pod \"d00c82d9-79da-498e-9070-9d1ab1eccf0c\" (UID: \"d00c82d9-79da-498e-9070-9d1ab1eccf0c\") "
Sep 05 06:33:20.034035 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.034007    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d00c82d9-79da-498e-9070-9d1ab1eccf0c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d00c82d9-79da-498e-9070-9d1ab1eccf0c" (UID: "d00c82d9-79da-498e-9070-9d1ab1eccf0c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:33:20.036208 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.036177    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d00c82d9-79da-498e-9070-9d1ab1eccf0c-kube-api-access-kktmf" (OuterVolumeSpecName: "kube-api-access-kktmf") pod "d00c82d9-79da-498e-9070-9d1ab1eccf0c" (UID: "d00c82d9-79da-498e-9070-9d1ab1eccf0c"). InnerVolumeSpecName "kube-api-access-kktmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:33:20.135128 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.135076    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d00c82d9-79da-498e-9070-9d1ab1eccf0c-extensions-socket-volume\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:33:20.135128 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.135119    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kktmf\" (UniqueName: \"kubernetes.io/projected/d00c82d9-79da-498e-9070-9d1ab1eccf0c-kube-api-access-kktmf\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:33:20.275332 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.275289    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"]
Sep 05 06:33:20.275543 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.275521    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2" podUID="acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89" containerName="manager" containerID="cri-o://2bd4c88e5deefb33f842cc043c58d7b4f4792a2a7455bdc8b1e2cb34637b657e" gracePeriod=2
Sep 05 06:33:20.287373 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.287341    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"]
Sep 05 06:33:20.289952 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.289913    2559 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-58fc56757c-mx2s2\": pods \"kuadrant-operator-controller-manager-58fc56757c-mx2s2\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:20.306445 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.306404    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"]
Sep 05 06:33:20.306785 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.306772    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89" containerName="manager"
Sep 05 06:33:20.306830 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.306788    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89" containerName="manager"
Sep 05 06:33:20.306865 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.306854    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89" containerName="manager"
Sep 05 06:33:20.310113 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.310086    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"
Sep 05 06:33:20.328814 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.328772    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"]
Sep 05 06:33:20.348451 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.348414    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"]
Sep 05 06:33:20.369077 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.369039    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"]
Sep 05 06:33:20.377806 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.377761    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-mx2s2\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:20.383664 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.383630    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"]
Sep 05 06:33:20.383924 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.383876    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5" podUID="6bc982ec-936c-4930-85ee-83504b0903f3" containerName="manager" containerID="cri-o://68a024743af1f92851e9f85538d93b7f6c16f4861177cf3c17637de51757555f" gracePeriod=2
Sep 05 06:33:20.401308 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.401272    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"]
Sep 05 06:33:20.408979 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.408907    2559 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"6bc982ec-936c-4930-85ee-83504b0903f3\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-6c549b6d85-ct7x5\": pods \"limitador-operator-controller-manager-6c549b6d85-ct7x5\" not found" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"
Sep 05 06:33:20.412852 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.412797    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-mx2s2\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:20.420604 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.420564    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-v9msv"]
Sep 05 06:33:20.420989 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.420969    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bc982ec-936c-4930-85ee-83504b0903f3" containerName="manager"
Sep 05 06:33:20.420989 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.420989    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bc982ec-936c-4930-85ee-83504b0903f3" containerName="manager"
Sep 05 06:33:20.421178 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.421045    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bc982ec-936c-4930-85ee-83504b0903f3" containerName="manager"
Sep 05 06:33:20.424163 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.424141    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-v9msv"
Sep 05 06:33:20.427299 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.427130    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-ct7x5\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="6bc982ec-936c-4930-85ee-83504b0903f3" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"
Sep 05 06:33:20.429360 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.429293    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-mx2s2\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:20.439292 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.439250    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvpjw\" (UniqueName: \"kubernetes.io/projected/8b36e819-3a44-439f-be31-91578bdcba05-kube-api-access-pvpjw\") pod \"kuadrant-operator-controller-manager-58fc56757c-89pqh\" (UID: \"8b36e819-3a44-439f-be31-91578bdcba05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"
Sep 05 06:33:20.439660 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.439634    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b36e819-3a44-439f-be31-91578bdcba05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-89pqh\" (UID: \"8b36e819-3a44-439f-be31-91578bdcba05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"
Sep 05 06:33:20.453733 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.453680    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-v9msv"]
Sep 05 06:33:20.540929 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.540872    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvpjw\" (UniqueName: \"kubernetes.io/projected/8b36e819-3a44-439f-be31-91578bdcba05-kube-api-access-pvpjw\") pod \"kuadrant-operator-controller-manager-58fc56757c-89pqh\" (UID: \"8b36e819-3a44-439f-be31-91578bdcba05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"
Sep 05 06:33:20.541207 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.541177    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vssr6\" (UniqueName: \"kubernetes.io/projected/5cb8f71d-990b-4ee1-b1e3-18931667fdac-kube-api-access-vssr6\") pod \"limitador-operator-controller-manager-6c549b6d85-v9msv\" (UID: \"5cb8f71d-990b-4ee1-b1e3-18931667fdac\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-v9msv"
Sep 05 06:33:20.541404 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.541369    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b36e819-3a44-439f-be31-91578bdcba05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-89pqh\" (UID: \"8b36e819-3a44-439f-be31-91578bdcba05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"
Sep 05 06:33:20.541759 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.541738    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b36e819-3a44-439f-be31-91578bdcba05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-89pqh\" (UID: \"8b36e819-3a44-439f-be31-91578bdcba05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"
Sep 05 06:33:20.546999 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.546979    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:20.549684 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.549657    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-mx2s2\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:20.552652 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.552624    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-ct7x5\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="6bc982ec-936c-4930-85ee-83504b0903f3" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"
Sep 05 06:33:20.561789 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.561763    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvpjw\" (UniqueName: \"kubernetes.io/projected/8b36e819-3a44-439f-be31-91578bdcba05-kube-api-access-pvpjw\") pod \"kuadrant-operator-controller-manager-58fc56757c-89pqh\" (UID: \"8b36e819-3a44-439f-be31-91578bdcba05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"
Sep 05 06:33:20.642361 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.642326    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89-extensions-socket-volume\") pod \"acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89\" (UID: \"acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89\") "
Sep 05 06:33:20.642532 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.642449    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzztd\" (UniqueName: \"kubernetes.io/projected/acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89-kube-api-access-bzztd\") pod \"acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89\" (UID: \"acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89\") "
Sep 05 06:33:20.642664 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.642642    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vssr6\" (UniqueName: \"kubernetes.io/projected/5cb8f71d-990b-4ee1-b1e3-18931667fdac-kube-api-access-vssr6\") pod \"limitador-operator-controller-manager-6c549b6d85-v9msv\" (UID: \"5cb8f71d-990b-4ee1-b1e3-18931667fdac\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-v9msv"
Sep 05 06:33:20.642808 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.642783    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89" (UID: "acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:33:20.644818 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.644791    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89-kube-api-access-bzztd" (OuterVolumeSpecName: "kube-api-access-bzztd") pod "acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89" (UID: "acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89"). InnerVolumeSpecName "kube-api-access-bzztd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:33:20.651698 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.651671    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vssr6\" (UniqueName: \"kubernetes.io/projected/5cb8f71d-990b-4ee1-b1e3-18931667fdac-kube-api-access-vssr6\") pod \"limitador-operator-controller-manager-6c549b6d85-v9msv\" (UID: \"5cb8f71d-990b-4ee1-b1e3-18931667fdac\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-v9msv"
Sep 05 06:33:20.653101 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.653081    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"
Sep 05 06:33:20.655264 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.655246    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"
Sep 05 06:33:20.655475 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.655446    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-ct7x5\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="6bc982ec-936c-4930-85ee-83504b0903f3" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"
Sep 05 06:33:20.657479 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.657450    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-mx2s2\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:20.743801 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.743751    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8s6s\" (UniqueName: \"kubernetes.io/projected/6bc982ec-936c-4930-85ee-83504b0903f3-kube-api-access-l8s6s\") pod \"6bc982ec-936c-4930-85ee-83504b0903f3\" (UID: \"6bc982ec-936c-4930-85ee-83504b0903f3\") "
Sep 05 06:33:20.744036 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.743996    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzztd\" (UniqueName: \"kubernetes.io/projected/acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89-kube-api-access-bzztd\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:33:20.744036 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.744010    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89-extensions-socket-volume\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:33:20.747120 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.747084    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bc982ec-936c-4930-85ee-83504b0903f3-kube-api-access-l8s6s" (OuterVolumeSpecName: "kube-api-access-l8s6s") pod "6bc982ec-936c-4930-85ee-83504b0903f3" (UID: "6bc982ec-936c-4930-85ee-83504b0903f3"). InnerVolumeSpecName "kube-api-access-l8s6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:33:20.766291 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.766258    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-v9msv"
Sep 05 06:33:20.844429 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.844390    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8s6s\" (UniqueName: \"kubernetes.io/projected/6bc982ec-936c-4930-85ee-83504b0903f3-kube-api-access-l8s6s\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:33:20.930251 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.930202    2559 generic.go:356] "Generic (PLEG): container finished" podID="acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89" containerID="2bd4c88e5deefb33f842cc043c58d7b4f4792a2a7455bdc8b1e2cb34637b657e" exitCode=0
Sep 05 06:33:20.930251 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.930260    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:20.930796 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.930315    2559 scope.go:117] "RemoveContainer" containerID="2bd4c88e5deefb33f842cc043c58d7b4f4792a2a7455bdc8b1e2cb34637b657e"
Sep 05 06:33:20.932190 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.932160    2559 generic.go:356] "Generic (PLEG): container finished" podID="6bc982ec-936c-4930-85ee-83504b0903f3" containerID="68a024743af1f92851e9f85538d93b7f6c16f4861177cf3c17637de51757555f" exitCode=0
Sep 05 06:33:20.932332 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.932215    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"
Sep 05 06:33:20.932564 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.932534    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"
Sep 05 06:33:20.932697 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.932669    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-mx2s2\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:20.934969 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.934939    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-ct7x5\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="6bc982ec-936c-4930-85ee-83504b0903f3" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"
Sep 05 06:33:20.937156 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.937123    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-mx2s2\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:20.939031 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.939000    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-ct7x5\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="6bc982ec-936c-4930-85ee-83504b0903f3" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"
Sep 05 06:33:20.941120 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.941079    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-pwv9l\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="d00c82d9-79da-498e-9070-9d1ab1eccf0c" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"
Sep 05 06:33:20.941189 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.941156    2559 scope.go:117] "RemoveContainer" containerID="2bd4c88e5deefb33f842cc043c58d7b4f4792a2a7455bdc8b1e2cb34637b657e"
Sep 05 06:33:20.941573 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.941545    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bd4c88e5deefb33f842cc043c58d7b4f4792a2a7455bdc8b1e2cb34637b657e\": container with ID starting with 2bd4c88e5deefb33f842cc043c58d7b4f4792a2a7455bdc8b1e2cb34637b657e not found: ID does not exist" containerID="2bd4c88e5deefb33f842cc043c58d7b4f4792a2a7455bdc8b1e2cb34637b657e"
Sep 05 06:33:20.941666 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.941579    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bd4c88e5deefb33f842cc043c58d7b4f4792a2a7455bdc8b1e2cb34637b657e"} err="failed to get container status \"2bd4c88e5deefb33f842cc043c58d7b4f4792a2a7455bdc8b1e2cb34637b657e\": rpc error: code = NotFound desc = could not find container \"2bd4c88e5deefb33f842cc043c58d7b4f4792a2a7455bdc8b1e2cb34637b657e\": container with ID starting with 2bd4c88e5deefb33f842cc043c58d7b4f4792a2a7455bdc8b1e2cb34637b657e not found: ID does not exist"
Sep 05 06:33:20.941666 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.941600    2559 scope.go:117] "RemoveContainer" containerID="68a024743af1f92851e9f85538d93b7f6c16f4861177cf3c17637de51757555f"
Sep 05 06:33:20.946268 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.946242    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-mx2s2\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:20.948318 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.948290    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-ct7x5\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="6bc982ec-936c-4930-85ee-83504b0903f3" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"
Sep 05 06:33:20.950354 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.950329    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-pwv9l\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="d00c82d9-79da-498e-9070-9d1ab1eccf0c" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"
Sep 05 06:33:20.951151 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.951126    2559 scope.go:117] "RemoveContainer" containerID="68a024743af1f92851e9f85538d93b7f6c16f4861177cf3c17637de51757555f"
Sep 05 06:33:20.951449 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.951431    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68a024743af1f92851e9f85538d93b7f6c16f4861177cf3c17637de51757555f\": container with ID starting with 68a024743af1f92851e9f85538d93b7f6c16f4861177cf3c17637de51757555f not found: ID does not exist" containerID="68a024743af1f92851e9f85538d93b7f6c16f4861177cf3c17637de51757555f"
Sep 05 06:33:20.951504 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.951461    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68a024743af1f92851e9f85538d93b7f6c16f4861177cf3c17637de51757555f"} err="failed to get container status \"68a024743af1f92851e9f85538d93b7f6c16f4861177cf3c17637de51757555f\": rpc error: code = NotFound desc = could not find container \"68a024743af1f92851e9f85538d93b7f6c16f4861177cf3c17637de51757555f\": container with ID starting with 68a024743af1f92851e9f85538d93b7f6c16f4861177cf3c17637de51757555f not found: ID does not exist"
Sep 05 06:33:20.953579 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.953498    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-ct7x5\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="6bc982ec-936c-4930-85ee-83504b0903f3" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-ct7x5"
Sep 05 06:33:20.955587 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.955561    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-pwv9l\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="d00c82d9-79da-498e-9070-9d1ab1eccf0c" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-pwv9l"
Sep 05 06:33:20.955762 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:20.955742    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"]
Sep 05 06:33:20.957451 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:20.957429    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-mx2s2\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-mx2s2"
Sep 05 06:33:20.958283 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:33:20.958265    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b36e819_3a44_439f_be31_91578bdcba05.slice/crio-c97c71cef951ec87036d12570a22028d0ab6f4dd14b1778a2d7d024acfbaa230 WatchSource:0}: Error finding container c97c71cef951ec87036d12570a22028d0ab6f4dd14b1778a2d7d024acfbaa230: Status 404 returned error can't find the container with id c97c71cef951ec87036d12570a22028d0ab6f4dd14b1778a2d7d024acfbaa230
Sep 05 06:33:21.050580 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:21.050550    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-v9msv"]
Sep 05 06:33:21.053188 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:33:21.053146    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb8f71d_990b_4ee1_b1e3_18931667fdac.slice/crio-b8cd64c07fd4618ba1e558f2e3c79fa54f7e47f7c19e5f11e81e1ea988a5911a WatchSource:0}: Error finding container b8cd64c07fd4618ba1e558f2e3c79fa54f7e47f7c19e5f11e81e1ea988a5911a: Status 404 returned error can't find the container with id b8cd64c07fd4618ba1e558f2e3c79fa54f7e47f7c19e5f11e81e1ea988a5911a
Sep 05 06:33:21.891807 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:21.891715    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bc982ec-936c-4930-85ee-83504b0903f3" path="/var/lib/kubelet/pods/6bc982ec-936c-4930-85ee-83504b0903f3/volumes"
Sep 05 06:33:21.892124 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:21.892110    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89" path="/var/lib/kubelet/pods/acac11fa-9ae6-4fb8-b0cd-bc55df3e5b89/volumes"
Sep 05 06:33:21.892416 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:21.892405    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d00c82d9-79da-498e-9070-9d1ab1eccf0c" path="/var/lib/kubelet/pods/d00c82d9-79da-498e-9070-9d1ab1eccf0c/volumes"
Sep 05 06:33:21.940052 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:21.940009    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh" event={"ID":"8b36e819-3a44-439f-be31-91578bdcba05","Type":"ContainerStarted","Data":"da35a9c10dd89068f91ddd8b3d487f1133d06b4bc45ac07f2a973fbbe342a5e4"}
Sep 05 06:33:21.940526 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:21.940073    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh" event={"ID":"8b36e819-3a44-439f-be31-91578bdcba05","Type":"ContainerStarted","Data":"c97c71cef951ec87036d12570a22028d0ab6f4dd14b1778a2d7d024acfbaa230"}
Sep 05 06:33:21.940526 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:21.940105    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"
Sep 05 06:33:21.944475 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:21.944445    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-v9msv" event={"ID":"5cb8f71d-990b-4ee1-b1e3-18931667fdac","Type":"ContainerStarted","Data":"a6a20fda7a6feb61f69f7e641368e3f5ac139e36756507d859ddf4d027f3ed87"}
Sep 05 06:33:21.944605 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:21.944491    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-v9msv" event={"ID":"5cb8f71d-990b-4ee1-b1e3-18931667fdac","Type":"ContainerStarted","Data":"b8cd64c07fd4618ba1e558f2e3c79fa54f7e47f7c19e5f11e81e1ea988a5911a"}
Sep 05 06:33:21.944605 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:21.944587    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-v9msv"
Sep 05 06:33:21.963498 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:21.963446    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh" podStartSLOduration=1.9634277949999999 podStartE2EDuration="1.963427795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:33:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:33:21.962459109 +0000 UTC m=+418.553018061" watchObservedRunningTime="2026-09-05 06:33:21.963427795 +0000 UTC m=+418.553986747"
Sep 05 06:33:21.989218 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:21.989155    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-v9msv" podStartSLOduration=1.989136361 podStartE2EDuration="1.989136361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:33:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:33:21.987863919 +0000 UTC m=+418.578422871" watchObservedRunningTime="2026-09-05 06:33:21.989136361 +0000 UTC m=+418.579695312"
Sep 05 06:33:25.266421 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.266375    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cdcd47765-kxwxl"]
Sep 05 06:33:25.271327 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.271299    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.274614 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.274588    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 05 06:33:25.275090 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.275073    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 05 06:33:25.275146 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.275084    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 05 06:33:25.275146 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.275107    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7r4f6\""
Sep 05 06:33:25.275596 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.275578    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 05 06:33:25.275845 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.275828    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 05 06:33:25.275921 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.275857    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 05 06:33:25.275921 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.275871    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 05 06:33:25.281654 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.281631    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 05 06:33:25.290560 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.290533    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cdcd47765-kxwxl"]
Sep 05 06:33:25.386929 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.386837    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-console-config\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.387137 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.386982    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-console-oauth-config\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.387137 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.387044    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-console-serving-cert\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.387137 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.387079    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-trusted-ca-bundle\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.387137 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.387117    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-oauth-serving-cert\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.387137 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.387135    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd6zv\" (UniqueName: \"kubernetes.io/projected/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-kube-api-access-dd6zv\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.387293 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.387200    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-service-ca\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.488470 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.488405    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-console-oauth-config\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.488470 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.488468    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-console-serving-cert\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.488716 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.488592    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-trusted-ca-bundle\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.488716 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.488663    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-oauth-serving-cert\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.488716 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.488687    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd6zv\" (UniqueName: \"kubernetes.io/projected/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-kube-api-access-dd6zv\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.488857 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.488738    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-service-ca\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.488857 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.488792    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-console-config\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.489680 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.489643    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-oauth-serving-cert\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.489828 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.489697    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-service-ca\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.489921 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.489858    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-console-config\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.489986 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.489915    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-trusted-ca-bundle\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.491335 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.491309    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-console-serving-cert\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.491441 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.491339    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-console-oauth-config\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.498949 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.498918    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd6zv\" (UniqueName: \"kubernetes.io/projected/6c86a979-0d7e-4cc2-a9e9-914f7c906c8b-kube-api-access-dd6zv\") pod \"console-5cdcd47765-kxwxl\" (UID: \"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b\") " pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.583088 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.582960    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:25.868683 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.868651    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cdcd47765-kxwxl"]
Sep 05 06:33:25.870728 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:33:25.870702    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c86a979_0d7e_4cc2_a9e9_914f7c906c8b.slice/crio-7e34003abb99a7870d5dd628f07338b8be888a9510e6a74887d4bfe0a50b9665 WatchSource:0}: Error finding container 7e34003abb99a7870d5dd628f07338b8be888a9510e6a74887d4bfe0a50b9665: Status 404 returned error can't find the container with id 7e34003abb99a7870d5dd628f07338b8be888a9510e6a74887d4bfe0a50b9665
Sep 05 06:33:25.969106 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.969066    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cdcd47765-kxwxl" event={"ID":"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b","Type":"ContainerStarted","Data":"a6ef941af38746d733c9977576b75350b4566a9c6d27f97e47465547d3e9ec0c"}
Sep 05 06:33:25.969333 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.969125    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cdcd47765-kxwxl" event={"ID":"6c86a979-0d7e-4cc2-a9e9-914f7c906c8b","Type":"ContainerStarted","Data":"7e34003abb99a7870d5dd628f07338b8be888a9510e6a74887d4bfe0a50b9665"}
Sep 05 06:33:25.991866 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:25.991797    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cdcd47765-kxwxl" podStartSLOduration=0.991777702 podStartE2EDuration="991.777702ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:33:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:33:25.990341633 +0000 UTC m=+422.580900587" watchObservedRunningTime="2026-09-05 06:33:25.991777702 +0000 UTC m=+422.582336653"
Sep 05 06:33:32.953359 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:32.953316    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"
Sep 05 06:33:32.953788 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:32.953511    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-v9msv"
Sep 05 06:33:35.584018 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:35.583971    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:35.584445 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:35.584046    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:35.589355 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:35.589324    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:36.026512 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.026478    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cdcd47765-kxwxl"
Sep 05 06:33:36.344838 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.344735    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"]
Sep 05 06:33:36.345045 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.344977    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh" podUID="8b36e819-3a44-439f-be31-91578bdcba05" containerName="manager" containerID="cri-o://da35a9c10dd89068f91ddd8b3d487f1133d06b4bc45ac07f2a973fbbe342a5e4" gracePeriod=10
Sep 05 06:33:36.599405 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.599368    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"]
Sep 05 06:33:36.604561 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.604529    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"
Sep 05 06:33:36.616735 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.616698    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"]
Sep 05 06:33:36.633765 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.633731    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"
Sep 05 06:33:36.694372 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.694309    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a96bbb54-ef69-4974-b0e2-4c165c01e8ca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-nvfdg\" (UID: \"a96bbb54-ef69-4974-b0e2-4c165c01e8ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"
Sep 05 06:33:36.694372 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.694360    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8zvk\" (UniqueName: \"kubernetes.io/projected/a96bbb54-ef69-4974-b0e2-4c165c01e8ca-kube-api-access-z8zvk\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-nvfdg\" (UID: \"a96bbb54-ef69-4974-b0e2-4c165c01e8ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"
Sep 05 06:33:36.795491 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.795444    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvpjw\" (UniqueName: \"kubernetes.io/projected/8b36e819-3a44-439f-be31-91578bdcba05-kube-api-access-pvpjw\") pod \"8b36e819-3a44-439f-be31-91578bdcba05\" (UID: \"8b36e819-3a44-439f-be31-91578bdcba05\") "
Sep 05 06:33:36.795737 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.795522    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b36e819-3a44-439f-be31-91578bdcba05-extensions-socket-volume\") pod \"8b36e819-3a44-439f-be31-91578bdcba05\" (UID: \"8b36e819-3a44-439f-be31-91578bdcba05\") "
Sep 05 06:33:36.795737 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.795705    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a96bbb54-ef69-4974-b0e2-4c165c01e8ca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-nvfdg\" (UID: \"a96bbb54-ef69-4974-b0e2-4c165c01e8ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"
Sep 05 06:33:36.795737 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.795729    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8zvk\" (UniqueName: \"kubernetes.io/projected/a96bbb54-ef69-4974-b0e2-4c165c01e8ca-kube-api-access-z8zvk\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-nvfdg\" (UID: \"a96bbb54-ef69-4974-b0e2-4c165c01e8ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"
Sep 05 06:33:36.796061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.796028    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b36e819-3a44-439f-be31-91578bdcba05-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8b36e819-3a44-439f-be31-91578bdcba05" (UID: "8b36e819-3a44-439f-be31-91578bdcba05"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:33:36.796194 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.796174    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a96bbb54-ef69-4974-b0e2-4c165c01e8ca-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-nvfdg\" (UID: \"a96bbb54-ef69-4974-b0e2-4c165c01e8ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"
Sep 05 06:33:36.797855 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.797830    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b36e819-3a44-439f-be31-91578bdcba05-kube-api-access-pvpjw" (OuterVolumeSpecName: "kube-api-access-pvpjw") pod "8b36e819-3a44-439f-be31-91578bdcba05" (UID: "8b36e819-3a44-439f-be31-91578bdcba05"). InnerVolumeSpecName "kube-api-access-pvpjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:33:36.810714 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.810680    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8zvk\" (UniqueName: \"kubernetes.io/projected/a96bbb54-ef69-4974-b0e2-4c165c01e8ca-kube-api-access-z8zvk\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-nvfdg\" (UID: \"a96bbb54-ef69-4974-b0e2-4c165c01e8ca\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"
Sep 05 06:33:36.896338 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.896230    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvpjw\" (UniqueName: \"kubernetes.io/projected/8b36e819-3a44-439f-be31-91578bdcba05-kube-api-access-pvpjw\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:33:36.896338 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.896264    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b36e819-3a44-439f-be31-91578bdcba05-extensions-socket-volume\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:33:36.923932 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:36.923859    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"
Sep 05 06:33:37.030368 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:37.030320    2559 generic.go:356] "Generic (PLEG): container finished" podID="8b36e819-3a44-439f-be31-91578bdcba05" containerID="da35a9c10dd89068f91ddd8b3d487f1133d06b4bc45ac07f2a973fbbe342a5e4" exitCode=0
Sep 05 06:33:37.030559 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:37.030392    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"
Sep 05 06:33:37.030559 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:37.030407    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh" event={"ID":"8b36e819-3a44-439f-be31-91578bdcba05","Type":"ContainerDied","Data":"da35a9c10dd89068f91ddd8b3d487f1133d06b4bc45ac07f2a973fbbe342a5e4"}
Sep 05 06:33:37.030559 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:37.030475    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh" event={"ID":"8b36e819-3a44-439f-be31-91578bdcba05","Type":"ContainerDied","Data":"c97c71cef951ec87036d12570a22028d0ab6f4dd14b1778a2d7d024acfbaa230"}
Sep 05 06:33:37.030559 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:37.030507    2559 scope.go:117] "RemoveContainer" containerID="da35a9c10dd89068f91ddd8b3d487f1133d06b4bc45ac07f2a973fbbe342a5e4"
Sep 05 06:33:37.040698 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:37.040675    2559 scope.go:117] "RemoveContainer" containerID="da35a9c10dd89068f91ddd8b3d487f1133d06b4bc45ac07f2a973fbbe342a5e4"
Sep 05 06:33:37.041187 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:33:37.041161    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da35a9c10dd89068f91ddd8b3d487f1133d06b4bc45ac07f2a973fbbe342a5e4\": container with ID starting with da35a9c10dd89068f91ddd8b3d487f1133d06b4bc45ac07f2a973fbbe342a5e4 not found: ID does not exist" containerID="da35a9c10dd89068f91ddd8b3d487f1133d06b4bc45ac07f2a973fbbe342a5e4"
Sep 05 06:33:37.041290 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:37.041196    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da35a9c10dd89068f91ddd8b3d487f1133d06b4bc45ac07f2a973fbbe342a5e4"} err="failed to get container status \"da35a9c10dd89068f91ddd8b3d487f1133d06b4bc45ac07f2a973fbbe342a5e4\": rpc error: code = NotFound desc = could not find container \"da35a9c10dd89068f91ddd8b3d487f1133d06b4bc45ac07f2a973fbbe342a5e4\": container with ID starting with da35a9c10dd89068f91ddd8b3d487f1133d06b4bc45ac07f2a973fbbe342a5e4 not found: ID does not exist"
Sep 05 06:33:37.055107 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:37.055072    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"]
Sep 05 06:33:37.061433 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:37.061398    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-89pqh"]
Sep 05 06:33:37.224101 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:37.224068    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"]
Sep 05 06:33:37.226023 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:33:37.225993    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda96bbb54_ef69_4974_b0e2_4c165c01e8ca.slice/crio-78193c35a58b2e353708c96c57f94ce6d3f186e5ca3549ecbac1b9807987d493 WatchSource:0}: Error finding container 78193c35a58b2e353708c96c57f94ce6d3f186e5ca3549ecbac1b9807987d493: Status 404 returned error can't find the container with id 78193c35a58b2e353708c96c57f94ce6d3f186e5ca3549ecbac1b9807987d493
Sep 05 06:33:37.892343 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:37.892296    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b36e819-3a44-439f-be31-91578bdcba05" path="/var/lib/kubelet/pods/8b36e819-3a44-439f-be31-91578bdcba05/volumes"
Sep 05 06:33:38.038580 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:38.038540    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg" event={"ID":"a96bbb54-ef69-4974-b0e2-4c165c01e8ca","Type":"ContainerStarted","Data":"17ab8ac482e4475dfc7f022d75de2ae1a1f6065880cd5155862821d5ec7bb399"}
Sep 05 06:33:38.038823 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:38.038595    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg" event={"ID":"a96bbb54-ef69-4974-b0e2-4c165c01e8ca","Type":"ContainerStarted","Data":"78193c35a58b2e353708c96c57f94ce6d3f186e5ca3549ecbac1b9807987d493"}
Sep 05 06:33:38.038823 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:38.038681    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"
Sep 05 06:33:38.060374 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:38.060313    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg" podStartSLOduration=2.060294232 podStartE2EDuration="2.060294232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:33:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:33:38.059662761 +0000 UTC m=+434.650221713" watchObservedRunningTime="2026-09-05 06:33:38.060294232 +0000 UTC m=+434.650853204"
Sep 05 06:33:49.047314 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:33:49.047226    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"
Sep 05 06:34:07.141067 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.141021    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:34:07.141501 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.141350    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b36e819-3a44-439f-be31-91578bdcba05" containerName="manager"
Sep 05 06:34:07.141501 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.141361    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b36e819-3a44-439f-be31-91578bdcba05" containerName="manager"
Sep 05 06:34:07.141501 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.141431    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b36e819-3a44-439f-be31-91578bdcba05" containerName="manager"
Sep 05 06:34:07.144032 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.144007    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"
Sep 05 06:34:07.148102 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.148079    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qckkf\""
Sep 05 06:34:07.148377 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.148357    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 05 06:34:07.163671 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.163629    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:34:07.240594 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.240556    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:34:07.254087 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.254055    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2e362e88-5eb3-4c01-ad5c-5978374f14a4-config-file\") pod \"limitador-limitador-85d9db4d5d-vn7br\" (UID: \"2e362e88-5eb3-4c01-ad5c-5978374f14a4\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"
Sep 05 06:34:07.254266 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.254116    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t9cs\" (UniqueName: \"kubernetes.io/projected/2e362e88-5eb3-4c01-ad5c-5978374f14a4-kube-api-access-5t9cs\") pod \"limitador-limitador-85d9db4d5d-vn7br\" (UID: \"2e362e88-5eb3-4c01-ad5c-5978374f14a4\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"
Sep 05 06:34:07.355576 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.355531    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t9cs\" (UniqueName: \"kubernetes.io/projected/2e362e88-5eb3-4c01-ad5c-5978374f14a4-kube-api-access-5t9cs\") pod \"limitador-limitador-85d9db4d5d-vn7br\" (UID: \"2e362e88-5eb3-4c01-ad5c-5978374f14a4\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"
Sep 05 06:34:07.355815 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.355638    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2e362e88-5eb3-4c01-ad5c-5978374f14a4-config-file\") pod \"limitador-limitador-85d9db4d5d-vn7br\" (UID: \"2e362e88-5eb3-4c01-ad5c-5978374f14a4\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"
Sep 05 06:34:07.356342 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.356321    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2e362e88-5eb3-4c01-ad5c-5978374f14a4-config-file\") pod \"limitador-limitador-85d9db4d5d-vn7br\" (UID: \"2e362e88-5eb3-4c01-ad5c-5978374f14a4\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"
Sep 05 06:34:07.372192 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.372160    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t9cs\" (UniqueName: \"kubernetes.io/projected/2e362e88-5eb3-4c01-ad5c-5978374f14a4-kube-api-access-5t9cs\") pod \"limitador-limitador-85d9db4d5d-vn7br\" (UID: \"2e362e88-5eb3-4c01-ad5c-5978374f14a4\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"
Sep 05 06:34:07.455378 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.455336    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"
Sep 05 06:34:07.577386 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.577349    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-nxqg8"]
Sep 05 06:34:07.579982 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.579960    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-nxqg8"
Sep 05 06:34:07.583113 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.583090    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nz49m\""
Sep 05 06:34:07.591611 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.591567    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-nxqg8"]
Sep 05 06:34:07.658091 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.658053    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmjr7\" (UniqueName: \"kubernetes.io/projected/94a8e4ed-8748-4c84-8c41-dbf996494c12-kube-api-access-pmjr7\") pod \"authorino-b54df94b4-nxqg8\" (UID: \"94a8e4ed-8748-4c84-8c41-dbf996494c12\") " pod="kuadrant-system/authorino-b54df94b4-nxqg8"
Sep 05 06:34:07.724834 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.724756    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xvfx9"]
Sep 05 06:34:07.727514 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.727488    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-xvfx9"
Sep 05 06:34:07.734325 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.734201    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:34:07.737082 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:34:07.737052    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e362e88_5eb3_4c01_ad5c_5978374f14a4.slice/crio-ff3f4ab21ea6b177c0a1ad13825a47b51c51453ed30a3bed8406d20a19a5b5e4 WatchSource:0}: Error finding container ff3f4ab21ea6b177c0a1ad13825a47b51c51453ed30a3bed8406d20a19a5b5e4: Status 404 returned error can't find the container with id ff3f4ab21ea6b177c0a1ad13825a47b51c51453ed30a3bed8406d20a19a5b5e4
Sep 05 06:34:07.737539 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.737516    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xvfx9"]
Sep 05 06:34:07.758895 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.758846    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmjr7\" (UniqueName: \"kubernetes.io/projected/94a8e4ed-8748-4c84-8c41-dbf996494c12-kube-api-access-pmjr7\") pod \"authorino-b54df94b4-nxqg8\" (UID: \"94a8e4ed-8748-4c84-8c41-dbf996494c12\") " pod="kuadrant-system/authorino-b54df94b4-nxqg8"
Sep 05 06:34:07.768817 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.768777    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmjr7\" (UniqueName: \"kubernetes.io/projected/94a8e4ed-8748-4c84-8c41-dbf996494c12-kube-api-access-pmjr7\") pod \"authorino-b54df94b4-nxqg8\" (UID: \"94a8e4ed-8748-4c84-8c41-dbf996494c12\") " pod="kuadrant-system/authorino-b54df94b4-nxqg8"
Sep 05 06:34:07.859692 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.859648    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dldxr\" (UniqueName: \"kubernetes.io/projected/1ad4c083-36a8-4111-9a5d-82ba4cf950a0-kube-api-access-dldxr\") pod \"authorino-c548b5d7d-xvfx9\" (UID: \"1ad4c083-36a8-4111-9a5d-82ba4cf950a0\") " pod="kuadrant-system/authorino-c548b5d7d-xvfx9"
Sep 05 06:34:07.890579 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.890544    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-nxqg8"
Sep 05 06:34:07.960459 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.960426    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dldxr\" (UniqueName: \"kubernetes.io/projected/1ad4c083-36a8-4111-9a5d-82ba4cf950a0-kube-api-access-dldxr\") pod \"authorino-c548b5d7d-xvfx9\" (UID: \"1ad4c083-36a8-4111-9a5d-82ba4cf950a0\") " pod="kuadrant-system/authorino-c548b5d7d-xvfx9"
Sep 05 06:34:07.970399 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:07.970371    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dldxr\" (UniqueName: \"kubernetes.io/projected/1ad4c083-36a8-4111-9a5d-82ba4cf950a0-kube-api-access-dldxr\") pod \"authorino-c548b5d7d-xvfx9\" (UID: \"1ad4c083-36a8-4111-9a5d-82ba4cf950a0\") " pod="kuadrant-system/authorino-c548b5d7d-xvfx9"
Sep 05 06:34:08.046053 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:08.045961    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-xvfx9"
Sep 05 06:34:08.165268 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:08.165240    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-nxqg8"]
Sep 05 06:34:08.166825 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:34:08.166796    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94a8e4ed_8748_4c84_8c41_dbf996494c12.slice/crio-24998bb790794f531c3add8d036de4f31e8fd3c8e632a3ee0023872b7b42802f WatchSource:0}: Error finding container 24998bb790794f531c3add8d036de4f31e8fd3c8e632a3ee0023872b7b42802f: Status 404 returned error can't find the container with id 24998bb790794f531c3add8d036de4f31e8fd3c8e632a3ee0023872b7b42802f
Sep 05 06:34:08.202048 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:08.202004    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-nxqg8" event={"ID":"94a8e4ed-8748-4c84-8c41-dbf996494c12","Type":"ContainerStarted","Data":"24998bb790794f531c3add8d036de4f31e8fd3c8e632a3ee0023872b7b42802f"}
Sep 05 06:34:08.203316 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:08.203293    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-vn7br" event={"ID":"2e362e88-5eb3-4c01-ad5c-5978374f14a4","Type":"ContainerStarted","Data":"ff3f4ab21ea6b177c0a1ad13825a47b51c51453ed30a3bed8406d20a19a5b5e4"}
Sep 05 06:34:08.318713 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:08.318679    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xvfx9"]
Sep 05 06:34:08.320308 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:34:08.320279    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ad4c083_36a8_4111_9a5d_82ba4cf950a0.slice/crio-cbbd3c5a83fa1a65e2738c7d810a8519c650e14a2f8a17e77adaee0df939d914 WatchSource:0}: Error finding container cbbd3c5a83fa1a65e2738c7d810a8519c650e14a2f8a17e77adaee0df939d914: Status 404 returned error can't find the container with id cbbd3c5a83fa1a65e2738c7d810a8519c650e14a2f8a17e77adaee0df939d914
Sep 05 06:34:09.214843 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:09.214799    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-vn7br" event={"ID":"2e362e88-5eb3-4c01-ad5c-5978374f14a4","Type":"ContainerStarted","Data":"256fca059bfe2e7d47796e88b39dec102632b9d86477dcd3c4655d7ac0be231f"}
Sep 05 06:34:09.215386 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:09.215079    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"
Sep 05 06:34:09.216998 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:09.216968    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-xvfx9" event={"ID":"1ad4c083-36a8-4111-9a5d-82ba4cf950a0","Type":"ContainerStarted","Data":"cbbd3c5a83fa1a65e2738c7d810a8519c650e14a2f8a17e77adaee0df939d914"}
Sep 05 06:34:09.237782 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:09.237714    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-vn7br" podStartSLOduration=1.114364145 podStartE2EDuration="2.23769143s" totalImagesPullingTime="1.123327285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:34:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:34:09.236054108 +0000 UTC m=+465.826613068" watchObservedRunningTime="2026-09-05 06:34:09.23769143 +0000 UTC m=+465.828250383"
Sep 05 06:34:10.223979 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:10.223849    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-xvfx9" event={"ID":"1ad4c083-36a8-4111-9a5d-82ba4cf950a0","Type":"ContainerStarted","Data":"5a5bc93d44f534d62cbafe7f314378518f234180dde9ca6e629896a640a65339"}
Sep 05 06:34:10.225614 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:10.225585    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-nxqg8" event={"ID":"94a8e4ed-8748-4c84-8c41-dbf996494c12","Type":"ContainerStarted","Data":"3b1a7cbe05470e2f8c76fbffd6a51200954ed586686262f6f2ef09e60d9b01a4"}
Sep 05 06:34:10.243777 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:10.243720    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-xvfx9" podStartSLOduration=1.592701771 podStartE2EDuration="3.24370279s" totalImagesPullingTime="1.651001019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:34:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:34:10.242260799 +0000 UTC m=+466.832819789" watchObservedRunningTime="2026-09-05 06:34:10.24370279 +0000 UTC m=+466.834261758"
Sep 05 06:34:10.259170 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:10.259114    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-nxqg8" podStartSLOduration=1.461223119 podStartE2EDuration="3.259096314s" totalImagesPullingTime="1.797873195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:34:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:34:10.259042474 +0000 UTC m=+466.849601452" watchObservedRunningTime="2026-09-05 06:34:10.259096314 +0000 UTC m=+466.849655335"
Sep 05 06:34:10.288092 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:10.288017    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-nxqg8"]
Sep 05 06:34:12.236229 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:12.236192    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-nxqg8" podUID="94a8e4ed-8748-4c84-8c41-dbf996494c12" containerName="authorino" containerID="cri-o://3b1a7cbe05470e2f8c76fbffd6a51200954ed586686262f6f2ef09e60d9b01a4" gracePeriod=30
Sep 05 06:34:12.492339 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:12.492271    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-nxqg8"
Sep 05 06:34:12.601605 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:12.601570    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmjr7\" (UniqueName: \"kubernetes.io/projected/94a8e4ed-8748-4c84-8c41-dbf996494c12-kube-api-access-pmjr7\") pod \"94a8e4ed-8748-4c84-8c41-dbf996494c12\" (UID: \"94a8e4ed-8748-4c84-8c41-dbf996494c12\") "
Sep 05 06:34:12.603694 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:12.603662    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94a8e4ed-8748-4c84-8c41-dbf996494c12-kube-api-access-pmjr7" (OuterVolumeSpecName: "kube-api-access-pmjr7") pod "94a8e4ed-8748-4c84-8c41-dbf996494c12" (UID: "94a8e4ed-8748-4c84-8c41-dbf996494c12"). InnerVolumeSpecName "kube-api-access-pmjr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:34:12.702407 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:12.702362    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmjr7\" (UniqueName: \"kubernetes.io/projected/94a8e4ed-8748-4c84-8c41-dbf996494c12-kube-api-access-pmjr7\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:34:13.243083 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:13.243042    2559 generic.go:356] "Generic (PLEG): container finished" podID="94a8e4ed-8748-4c84-8c41-dbf996494c12" containerID="3b1a7cbe05470e2f8c76fbffd6a51200954ed586686262f6f2ef09e60d9b01a4" exitCode=0
Sep 05 06:34:13.243083 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:13.243089    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-nxqg8"
Sep 05 06:34:13.243575 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:13.243186    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-nxqg8" event={"ID":"94a8e4ed-8748-4c84-8c41-dbf996494c12","Type":"ContainerDied","Data":"3b1a7cbe05470e2f8c76fbffd6a51200954ed586686262f6f2ef09e60d9b01a4"}
Sep 05 06:34:13.243575 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:13.243231    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-nxqg8" event={"ID":"94a8e4ed-8748-4c84-8c41-dbf996494c12","Type":"ContainerDied","Data":"24998bb790794f531c3add8d036de4f31e8fd3c8e632a3ee0023872b7b42802f"}
Sep 05 06:34:13.243575 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:13.243249    2559 scope.go:117] "RemoveContainer" containerID="3b1a7cbe05470e2f8c76fbffd6a51200954ed586686262f6f2ef09e60d9b01a4"
Sep 05 06:34:13.252792 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:13.252763    2559 scope.go:117] "RemoveContainer" containerID="3b1a7cbe05470e2f8c76fbffd6a51200954ed586686262f6f2ef09e60d9b01a4"
Sep 05 06:34:13.253122 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:34:13.253101    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b1a7cbe05470e2f8c76fbffd6a51200954ed586686262f6f2ef09e60d9b01a4\": container with ID starting with 3b1a7cbe05470e2f8c76fbffd6a51200954ed586686262f6f2ef09e60d9b01a4 not found: ID does not exist" containerID="3b1a7cbe05470e2f8c76fbffd6a51200954ed586686262f6f2ef09e60d9b01a4"
Sep 05 06:34:13.253179 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:13.253132    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b1a7cbe05470e2f8c76fbffd6a51200954ed586686262f6f2ef09e60d9b01a4"} err="failed to get container status \"3b1a7cbe05470e2f8c76fbffd6a51200954ed586686262f6f2ef09e60d9b01a4\": rpc error: code = NotFound desc = could not find container \"3b1a7cbe05470e2f8c76fbffd6a51200954ed586686262f6f2ef09e60d9b01a4\": container with ID starting with 3b1a7cbe05470e2f8c76fbffd6a51200954ed586686262f6f2ef09e60d9b01a4 not found: ID does not exist"
Sep 05 06:34:13.267588 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:13.267559    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-nxqg8"]
Sep 05 06:34:13.272632 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:13.272600    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-nxqg8"]
Sep 05 06:34:13.892488 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:13.892456    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94a8e4ed-8748-4c84-8c41-dbf996494c12" path="/var/lib/kubelet/pods/94a8e4ed-8748-4c84-8c41-dbf996494c12/volumes"
Sep 05 06:34:20.227082 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:20.227036    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"
Sep 05 06:34:35.250176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.250135    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-zk8v5"]
Sep 05 06:34:35.250686 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.250489    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94a8e4ed-8748-4c84-8c41-dbf996494c12" containerName="authorino"
Sep 05 06:34:35.250686 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.250500    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="94a8e4ed-8748-4c84-8c41-dbf996494c12" containerName="authorino"
Sep 05 06:34:35.250686 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.250577    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="94a8e4ed-8748-4c84-8c41-dbf996494c12" containerName="authorino"
Sep 05 06:34:35.253099 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.253076    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-zk8v5"
Sep 05 06:34:35.262463 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.262430    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-zk8v5"]
Sep 05 06:34:35.288779 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.288743    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-zk8v5"]
Sep 05 06:34:35.289063 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:34:35.289024    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-fjbzq], unattached volumes=[], failed to process volumes=[kube-api-access-fjbzq]: context canceled" pod="kuadrant-system/authorino-5dc7ccb5f-zk8v5" podUID="93f3a94e-70e1-4f10-9f54-6b14a3fdc06a"
Sep 05 06:34:35.345855 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.345806    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-687bd45585-cqx78"]
Sep 05 06:34:35.348843 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.348813    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-cqx78"
Sep 05 06:34:35.353384 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.353355    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 05 06:34:35.371419 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.371384    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-zk8v5"
Sep 05 06:34:35.372975 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.372940    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-687bd45585-cqx78"]
Sep 05 06:34:35.382581 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.382552    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-zk8v5"
Sep 05 06:34:35.400141 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.400112    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjbzq\" (UniqueName: \"kubernetes.io/projected/93f3a94e-70e1-4f10-9f54-6b14a3fdc06a-kube-api-access-fjbzq\") pod \"authorino-5dc7ccb5f-zk8v5\" (UID: \"93f3a94e-70e1-4f10-9f54-6b14a3fdc06a\") " pod="kuadrant-system/authorino-5dc7ccb5f-zk8v5"
Sep 05 06:34:35.501547 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.501452    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjbzq\" (UniqueName: \"kubernetes.io/projected/93f3a94e-70e1-4f10-9f54-6b14a3fdc06a-kube-api-access-fjbzq\") pod \"authorino-5dc7ccb5f-zk8v5\" (UID: \"93f3a94e-70e1-4f10-9f54-6b14a3fdc06a\") " pod="kuadrant-system/authorino-5dc7ccb5f-zk8v5"
Sep 05 06:34:35.501725 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.501541    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dslb\" (UniqueName: \"kubernetes.io/projected/f6d44f65-01a9-48e2-9826-227d40152b7a-kube-api-access-6dslb\") pod \"authorino-687bd45585-cqx78\" (UID: \"f6d44f65-01a9-48e2-9826-227d40152b7a\") " pod="kuadrant-system/authorino-687bd45585-cqx78"
Sep 05 06:34:35.501725 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.501580    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6d44f65-01a9-48e2-9826-227d40152b7a-tls-cert\") pod \"authorino-687bd45585-cqx78\" (UID: \"f6d44f65-01a9-48e2-9826-227d40152b7a\") " pod="kuadrant-system/authorino-687bd45585-cqx78"
Sep 05 06:34:35.512262 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.512227    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjbzq\" (UniqueName: \"kubernetes.io/projected/93f3a94e-70e1-4f10-9f54-6b14a3fdc06a-kube-api-access-fjbzq\") pod \"authorino-5dc7ccb5f-zk8v5\" (UID: \"93f3a94e-70e1-4f10-9f54-6b14a3fdc06a\") " pod="kuadrant-system/authorino-5dc7ccb5f-zk8v5"
Sep 05 06:34:35.552114 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.552075    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-cqx78"]
Sep 05 06:34:35.552371 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:34:35.552351    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-6dslb tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-687bd45585-cqx78" podUID="f6d44f65-01a9-48e2-9826-227d40152b7a"
Sep 05 06:34:35.602636 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.602598    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dslb\" (UniqueName: \"kubernetes.io/projected/f6d44f65-01a9-48e2-9826-227d40152b7a-kube-api-access-6dslb\") pod \"authorino-687bd45585-cqx78\" (UID: \"f6d44f65-01a9-48e2-9826-227d40152b7a\") " pod="kuadrant-system/authorino-687bd45585-cqx78"
Sep 05 06:34:35.602636 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.602645    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6d44f65-01a9-48e2-9826-227d40152b7a-tls-cert\") pod \"authorino-687bd45585-cqx78\" (UID: \"f6d44f65-01a9-48e2-9826-227d40152b7a\") " pod="kuadrant-system/authorino-687bd45585-cqx78"
Sep 05 06:34:35.606234 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.606193    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6b64d87cdc-8dk7q"]
Sep 05 06:34:35.609897 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.609847    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b64d87cdc-8dk7q"
Sep 05 06:34:35.612997 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.612969    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6d44f65-01a9-48e2-9826-227d40152b7a-tls-cert\") pod \"authorino-687bd45585-cqx78\" (UID: \"f6d44f65-01a9-48e2-9826-227d40152b7a\") " pod="kuadrant-system/authorino-687bd45585-cqx78"
Sep 05 06:34:35.620394 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.620364    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dslb\" (UniqueName: \"kubernetes.io/projected/f6d44f65-01a9-48e2-9826-227d40152b7a-kube-api-access-6dslb\") pod \"authorino-687bd45585-cqx78\" (UID: \"f6d44f65-01a9-48e2-9826-227d40152b7a\") " pod="kuadrant-system/authorino-687bd45585-cqx78"
Sep 05 06:34:35.622241 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.622213    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b64d87cdc-8dk7q"]
Sep 05 06:34:35.703298 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.703261    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjbzq\" (UniqueName: \"kubernetes.io/projected/93f3a94e-70e1-4f10-9f54-6b14a3fdc06a-kube-api-access-fjbzq\") pod \"93f3a94e-70e1-4f10-9f54-6b14a3fdc06a\" (UID: \"93f3a94e-70e1-4f10-9f54-6b14a3fdc06a\") "
Sep 05 06:34:35.705496 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.705463    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93f3a94e-70e1-4f10-9f54-6b14a3fdc06a-kube-api-access-fjbzq" (OuterVolumeSpecName: "kube-api-access-fjbzq") pod "93f3a94e-70e1-4f10-9f54-6b14a3fdc06a" (UID: "93f3a94e-70e1-4f10-9f54-6b14a3fdc06a"). InnerVolumeSpecName "kube-api-access-fjbzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:34:35.804031 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.803879    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f1eb5f04-d170-4a47-a67e-adcfe4e15f52-tls-cert\") pod \"authorino-6b64d87cdc-8dk7q\" (UID: \"f1eb5f04-d170-4a47-a67e-adcfe4e15f52\") " pod="kuadrant-system/authorino-6b64d87cdc-8dk7q"
Sep 05 06:34:35.804031 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.803952    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gck7q\" (UniqueName: \"kubernetes.io/projected/f1eb5f04-d170-4a47-a67e-adcfe4e15f52-kube-api-access-gck7q\") pod \"authorino-6b64d87cdc-8dk7q\" (UID: \"f1eb5f04-d170-4a47-a67e-adcfe4e15f52\") " pod="kuadrant-system/authorino-6b64d87cdc-8dk7q"
Sep 05 06:34:35.804234 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.804060    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjbzq\" (UniqueName: \"kubernetes.io/projected/93f3a94e-70e1-4f10-9f54-6b14a3fdc06a-kube-api-access-fjbzq\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:34:35.905196 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.905159    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f1eb5f04-d170-4a47-a67e-adcfe4e15f52-tls-cert\") pod \"authorino-6b64d87cdc-8dk7q\" (UID: \"f1eb5f04-d170-4a47-a67e-adcfe4e15f52\") " pod="kuadrant-system/authorino-6b64d87cdc-8dk7q"
Sep 05 06:34:35.905358 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.905202    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gck7q\" (UniqueName: \"kubernetes.io/projected/f1eb5f04-d170-4a47-a67e-adcfe4e15f52-kube-api-access-gck7q\") pod \"authorino-6b64d87cdc-8dk7q\" (UID: \"f1eb5f04-d170-4a47-a67e-adcfe4e15f52\") " pod="kuadrant-system/authorino-6b64d87cdc-8dk7q"
Sep 05 06:34:35.907862 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.907837    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f1eb5f04-d170-4a47-a67e-adcfe4e15f52-tls-cert\") pod \"authorino-6b64d87cdc-8dk7q\" (UID: \"f1eb5f04-d170-4a47-a67e-adcfe4e15f52\") " pod="kuadrant-system/authorino-6b64d87cdc-8dk7q"
Sep 05 06:34:35.915834 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.915798    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gck7q\" (UniqueName: \"kubernetes.io/projected/f1eb5f04-d170-4a47-a67e-adcfe4e15f52-kube-api-access-gck7q\") pod \"authorino-6b64d87cdc-8dk7q\" (UID: \"f1eb5f04-d170-4a47-a67e-adcfe4e15f52\") " pod="kuadrant-system/authorino-6b64d87cdc-8dk7q"
Sep 05 06:34:35.933293 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:35.933265    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b64d87cdc-8dk7q"
Sep 05 06:34:36.211280 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:36.211248    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6b64d87cdc-8dk7q"]
Sep 05 06:34:36.213224 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:34:36.213188    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1eb5f04_d170_4a47_a67e_adcfe4e15f52.slice/crio-a6d13b5709db357ba755438d422a120d6d4dd598b23df1f7df06a723f426c9db WatchSource:0}: Error finding container a6d13b5709db357ba755438d422a120d6d4dd598b23df1f7df06a723f426c9db: Status 404 returned error can't find the container with id a6d13b5709db357ba755438d422a120d6d4dd598b23df1f7df06a723f426c9db
Sep 05 06:34:36.378212 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:36.378161    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b64d87cdc-8dk7q" event={"ID":"f1eb5f04-d170-4a47-a67e-adcfe4e15f52","Type":"ContainerStarted","Data":"a6d13b5709db357ba755438d422a120d6d4dd598b23df1f7df06a723f426c9db"}
Sep 05 06:34:36.378750 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:36.378278    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-cqx78"
Sep 05 06:34:36.378750 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:36.378280    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-zk8v5"
Sep 05 06:34:36.387170 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:36.387140    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-cqx78"
Sep 05 06:34:36.408570 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:36.408534    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6d44f65-01a9-48e2-9826-227d40152b7a-tls-cert\") pod \"f6d44f65-01a9-48e2-9826-227d40152b7a\" (UID: \"f6d44f65-01a9-48e2-9826-227d40152b7a\") "
Sep 05 06:34:36.408570 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:36.408574    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dslb\" (UniqueName: \"kubernetes.io/projected/f6d44f65-01a9-48e2-9826-227d40152b7a-kube-api-access-6dslb\") pod \"f6d44f65-01a9-48e2-9826-227d40152b7a\" (UID: \"f6d44f65-01a9-48e2-9826-227d40152b7a\") "
Sep 05 06:34:36.410726 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:36.410691    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6d44f65-01a9-48e2-9826-227d40152b7a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f6d44f65-01a9-48e2-9826-227d40152b7a" (UID: "f6d44f65-01a9-48e2-9826-227d40152b7a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:34:36.410860 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:36.410742    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6d44f65-01a9-48e2-9826-227d40152b7a-kube-api-access-6dslb" (OuterVolumeSpecName: "kube-api-access-6dslb") pod "f6d44f65-01a9-48e2-9826-227d40152b7a" (UID: "f6d44f65-01a9-48e2-9826-227d40152b7a"). InnerVolumeSpecName "kube-api-access-6dslb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:34:36.433034 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:36.433000    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-zk8v5"]
Sep 05 06:34:36.437319 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:36.437288    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-zk8v5"]
Sep 05 06:34:36.509294 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:36.509189    2559 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f6d44f65-01a9-48e2-9826-227d40152b7a-tls-cert\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:34:36.509294 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:36.509227    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dslb\" (UniqueName: \"kubernetes.io/projected/f6d44f65-01a9-48e2-9826-227d40152b7a-kube-api-access-6dslb\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:34:37.385777 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:37.385732    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b64d87cdc-8dk7q" event={"ID":"f1eb5f04-d170-4a47-a67e-adcfe4e15f52","Type":"ContainerStarted","Data":"84013d5835c5f6486fbeb2e6b9d45fa27f2f38303a79f030c3ab7a278724386e"}
Sep 05 06:34:37.385777 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:37.385767    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687bd45585-cqx78"
Sep 05 06:34:37.416425 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:37.416371    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6b64d87cdc-8dk7q" podStartSLOduration=1.905056804 podStartE2EDuration="2.416353717s" totalImagesPullingTime="511.296913ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:34:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:34:37.415593491 +0000 UTC m=+494.006152444" watchObservedRunningTime="2026-09-05 06:34:37.416353717 +0000 UTC m=+494.006912995"
Sep 05 06:34:37.446665 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:37.446627    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687bd45585-cqx78"]
Sep 05 06:34:37.453780 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:37.453740    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-687bd45585-cqx78"]
Sep 05 06:34:37.456127 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:37.456097    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xvfx9"]
Sep 05 06:34:37.456343 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:37.456310    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-xvfx9" podUID="1ad4c083-36a8-4111-9a5d-82ba4cf950a0" containerName="authorino" containerID="cri-o://5a5bc93d44f534d62cbafe7f314378518f234180dde9ca6e629896a640a65339" gracePeriod=30
Sep 05 06:34:37.703536 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:37.703508    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-xvfx9"
Sep 05 06:34:37.719030 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:37.719001    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dldxr\" (UniqueName: \"kubernetes.io/projected/1ad4c083-36a8-4111-9a5d-82ba4cf950a0-kube-api-access-dldxr\") pod \"1ad4c083-36a8-4111-9a5d-82ba4cf950a0\" (UID: \"1ad4c083-36a8-4111-9a5d-82ba4cf950a0\") "
Sep 05 06:34:37.721065 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:37.721032    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ad4c083-36a8-4111-9a5d-82ba4cf950a0-kube-api-access-dldxr" (OuterVolumeSpecName: "kube-api-access-dldxr") pod "1ad4c083-36a8-4111-9a5d-82ba4cf950a0" (UID: "1ad4c083-36a8-4111-9a5d-82ba4cf950a0"). InnerVolumeSpecName "kube-api-access-dldxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:34:37.819591 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:37.819549    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dldxr\" (UniqueName: \"kubernetes.io/projected/1ad4c083-36a8-4111-9a5d-82ba4cf950a0-kube-api-access-dldxr\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:34:37.893049 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:37.893012    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93f3a94e-70e1-4f10-9f54-6b14a3fdc06a" path="/var/lib/kubelet/pods/93f3a94e-70e1-4f10-9f54-6b14a3fdc06a/volumes"
Sep 05 06:34:37.893323 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:37.893310    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6d44f65-01a9-48e2-9826-227d40152b7a" path="/var/lib/kubelet/pods/f6d44f65-01a9-48e2-9826-227d40152b7a/volumes"
Sep 05 06:34:38.392754 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:38.392712    2559 generic.go:356] "Generic (PLEG): container finished" podID="1ad4c083-36a8-4111-9a5d-82ba4cf950a0" containerID="5a5bc93d44f534d62cbafe7f314378518f234180dde9ca6e629896a640a65339" exitCode=0
Sep 05 06:34:38.393299 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:38.392761    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-xvfx9"
Sep 05 06:34:38.393299 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:38.392802    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-xvfx9" event={"ID":"1ad4c083-36a8-4111-9a5d-82ba4cf950a0","Type":"ContainerDied","Data":"5a5bc93d44f534d62cbafe7f314378518f234180dde9ca6e629896a640a65339"}
Sep 05 06:34:38.393299 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:38.392858    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-xvfx9" event={"ID":"1ad4c083-36a8-4111-9a5d-82ba4cf950a0","Type":"ContainerDied","Data":"cbbd3c5a83fa1a65e2738c7d810a8519c650e14a2f8a17e77adaee0df939d914"}
Sep 05 06:34:38.393299 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:38.392910    2559 scope.go:117] "RemoveContainer" containerID="5a5bc93d44f534d62cbafe7f314378518f234180dde9ca6e629896a640a65339"
Sep 05 06:34:38.402184 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:38.402159    2559 scope.go:117] "RemoveContainer" containerID="5a5bc93d44f534d62cbafe7f314378518f234180dde9ca6e629896a640a65339"
Sep 05 06:34:38.402463 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:34:38.402445    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a5bc93d44f534d62cbafe7f314378518f234180dde9ca6e629896a640a65339\": container with ID starting with 5a5bc93d44f534d62cbafe7f314378518f234180dde9ca6e629896a640a65339 not found: ID does not exist" containerID="5a5bc93d44f534d62cbafe7f314378518f234180dde9ca6e629896a640a65339"
Sep 05 06:34:38.402520 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:38.402472    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a5bc93d44f534d62cbafe7f314378518f234180dde9ca6e629896a640a65339"} err="failed to get container status \"5a5bc93d44f534d62cbafe7f314378518f234180dde9ca6e629896a640a65339\": rpc error: code = NotFound desc = could not find container \"5a5bc93d44f534d62cbafe7f314378518f234180dde9ca6e629896a640a65339\": container with ID starting with 5a5bc93d44f534d62cbafe7f314378518f234180dde9ca6e629896a640a65339 not found: ID does not exist"
Sep 05 06:34:38.417269 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:38.417241    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xvfx9"]
Sep 05 06:34:38.423536 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:38.423511    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-xvfx9"]
Sep 05 06:34:39.891983 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:34:39.891942    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ad4c083-36a8-4111-9a5d-82ba4cf950a0" path="/var/lib/kubelet/pods/1ad4c083-36a8-4111-9a5d-82ba4cf950a0/volumes"
Sep 05 06:35:22.543197 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:35:22.542953    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:35:45.218050 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:35:45.218009    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:36:02.489904 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.489837    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"]
Sep 05 06:36:02.490399 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.490372    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ad4c083-36a8-4111-9a5d-82ba4cf950a0" containerName="authorino"
Sep 05 06:36:02.490399 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.490393    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ad4c083-36a8-4111-9a5d-82ba4cf950a0" containerName="authorino"
Sep 05 06:36:02.490526 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.490513    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ad4c083-36a8-4111-9a5d-82ba4cf950a0" containerName="authorino"
Sep 05 06:36:02.492579 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.492550    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.496825 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.496801    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jfksv\""
Sep 05 06:36:02.496825 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.496814    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 05 06:36:02.497042 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.496860    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 05 06:36:02.497042 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.496815    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 05 06:36:02.503814 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.503786    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"]
Sep 05 06:36:02.533173 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.533140    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/133997e4-7df9-4d51-ae1a-315fd98e7c7f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.533345 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.533179    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/133997e4-7df9-4d51-ae1a-315fd98e7c7f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.533345 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.533218    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns6wv\" (UniqueName: \"kubernetes.io/projected/133997e4-7df9-4d51-ae1a-315fd98e7c7f-kube-api-access-ns6wv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.533345 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.533264    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/133997e4-7df9-4d51-ae1a-315fd98e7c7f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.533345 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.533313    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/133997e4-7df9-4d51-ae1a-315fd98e7c7f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.533532 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.533392    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/133997e4-7df9-4d51-ae1a-315fd98e7c7f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.634867 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.634825    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/133997e4-7df9-4d51-ae1a-315fd98e7c7f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.634867 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.634866    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/133997e4-7df9-4d51-ae1a-315fd98e7c7f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.635161 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.635014    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/133997e4-7df9-4d51-ae1a-315fd98e7c7f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.635161 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.635061    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/133997e4-7df9-4d51-ae1a-315fd98e7c7f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.635161 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.635088    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/133997e4-7df9-4d51-ae1a-315fd98e7c7f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.635161 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.635156    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns6wv\" (UniqueName: \"kubernetes.io/projected/133997e4-7df9-4d51-ae1a-315fd98e7c7f-kube-api-access-ns6wv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.635408 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.635386    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/133997e4-7df9-4d51-ae1a-315fd98e7c7f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.635487 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.635463    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/133997e4-7df9-4d51-ae1a-315fd98e7c7f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.635549 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.635508    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/133997e4-7df9-4d51-ae1a-315fd98e7c7f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.637351 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.637321    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/133997e4-7df9-4d51-ae1a-315fd98e7c7f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.637604 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.637584    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/133997e4-7df9-4d51-ae1a-315fd98e7c7f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.645077 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.645050    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns6wv\" (UniqueName: \"kubernetes.io/projected/133997e4-7df9-4d51-ae1a-315fd98e7c7f-kube-api-access-ns6wv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bcw86\" (UID: \"133997e4-7df9-4d51-ae1a-315fd98e7c7f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:02.804196 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:02.804086    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:03.052140 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:03.052095    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:36:03.113165 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:03.113126    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"]
Sep 05 06:36:03.113434 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:36:03.113406    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod133997e4_7df9_4d51_ae1a_315fd98e7c7f.slice/crio-aa74831121b72d6e03e8218ccf455f5122510cdb2d805cb6ae289f95d55ecfd6 WatchSource:0}: Error finding container aa74831121b72d6e03e8218ccf455f5122510cdb2d805cb6ae289f95d55ecfd6: Status 404 returned error can't find the container with id aa74831121b72d6e03e8218ccf455f5122510cdb2d805cb6ae289f95d55ecfd6
Sep 05 06:36:03.863238 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:03.863182    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86" event={"ID":"133997e4-7df9-4d51-ae1a-315fd98e7c7f","Type":"ContainerStarted","Data":"aa74831121b72d6e03e8218ccf455f5122510cdb2d805cb6ae289f95d55ecfd6"}
Sep 05 06:36:04.869827 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:04.869790    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86" event={"ID":"133997e4-7df9-4d51-ae1a-315fd98e7c7f","Type":"ContainerStarted","Data":"15c6ebcf363c1ab3889b777c3ec95fcaa734629bcec83d7f96121843359648d1"}
Sep 05 06:36:04.891736 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:04.891678    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86" podStartSLOduration=1.869236905 podStartE2EDuration="2.891659863s" totalImagesPullingTime="1.022422958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:36:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:36:04.890157653 +0000 UTC m=+581.480716619" watchObservedRunningTime="2026-09-05 06:36:04.891659863 +0000 UTC m=+581.482218809"
Sep 05 06:36:11.171775 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.171736    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"]
Sep 05 06:36:11.174237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.174217    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.180290 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.180256    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 05 06:36:11.187523 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.187496    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"]
Sep 05 06:36:11.315257 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.315216    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1962265-ecfd-4996-9a59-c6e1fd72e72f-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.315459 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.315321    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1962265-ecfd-4996-9a59-c6e1fd72e72f-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.315459 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.315358    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1962265-ecfd-4996-9a59-c6e1fd72e72f-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.315459 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.315421    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz9dm\" (UniqueName: \"kubernetes.io/projected/b1962265-ecfd-4996-9a59-c6e1fd72e72f-kube-api-access-wz9dm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.315607 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.315475    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1962265-ecfd-4996-9a59-c6e1fd72e72f-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.315607 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.315499    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1962265-ecfd-4996-9a59-c6e1fd72e72f-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.416292 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.416239    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz9dm\" (UniqueName: \"kubernetes.io/projected/b1962265-ecfd-4996-9a59-c6e1fd72e72f-kube-api-access-wz9dm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.416292 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.416292    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1962265-ecfd-4996-9a59-c6e1fd72e72f-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.416593 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.416325    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1962265-ecfd-4996-9a59-c6e1fd72e72f-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.416593 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.416370    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1962265-ecfd-4996-9a59-c6e1fd72e72f-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.416593 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.416421    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1962265-ecfd-4996-9a59-c6e1fd72e72f-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.416593 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.416444    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1962265-ecfd-4996-9a59-c6e1fd72e72f-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.416840 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.416811    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1962265-ecfd-4996-9a59-c6e1fd72e72f-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.416840 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.416830    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1962265-ecfd-4996-9a59-c6e1fd72e72f-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.416960 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.416916    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1962265-ecfd-4996-9a59-c6e1fd72e72f-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.418809 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.418775    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1962265-ecfd-4996-9a59-c6e1fd72e72f-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.419195 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.419170    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1962265-ecfd-4996-9a59-c6e1fd72e72f-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.427547 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.427473    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz9dm\" (UniqueName: \"kubernetes.io/projected/b1962265-ecfd-4996-9a59-c6e1fd72e72f-kube-api-access-wz9dm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l\" (UID: \"b1962265-ecfd-4996-9a59-c6e1fd72e72f\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.485296 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.485254    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:11.762584 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.762553    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"]
Sep 05 06:36:11.764226 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:36:11.764198    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1962265_ecfd_4996_9a59_c6e1fd72e72f.slice/crio-a36913bb759b5df23b85fbfffb57494deabc2dc4904f59005807e41eafce5b51 WatchSource:0}: Error finding container a36913bb759b5df23b85fbfffb57494deabc2dc4904f59005807e41eafce5b51: Status 404 returned error can't find the container with id a36913bb759b5df23b85fbfffb57494deabc2dc4904f59005807e41eafce5b51
Sep 05 06:36:11.908167 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:11.908124    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l" event={"ID":"b1962265-ecfd-4996-9a59-c6e1fd72e72f","Type":"ContainerStarted","Data":"a36913bb759b5df23b85fbfffb57494deabc2dc4904f59005807e41eafce5b51"}
Sep 05 06:36:12.496703 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:12.496655    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:36:12.804324 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:12.804205    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:12.804324 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:12.804282    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:12.819802 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:12.819758    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:12.915935 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:12.915894    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l" event={"ID":"b1962265-ecfd-4996-9a59-c6e1fd72e72f","Type":"ContainerStarted","Data":"0fad4e8b42b47d24a344cad9c24245169dc8fe3b217fad6478295c728ab65dbe"}
Sep 05 06:36:12.930358 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:12.930320    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bcw86"
Sep 05 06:36:12.956963 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:12.956871    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l" podStartSLOduration=1.807190767 podStartE2EDuration="1.956843608s" totalImagesPullingTime="149.652841ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:36:12.936023624 +0000 UTC m=+589.526582591" watchObservedRunningTime="2026-09-05 06:36:12.956843608 +0000 UTC m=+589.547402558"
Sep 05 06:36:21.485673 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:21.485622    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:21.486353 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:21.485750    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:21.498659 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:21.498626    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:21.985840 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:21.985797    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l"
Sep 05 06:36:22.030848 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:22.030811    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:36:50.297730 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.297649    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"]
Sep 05 06:36:50.300063 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.300043    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.303184 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.303166    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 05 06:36:50.316536 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.316513    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"]
Sep 05 06:36:50.341538 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.341513    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.341646 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.341548    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l95b6\" (UniqueName: \"kubernetes.io/projected/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-kube-api-access-l95b6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.341646 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.341574    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.341646 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.341634    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.341765 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.341676    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.341765 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.341730    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.442826 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.442797    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.442995 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.442847    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.442995 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.442918    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.442995 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.442970    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.443153 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.443004    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l95b6\" (UniqueName: \"kubernetes.io/projected/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-kube-api-access-l95b6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.443153 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.443031    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.443349 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.443313    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.443422 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.443374    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.443462 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.443417    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.445119 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.445099    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.445373 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.445353    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.452708 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.452689    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l95b6\" (UniqueName: \"kubernetes.io/projected/bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca-kube-api-access-l95b6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9\" (UID: \"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.610763 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.610699    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:36:50.886422 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.886393    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"]
Sep 05 06:36:50.888207 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:36:50.888174    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb22452a_2e27_4b6d_8f3a_fe66c0ed96ca.slice/crio-ad3e588eb128b6c2c48f96ce4aef1a393549c1ece4fd48fcffa6886c63754723 WatchSource:0}: Error finding container ad3e588eb128b6c2c48f96ce4aef1a393549c1ece4fd48fcffa6886c63754723: Status 404 returned error can't find the container with id ad3e588eb128b6c2c48f96ce4aef1a393549c1ece4fd48fcffa6886c63754723
Sep 05 06:36:50.889721 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:50.889701    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 06:36:51.133557 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:51.133519    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9" event={"ID":"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca","Type":"ContainerStarted","Data":"e0826f087bf73bef144c5a4f9a30549794754bf3142f75c3742a287905dfdb46"}
Sep 05 06:36:51.133711 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:51.133680    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9" event={"ID":"bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca","Type":"ContainerStarted","Data":"ad3e588eb128b6c2c48f96ce4aef1a393549c1ece4fd48fcffa6886c63754723"}
Sep 05 06:36:51.153671 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:51.153581    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9" podStartSLOduration=0.990785907 podStartE2EDuration="1.153563755s" totalImagesPullingTime="162.777848ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:36:51.153042149 +0000 UTC m=+627.743601122" watchObservedRunningTime="2026-09-05 06:36:51.153563755 +0000 UTC m=+627.744122708"
Sep 05 06:36:51.893345 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:51.893306    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:36:56.277592 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.277553    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"]
Sep 05 06:36:56.281322 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.281299    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.284319 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.284296    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 05 06:36:56.295366 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.295343    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"]
Sep 05 06:36:56.391217 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.391191    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b189488c-63bc-4885-ba0a-3a675c048bd6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.391377 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.391229    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b189488c-63bc-4885-ba0a-3a675c048bd6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.391377 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.391252    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzjxt\" (UniqueName: \"kubernetes.io/projected/b189488c-63bc-4885-ba0a-3a675c048bd6-kube-api-access-lzjxt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.391377 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.391350    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b189488c-63bc-4885-ba0a-3a675c048bd6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.391536 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.391405    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b189488c-63bc-4885-ba0a-3a675c048bd6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.391536 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.391436    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b189488c-63bc-4885-ba0a-3a675c048bd6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.492516 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.492481    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b189488c-63bc-4885-ba0a-3a675c048bd6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.492666 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.492530    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b189488c-63bc-4885-ba0a-3a675c048bd6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.492666 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.492557    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzjxt\" (UniqueName: \"kubernetes.io/projected/b189488c-63bc-4885-ba0a-3a675c048bd6-kube-api-access-lzjxt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.492666 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.492584    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b189488c-63bc-4885-ba0a-3a675c048bd6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.492666 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.492603    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b189488c-63bc-4885-ba0a-3a675c048bd6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.492666 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.492625    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b189488c-63bc-4885-ba0a-3a675c048bd6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.492973 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.492917    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b189488c-63bc-4885-ba0a-3a675c048bd6-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.493029 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.492987    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b189488c-63bc-4885-ba0a-3a675c048bd6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.493088 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.493067    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b189488c-63bc-4885-ba0a-3a675c048bd6-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.494675 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.494650    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b189488c-63bc-4885-ba0a-3a675c048bd6-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.495020 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.495000    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b189488c-63bc-4885-ba0a-3a675c048bd6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.501375 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.501346    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzjxt\" (UniqueName: \"kubernetes.io/projected/b189488c-63bc-4885-ba0a-3a675c048bd6-kube-api-access-lzjxt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2p5z4\" (UID: \"b189488c-63bc-4885-ba0a-3a675c048bd6\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.592552 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.592489    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:36:56.870716 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:56.870680    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"]
Sep 05 06:36:57.167969 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:57.167850    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4" event={"ID":"b189488c-63bc-4885-ba0a-3a675c048bd6","Type":"ContainerStarted","Data":"7fd21f33e8f7227aa8b4323be5f4d950cd2d0d76c82ca098e74a5f8e6e12f8d3"}
Sep 05 06:36:57.167969 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:57.167932    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4" event={"ID":"b189488c-63bc-4885-ba0a-3a675c048bd6","Type":"ContainerStarted","Data":"760b9c9147debabc1b22bd6f6a8c4b721e9e73aa52485e5c49c34ef6a077a6da"}
Sep 05 06:36:57.186779 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:57.186731    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4" podStartSLOduration=1.01375495 podStartE2EDuration="1.186711724s" totalImagesPullingTime="172.956774ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:36:57.186171502 +0000 UTC m=+633.776730455" watchObservedRunningTime="2026-09-05 06:36:57.186711724 +0000 UTC m=+633.777270678"
Sep 05 06:36:58.389847 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:36:58.389814    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:37:00.611690 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:00.611656    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:37:00.612090 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:00.611984    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:37:00.625473 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:00.625442    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:37:01.203504 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:01.203477    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9"
Sep 05 06:37:06.593227 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.593192    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:37:06.593583 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.593265    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:37:06.605544 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.605519    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:37:06.680595 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.680565    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"]
Sep 05 06:37:06.684988 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.684968    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.687510 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.687490    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 05 06:37:06.694103 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.694079    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"]
Sep 05 06:37:06.775618 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.775586    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c686d0fe-cc66-4a2a-a592-48f209314c9f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.775760 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.775630    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c686d0fe-cc66-4a2a-a592-48f209314c9f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.775760 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.775707    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c686d0fe-cc66-4a2a-a592-48f209314c9f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.775839 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.775766    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c686d0fe-cc66-4a2a-a592-48f209314c9f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.775839 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.775818    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xlfp\" (UniqueName: \"kubernetes.io/projected/c686d0fe-cc66-4a2a-a592-48f209314c9f-kube-api-access-6xlfp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.775958 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.775851    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c686d0fe-cc66-4a2a-a592-48f209314c9f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.876940 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.876842    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xlfp\" (UniqueName: \"kubernetes.io/projected/c686d0fe-cc66-4a2a-a592-48f209314c9f-kube-api-access-6xlfp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.876940 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.876916    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c686d0fe-cc66-4a2a-a592-48f209314c9f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.877115 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.876949    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c686d0fe-cc66-4a2a-a592-48f209314c9f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.877115 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.876980    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c686d0fe-cc66-4a2a-a592-48f209314c9f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.877115 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.877044    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c686d0fe-cc66-4a2a-a592-48f209314c9f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.877115 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.877086    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c686d0fe-cc66-4a2a-a592-48f209314c9f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.877385 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.877362    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c686d0fe-cc66-4a2a-a592-48f209314c9f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.877385 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.877374    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c686d0fe-cc66-4a2a-a592-48f209314c9f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.877499 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.877443    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c686d0fe-cc66-4a2a-a592-48f209314c9f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.879364 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.879335    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c686d0fe-cc66-4a2a-a592-48f209314c9f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.879476 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.879456    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c686d0fe-cc66-4a2a-a592-48f209314c9f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.886068 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.886046    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xlfp\" (UniqueName: \"kubernetes.io/projected/c686d0fe-cc66-4a2a-a592-48f209314c9f-kube-api-access-6xlfp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-grthp\" (UID: \"c686d0fe-cc66-4a2a-a592-48f209314c9f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:06.995431 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:06.995410    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:07.250780 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:07.250751    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2p5z4"
Sep 05 06:37:07.269121 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:07.269094    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"]
Sep 05 06:37:07.270526 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:37:07.270488    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc686d0fe_cc66_4a2a_a592_48f209314c9f.slice/crio-fdce7f7b6d178cb7e2c5dbba10878997694261911dcb99e61ab4ec94eab55a60 WatchSource:0}: Error finding container fdce7f7b6d178cb7e2c5dbba10878997694261911dcb99e61ab4ec94eab55a60: Status 404 returned error can't find the container with id fdce7f7b6d178cb7e2c5dbba10878997694261911dcb99e61ab4ec94eab55a60
Sep 05 06:37:08.246357 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:08.246315    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp" event={"ID":"c686d0fe-cc66-4a2a-a592-48f209314c9f","Type":"ContainerStarted","Data":"9b7de2e7292fcbf4b0e0a52c05925c0cd5b13263edcb429d31b3719f5f7d581e"}
Sep 05 06:37:08.246734 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:08.246370    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp" event={"ID":"c686d0fe-cc66-4a2a-a592-48f209314c9f","Type":"ContainerStarted","Data":"fdce7f7b6d178cb7e2c5dbba10878997694261911dcb99e61ab4ec94eab55a60"}
Sep 05 06:37:08.266642 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:08.266594    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp" podStartSLOduration=2.101646648 podStartE2EDuration="2.266577863s" totalImagesPullingTime="164.931215ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:37:08.265579435 +0000 UTC m=+644.856138387" watchObservedRunningTime="2026-09-05 06:37:08.266577863 +0000 UTC m=+644.857136872"
Sep 05 06:37:09.082854 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:09.082819    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:37:16.996283 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:16.996248    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:16.996661 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:16.996295    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:17.008651 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:17.008624    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:37:17.306152 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:37:17.306072    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-grthp"
Sep 05 06:38:08.262678 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:08.262640    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57cf4f5b8b-2vcjq"]
Sep 05 06:38:08.266202 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:08.266180    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57cf4f5b8b-2vcjq"
Sep 05 06:38:08.275294 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:08.275268    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57cf4f5b8b-2vcjq"]
Sep 05 06:38:08.349313 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:08.349285    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhts6\" (UniqueName: \"kubernetes.io/projected/b9279c18-f100-497c-b1c1-01175bd1e84b-kube-api-access-mhts6\") pod \"authorino-57cf4f5b8b-2vcjq\" (UID: \"b9279c18-f100-497c-b1c1-01175bd1e84b\") " pod="kuadrant-system/authorino-57cf4f5b8b-2vcjq"
Sep 05 06:38:08.349434 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:08.349330    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b9279c18-f100-497c-b1c1-01175bd1e84b-tls-cert\") pod \"authorino-57cf4f5b8b-2vcjq\" (UID: \"b9279c18-f100-497c-b1c1-01175bd1e84b\") " pod="kuadrant-system/authorino-57cf4f5b8b-2vcjq"
Sep 05 06:38:08.450507 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:08.450477    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhts6\" (UniqueName: \"kubernetes.io/projected/b9279c18-f100-497c-b1c1-01175bd1e84b-kube-api-access-mhts6\") pod \"authorino-57cf4f5b8b-2vcjq\" (UID: \"b9279c18-f100-497c-b1c1-01175bd1e84b\") " pod="kuadrant-system/authorino-57cf4f5b8b-2vcjq"
Sep 05 06:38:08.450631 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:08.450528    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b9279c18-f100-497c-b1c1-01175bd1e84b-tls-cert\") pod \"authorino-57cf4f5b8b-2vcjq\" (UID: \"b9279c18-f100-497c-b1c1-01175bd1e84b\") " pod="kuadrant-system/authorino-57cf4f5b8b-2vcjq"
Sep 05 06:38:08.452824 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:08.452798    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b9279c18-f100-497c-b1c1-01175bd1e84b-tls-cert\") pod \"authorino-57cf4f5b8b-2vcjq\" (UID: \"b9279c18-f100-497c-b1c1-01175bd1e84b\") " pod="kuadrant-system/authorino-57cf4f5b8b-2vcjq"
Sep 05 06:38:08.460273 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:08.460252    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhts6\" (UniqueName: \"kubernetes.io/projected/b9279c18-f100-497c-b1c1-01175bd1e84b-kube-api-access-mhts6\") pod \"authorino-57cf4f5b8b-2vcjq\" (UID: \"b9279c18-f100-497c-b1c1-01175bd1e84b\") " pod="kuadrant-system/authorino-57cf4f5b8b-2vcjq"
Sep 05 06:38:08.575476 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:08.575404    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57cf4f5b8b-2vcjq"
Sep 05 06:38:08.845970 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:08.845941    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57cf4f5b8b-2vcjq"]
Sep 05 06:38:08.847410 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:38:08.847383    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9279c18_f100_497c_b1c1_01175bd1e84b.slice/crio-3fcec9afb321be651bc96c18b0e42d6396043c72fce19aba27e2575e87918bbf WatchSource:0}: Error finding container 3fcec9afb321be651bc96c18b0e42d6396043c72fce19aba27e2575e87918bbf: Status 404 returned error can't find the container with id 3fcec9afb321be651bc96c18b0e42d6396043c72fce19aba27e2575e87918bbf
Sep 05 06:38:09.568954 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:09.568922    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57cf4f5b8b-2vcjq" event={"ID":"b9279c18-f100-497c-b1c1-01175bd1e84b","Type":"ContainerStarted","Data":"c7c858d3dfeed5a4dc1f493aa538d1d65601525e9e12c856dea3bfdafb1ec577"}
Sep 05 06:38:09.569255 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:09.568969    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57cf4f5b8b-2vcjq" event={"ID":"b9279c18-f100-497c-b1c1-01175bd1e84b","Type":"ContainerStarted","Data":"3fcec9afb321be651bc96c18b0e42d6396043c72fce19aba27e2575e87918bbf"}
Sep 05 06:38:09.619779 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:09.619722    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-57cf4f5b8b-2vcjq" podStartSLOduration=0.978640439 podStartE2EDuration="1.619699759s" totalImagesPullingTime="641.05932ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:38:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:38:09.588508337 +0000 UTC m=+706.179067291" watchObservedRunningTime="2026-09-05 06:38:09.619699759 +0000 UTC m=+706.210258712"
Sep 05 06:38:09.622560 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:09.622531    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6b64d87cdc-8dk7q"]
Sep 05 06:38:09.622754 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:09.622729    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6b64d87cdc-8dk7q" podUID="f1eb5f04-d170-4a47-a67e-adcfe4e15f52" containerName="authorino" containerID="cri-o://84013d5835c5f6486fbeb2e6b9d45fa27f2f38303a79f030c3ab7a278724386e" gracePeriod=30
Sep 05 06:38:09.875395 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:09.875366    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b64d87cdc-8dk7q"
Sep 05 06:38:09.963162 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:09.963127    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gck7q\" (UniqueName: \"kubernetes.io/projected/f1eb5f04-d170-4a47-a67e-adcfe4e15f52-kube-api-access-gck7q\") pod \"f1eb5f04-d170-4a47-a67e-adcfe4e15f52\" (UID: \"f1eb5f04-d170-4a47-a67e-adcfe4e15f52\") "
Sep 05 06:38:09.963346 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:09.963215    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f1eb5f04-d170-4a47-a67e-adcfe4e15f52-tls-cert\") pod \"f1eb5f04-d170-4a47-a67e-adcfe4e15f52\" (UID: \"f1eb5f04-d170-4a47-a67e-adcfe4e15f52\") "
Sep 05 06:38:09.965248 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:09.965221    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1eb5f04-d170-4a47-a67e-adcfe4e15f52-kube-api-access-gck7q" (OuterVolumeSpecName: "kube-api-access-gck7q") pod "f1eb5f04-d170-4a47-a67e-adcfe4e15f52" (UID: "f1eb5f04-d170-4a47-a67e-adcfe4e15f52"). InnerVolumeSpecName "kube-api-access-gck7q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:38:09.973697 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:09.973667    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1eb5f04-d170-4a47-a67e-adcfe4e15f52-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "f1eb5f04-d170-4a47-a67e-adcfe4e15f52" (UID: "f1eb5f04-d170-4a47-a67e-adcfe4e15f52"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:38:10.064624 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:10.064545    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gck7q\" (UniqueName: \"kubernetes.io/projected/f1eb5f04-d170-4a47-a67e-adcfe4e15f52-kube-api-access-gck7q\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:38:10.064624 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:10.064572    2559 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f1eb5f04-d170-4a47-a67e-adcfe4e15f52-tls-cert\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:38:10.575307 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:10.575271    2559 generic.go:356] "Generic (PLEG): container finished" podID="f1eb5f04-d170-4a47-a67e-adcfe4e15f52" containerID="84013d5835c5f6486fbeb2e6b9d45fa27f2f38303a79f030c3ab7a278724386e" exitCode=0
Sep 05 06:38:10.575736 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:10.575313    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6b64d87cdc-8dk7q"
Sep 05 06:38:10.575736 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:10.575363    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b64d87cdc-8dk7q" event={"ID":"f1eb5f04-d170-4a47-a67e-adcfe4e15f52","Type":"ContainerDied","Data":"84013d5835c5f6486fbeb2e6b9d45fa27f2f38303a79f030c3ab7a278724386e"}
Sep 05 06:38:10.575736 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:10.575401    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6b64d87cdc-8dk7q" event={"ID":"f1eb5f04-d170-4a47-a67e-adcfe4e15f52","Type":"ContainerDied","Data":"a6d13b5709db357ba755438d422a120d6d4dd598b23df1f7df06a723f426c9db"}
Sep 05 06:38:10.575736 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:10.575422    2559 scope.go:117] "RemoveContainer" containerID="84013d5835c5f6486fbeb2e6b9d45fa27f2f38303a79f030c3ab7a278724386e"
Sep 05 06:38:10.585476 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:10.585453    2559 scope.go:117] "RemoveContainer" containerID="84013d5835c5f6486fbeb2e6b9d45fa27f2f38303a79f030c3ab7a278724386e"
Sep 05 06:38:10.585790 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:38:10.585769    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84013d5835c5f6486fbeb2e6b9d45fa27f2f38303a79f030c3ab7a278724386e\": container with ID starting with 84013d5835c5f6486fbeb2e6b9d45fa27f2f38303a79f030c3ab7a278724386e not found: ID does not exist" containerID="84013d5835c5f6486fbeb2e6b9d45fa27f2f38303a79f030c3ab7a278724386e"
Sep 05 06:38:10.585855 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:10.585800    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84013d5835c5f6486fbeb2e6b9d45fa27f2f38303a79f030c3ab7a278724386e"} err="failed to get container status \"84013d5835c5f6486fbeb2e6b9d45fa27f2f38303a79f030c3ab7a278724386e\": rpc error: code = NotFound desc = could not find container \"84013d5835c5f6486fbeb2e6b9d45fa27f2f38303a79f030c3ab7a278724386e\": container with ID starting with 84013d5835c5f6486fbeb2e6b9d45fa27f2f38303a79f030c3ab7a278724386e not found: ID does not exist"
Sep 05 06:38:10.598997 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:10.598973    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6b64d87cdc-8dk7q"]
Sep 05 06:38:10.607238 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:10.607217    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6b64d87cdc-8dk7q"]
Sep 05 06:38:11.892787 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:11.892749    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1eb5f04-d170-4a47-a67e-adcfe4e15f52" path="/var/lib/kubelet/pods/f1eb5f04-d170-4a47-a67e-adcfe4e15f52/volumes"
Sep 05 06:38:31.821987 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:31.821949    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:38:41.491768 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:41.491732    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:38:54.923075 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:38:54.923036    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:39:00.670027 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.669994    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"]
Sep 05 06:39:00.670414 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.670402    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1eb5f04-d170-4a47-a67e-adcfe4e15f52" containerName="authorino"
Sep 05 06:39:00.670501 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.670416    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1eb5f04-d170-4a47-a67e-adcfe4e15f52" containerName="authorino"
Sep 05 06:39:00.670501 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.670490    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1eb5f04-d170-4a47-a67e-adcfe4e15f52" containerName="authorino"
Sep 05 06:39:00.673690 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.673673    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.679955 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.676189    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-c4e74764-kserve-self-signed-certs\""
Sep 05 06:39:00.684035 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.684008    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"]
Sep 05 06:39:00.686918 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.686872    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-home\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.687041 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.686986    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-model-cache\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.687107 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.687060    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp5sm\" (UniqueName: \"kubernetes.io/projected/967d2129-2c77-42b1-9dcf-82372113175a-kube-api-access-gp5sm\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.687478 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.687127    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/967d2129-2c77-42b1-9dcf-82372113175a-tls-certs\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.687478 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.687181    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-tmp-dir\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.687478 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.687209    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-dshm\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.787823 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.787793    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/967d2129-2c77-42b1-9dcf-82372113175a-tls-certs\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.787989 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.787840    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-tmp-dir\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.787989 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.787865    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-dshm\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.787989 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.787930    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-home\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.788158 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.788048    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-model-cache\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.788158 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.788109    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp5sm\" (UniqueName: \"kubernetes.io/projected/967d2129-2c77-42b1-9dcf-82372113175a-kube-api-access-gp5sm\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.788369 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.788346    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-home\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.788369 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.788360    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-tmp-dir\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.788469 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.788410    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-model-cache\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.790043 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.790012    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-dshm\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.790277 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.790262    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/967d2129-2c77-42b1-9dcf-82372113175a-tls-certs\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.802872 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.802848    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp5sm\" (UniqueName: \"kubernetes.io/projected/967d2129-2c77-42b1-9dcf-82372113175a-kube-api-access-gp5sm\") pod \"e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") " pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:00.989805 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:00.989777    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:01.270810 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:01.270789    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"]
Sep 05 06:39:01.272809 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:39:01.272782    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod967d2129_2c77_42b1_9dcf_82372113175a.slice/crio-da81513a6d970dd25709037a31f2a61fe2908e30d80493d21a04d655e8248da2 WatchSource:0}: Error finding container da81513a6d970dd25709037a31f2a61fe2908e30d80493d21a04d655e8248da2: Status 404 returned error can't find the container with id da81513a6d970dd25709037a31f2a61fe2908e30d80493d21a04d655e8248da2
Sep 05 06:39:01.868750 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:01.868715    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr" event={"ID":"967d2129-2c77-42b1-9dcf-82372113175a","Type":"ContainerStarted","Data":"19a437299e2857729fe8fa680e7d55a64da3abddc27a8aa9180b68f60a4cc5de"}
Sep 05 06:39:01.869147 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:01.868762    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr" event={"ID":"967d2129-2c77-42b1-9dcf-82372113175a","Type":"ContainerStarted","Data":"da81513a6d970dd25709037a31f2a61fe2908e30d80493d21a04d655e8248da2"}
Sep 05 06:39:01.891742 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:01.891696    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr" podStartSLOduration=1.891680951 podStartE2EDuration="1.891680951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:39:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:39:01.890976915 +0000 UTC m=+758.481535871" watchObservedRunningTime="2026-09-05 06:39:01.891680951 +0000 UTC m=+758.482239954"
Sep 05 06:39:06.051763 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:06.051727    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm"]
Sep 05 06:39:06.055306 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:06.055288    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm"
Sep 05 06:39:06.057790 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:06.057770    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slfqj\""
Sep 05 06:39:06.057907 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:06.057793    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 06:39:06.062931 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:06.062835    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm"]
Sep 05 06:39:06.137829 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:06.137798    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7264609-4ca1-4993-b71b-c1dad2affa04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm\" (UID: \"d7264609-4ca1-4993-b71b-c1dad2affa04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm"
Sep 05 06:39:06.238946 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:06.238910    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7264609-4ca1-4993-b71b-c1dad2affa04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm\" (UID: \"d7264609-4ca1-4993-b71b-c1dad2affa04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm"
Sep 05 06:39:06.239561 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:06.239540    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7264609-4ca1-4993-b71b-c1dad2affa04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm\" (UID: \"d7264609-4ca1-4993-b71b-c1dad2affa04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm"
Sep 05 06:39:06.366154 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:06.366079    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm"
Sep 05 06:39:06.638541 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:06.638514    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm"]
Sep 05 06:39:06.640241 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:39:06.640213    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7264609_4ca1_4993_b71b_c1dad2affa04.slice/crio-06ea42dcdd337c8d1fd6d8a3286d2b29ffbfd297ee324d9309203a01c838f25b WatchSource:0}: Error finding container 06ea42dcdd337c8d1fd6d8a3286d2b29ffbfd297ee324d9309203a01c838f25b: Status 404 returned error can't find the container with id 06ea42dcdd337c8d1fd6d8a3286d2b29ffbfd297ee324d9309203a01c838f25b
Sep 05 06:39:06.899578 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:06.899492    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm" event={"ID":"d7264609-4ca1-4993-b71b-c1dad2affa04","Type":"ContainerStarted","Data":"06ea42dcdd337c8d1fd6d8a3286d2b29ffbfd297ee324d9309203a01c838f25b"}
Sep 05 06:39:09.919780 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:09.919698    2559 generic.go:356] "Generic (PLEG): container finished" podID="d7264609-4ca1-4993-b71b-c1dad2affa04" containerID="56f779f12e7412541fb192e0ac9a516dbdfe88f3bf8ea1b47480db21a56b3efe" exitCode=0
Sep 05 06:39:09.920157 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:09.919784    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm" event={"ID":"d7264609-4ca1-4993-b71b-c1dad2affa04","Type":"ContainerDied","Data":"56f779f12e7412541fb192e0ac9a516dbdfe88f3bf8ea1b47480db21a56b3efe"}
Sep 05 06:39:10.292326 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:10.292288    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:39:10.990288 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:10.990252    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:10.990672 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:10.990299    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:11.006583 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:11.006556    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:11.082827 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:11.082803    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm"
Sep 05 06:39:11.181094 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:11.181021    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7264609-4ca1-4993-b71b-c1dad2affa04-maas-api-service-ca\") pod \"d7264609-4ca1-4993-b71b-c1dad2affa04\" (UID: \"d7264609-4ca1-4993-b71b-c1dad2affa04\") "
Sep 05 06:39:11.181468 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:11.181436    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7264609-4ca1-4993-b71b-c1dad2affa04-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d7264609-4ca1-4993-b71b-c1dad2affa04" (UID: "d7264609-4ca1-4993-b71b-c1dad2affa04"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:39:11.282126 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:11.282088    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7264609-4ca1-4993-b71b-c1dad2affa04-maas-api-service-ca\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:39:11.933512 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:11.933488    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm"
Sep 05 06:39:11.933512 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:11.933486    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm" event={"ID":"d7264609-4ca1-4993-b71b-c1dad2affa04","Type":"ContainerDied","Data":"06ea42dcdd337c8d1fd6d8a3286d2b29ffbfd297ee324d9309203a01c838f25b"}
Sep 05 06:39:11.933749 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:11.933542    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06ea42dcdd337c8d1fd6d8a3286d2b29ffbfd297ee324d9309203a01c838f25b"
Sep 05 06:39:11.945833 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:11.945805    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:39:22.583770 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:22.583737    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm"]
Sep 05 06:39:22.585795 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:22.585773    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-b1d1c87d-s5wwm"]
Sep 05 06:39:23.854901 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.854846    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"]
Sep 05 06:39:23.855453 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.855425    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7264609-4ca1-4993-b71b-c1dad2affa04" containerName="revoke-keys"
Sep 05 06:39:23.855453 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.855447    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7264609-4ca1-4993-b71b-c1dad2affa04" containerName="revoke-keys"
Sep 05 06:39:23.855573 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.855524    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7264609-4ca1-4993-b71b-c1dad2affa04" containerName="revoke-keys"
Sep 05 06:39:23.861756 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.861735    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:23.864600 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.864580    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 05 06:39:23.864711 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.864659    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-2da41f75-gw-openshift-default-dockercfg-kr4tx\""
Sep 05 06:39:23.872086 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.872063    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"]
Sep 05 06:39:23.893162 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.893135    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7264609-4ca1-4993-b71b-c1dad2affa04" path="/var/lib/kubelet/pods/d7264609-4ca1-4993-b71b-c1dad2affa04/volumes"
Sep 05 06:39:23.990876 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.990845    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-workload-socket\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:23.991061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.990903    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-istio-envoy\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:23.991061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.990992    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-workload-certs\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:23.991061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.991036    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-istio-data\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:23.991207 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.991109    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea4df8df-d892-45d3-b904-11d3191ba199-istio-token\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:23.991207 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.991146    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-credential-socket\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:23.991207 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.991183    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea4df8df-d892-45d3-b904-11d3191ba199-istiod-ca-cert\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:23.991322 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.991253    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24tnc\" (UniqueName: \"kubernetes.io/projected/ea4df8df-d892-45d3-b904-11d3191ba199-kube-api-access-24tnc\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:23.991322 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:23.991273    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea4df8df-d892-45d3-b904-11d3191ba199-istio-podinfo\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.092242 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.092212    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea4df8df-d892-45d3-b904-11d3191ba199-istio-token\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.092445 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.092259    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-credential-socket\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.092445 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.092273    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea4df8df-d892-45d3-b904-11d3191ba199-istiod-ca-cert\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.092445 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.092325    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24tnc\" (UniqueName: \"kubernetes.io/projected/ea4df8df-d892-45d3-b904-11d3191ba199-kube-api-access-24tnc\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.092445 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.092371    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea4df8df-d892-45d3-b904-11d3191ba199-istio-podinfo\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.092445 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.092443    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-workload-socket\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.092716 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.092474    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-istio-envoy\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.092716 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.092523    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-workload-certs\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.092716 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.092559    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-istio-data\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.092877 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.092755    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-credential-socket\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.092877 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.092845    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-workload-socket\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.093022 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.092925    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-workload-certs\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.093068 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.093037    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-istio-data\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.093104 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.093069    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea4df8df-d892-45d3-b904-11d3191ba199-istiod-ca-cert\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.094693 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.094671    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-istio-envoy\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.094951 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.094932    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea4df8df-d892-45d3-b904-11d3191ba199-istio-podinfo\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.102024 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.102002    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea4df8df-d892-45d3-b904-11d3191ba199-istio-token\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.102505 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.102485    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24tnc\" (UniqueName: \"kubernetes.io/projected/ea4df8df-d892-45d3-b904-11d3191ba199-kube-api-access-24tnc\") pod \"e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") " pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.172463 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.172386    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:24.454998 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:24.454965    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"]
Sep 05 06:39:24.456387 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:39:24.456360    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea4df8df_d892_45d3_b904_11d3191ba199.slice/crio-6762b46bc4abcdf34a491ec2f195228824dc4012cfaecb348f0c18d73c428894 WatchSource:0}: Error finding container 6762b46bc4abcdf34a491ec2f195228824dc4012cfaecb348f0c18d73c428894: Status 404 returned error can't find the container with id 6762b46bc4abcdf34a491ec2f195228824dc4012cfaecb348f0c18d73c428894
Sep 05 06:39:25.005413 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:25.005376    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7" event={"ID":"ea4df8df-d892-45d3-b904-11d3191ba199","Type":"ContainerStarted","Data":"6762b46bc4abcdf34a491ec2f195228824dc4012cfaecb348f0c18d73c428894"}
Sep 05 06:39:25.292985 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:25.292904    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:39:30.448874 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:30.448831    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:39:30.449175 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:30.448917    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:39:30.449175 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:30.448966    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:39:31.042854 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:31.042815    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7" event={"ID":"ea4df8df-d892-45d3-b904-11d3191ba199","Type":"ContainerStarted","Data":"3d1e6d62dec1c663b953a80970028130f2484602a36e8a3c8d8a05e2f26309f5"}
Sep 05 06:39:31.066444 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:31.066397    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7" podStartSLOduration=2.076416291 podStartE2EDuration="8.066382346s" totalImagesPullingTime="5.989966055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:39:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:39:31.064601209 +0000 UTC m=+787.655160159" watchObservedRunningTime="2026-09-05 06:39:31.066382346 +0000 UTC m=+787.656941349"
Sep 05 06:39:31.172696 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:31.172667    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:31.174160 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:31.174134    2559 patch_prober.go:28] interesting pod/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 05 06:39:31.174289 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:31.174183    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7" podUID="ea4df8df-d892-45d3-b904-11d3191ba199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 05 06:39:32.177512 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:32.177475    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:33.054766 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:33.054733    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:33.055570 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:33.055551    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:42.292086 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:42.292046    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:39:53.178128 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:53.178095    2559 generic.go:356] "Generic (PLEG): container finished" podID="ea4df8df-d892-45d3-b904-11d3191ba199" containerID="3d1e6d62dec1c663b953a80970028130f2484602a36e8a3c8d8a05e2f26309f5" exitCode=0
Sep 05 06:39:53.178513 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:53.178147    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7" event={"ID":"ea4df8df-d892-45d3-b904-11d3191ba199","Type":"ContainerDied","Data":"3d1e6d62dec1c663b953a80970028130f2484602a36e8a3c8d8a05e2f26309f5"}
Sep 05 06:39:53.178513 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:53.178479    2559 scope.go:117] "RemoveContainer" containerID="3d1e6d62dec1c663b953a80970028130f2484602a36e8a3c8d8a05e2f26309f5"
Sep 05 06:39:53.180510 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:53.180479    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:39:53.180603 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:53.180546    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:39:53.180603 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:53.180576    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:39:54.185665 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:54.185629    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7" event={"ID":"ea4df8df-d892-45d3-b904-11d3191ba199","Type":"ContainerStarted","Data":"887d8771d90efee1785b476317ce755498c557cc8b4460dbea21005d45844b61"}
Sep 05 06:39:55.173438 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:55.173399    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:55.173638 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:55.173594    2559 patch_prober.go:28] interesting pod/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 05 06:39:55.173720 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:55.173665    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7" podUID="ea4df8df-d892-45d3-b904-11d3191ba199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 05 06:39:56.173672 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:56.173636    2559 patch_prober.go:28] interesting pod/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 05 06:39:56.174040 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:56.173691    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7" podUID="ea4df8df-d892-45d3-b904-11d3191ba199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 05 06:39:57.172820 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:57.172790    2559 patch_prober.go:28] interesting pod/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 05 06:39:57.173038 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:57.172919    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7" podUID="ea4df8df-d892-45d3-b904-11d3191ba199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 05 06:39:57.528065 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:57.528027    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48"]
Sep 05 06:39:57.531448 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:57.531426    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48"
Sep 05 06:39:57.534967 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:57.534943    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slfqj\""
Sep 05 06:39:57.535143 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:57.535050    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 06:39:57.543851 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:57.543829    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48"]
Sep 05 06:39:57.587403 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:57.587374    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a137bca-0b9c-41a2-b528-34a29994d622-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48\" (UID: \"6a137bca-0b9c-41a2-b528-34a29994d622\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48"
Sep 05 06:39:57.688675 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:57.688649    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a137bca-0b9c-41a2-b528-34a29994d622-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48\" (UID: \"6a137bca-0b9c-41a2-b528-34a29994d622\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48"
Sep 05 06:39:57.689281 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:57.689258    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a137bca-0b9c-41a2-b528-34a29994d622-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48\" (UID: \"6a137bca-0b9c-41a2-b528-34a29994d622\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48"
Sep 05 06:39:57.851415 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:57.851335    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48"
Sep 05 06:39:58.132951 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:58.132607    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48"]
Sep 05 06:39:58.173395 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:58.173369    2559 patch_prober.go:28] interesting pod/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Sep 05 06:39:58.173488 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:58.173411    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7" podUID="ea4df8df-d892-45d3-b904-11d3191ba199" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Sep 05 06:39:58.207088 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:58.207061    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48" event={"ID":"6a137bca-0b9c-41a2-b528-34a29994d622","Type":"ContainerStarted","Data":"1e228a91331b93f5ddf37c05fc362df5df6933149650c54566328348eae50acd"}
Sep 05 06:39:59.216773 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:59.216740    2559 generic.go:356] "Generic (PLEG): container finished" podID="6a137bca-0b9c-41a2-b528-34a29994d622" containerID="7318916fcb8f06969bd86f0f62b3c115a66ace7e4b57a0af67bbd9bae547b2d1" exitCode=0
Sep 05 06:39:59.217161 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:59.216837    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48" event={"ID":"6a137bca-0b9c-41a2-b528-34a29994d622","Type":"ContainerDied","Data":"7318916fcb8f06969bd86f0f62b3c115a66ace7e4b57a0af67bbd9bae547b2d1"}
Sep 05 06:39:59.248985 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:59.248948    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:59.249386 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:59.249361    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:39:59.250509 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:39:59.250484    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:40:00.295337 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.295309    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"]
Sep 05 06:40:00.297564 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.297538    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.301373 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.301323    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-8956b5\"/\"default-dockercfg-g4gvw\""
Sep 05 06:40:00.301542 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.301398    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-8956b5\"/\"autores-model-8956b5-kserve-self-signed-certs\""
Sep 05 06:40:00.301542 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.301519    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-8956b5\"/\"openshift-service-ca.crt\""
Sep 05 06:40:00.301778 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.301756    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-8956b5\"/\"kube-root-ca.crt\""
Sep 05 06:40:00.308164 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.308138    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"]
Sep 05 06:40:00.365655 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.365630    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48"
Sep 05 06:40:00.413393 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.413361    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a137bca-0b9c-41a2-b528-34a29994d622-maas-api-service-ca\") pod \"6a137bca-0b9c-41a2-b528-34a29994d622\" (UID: \"6a137bca-0b9c-41a2-b528-34a29994d622\") "
Sep 05 06:40:00.413526 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.413503    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b980f008-b8c3-4802-b379-e31d4b63e167-tls-certs\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.413602 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.413525    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-dshm\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.413602 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.413540    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-home\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.413727 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.413684    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a137bca-0b9c-41a2-b528-34a29994d622-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6a137bca-0b9c-41a2-b528-34a29994d622" (UID: "6a137bca-0b9c-41a2-b528-34a29994d622"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:40:00.413782 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.413721    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c969k\" (UniqueName: \"kubernetes.io/projected/b980f008-b8c3-4802-b379-e31d4b63e167-kube-api-access-c969k\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.413782 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.413744    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-model-cache\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.413782 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.413760    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-tmp-dir\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.413991 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.413842    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a137bca-0b9c-41a2-b528-34a29994d622-maas-api-service-ca\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:00.515003 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.514936    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c969k\" (UniqueName: \"kubernetes.io/projected/b980f008-b8c3-4802-b379-e31d4b63e167-kube-api-access-c969k\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.515003 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.514967    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-model-cache\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.515003 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.514985    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-tmp-dir\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.515228 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.515041    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b980f008-b8c3-4802-b379-e31d4b63e167-tls-certs\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.515228 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.515084    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-dshm\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.515228 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.515132    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-home\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.515383 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.515330    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-tmp-dir\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.515442 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.515390    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-model-cache\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.515488 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.515477    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-home\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.517402 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.517380    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-dshm\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.517610 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.517592    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b980f008-b8c3-4802-b379-e31d4b63e167-tls-certs\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.524291 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.524201    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c969k\" (UniqueName: \"kubernetes.io/projected/b980f008-b8c3-4802-b379-e31d4b63e167-kube-api-access-c969k\") pod \"autores-model-8956b5-kserve-7d6cdd846c-qkf2r\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") " pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.585155 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.585128    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:40:00.608623 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.608596    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:00.884909 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:00.884858    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"]
Sep 05 06:40:00.886527 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:40:00.886501    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb980f008_b8c3_4802_b379_e31d4b63e167.slice/crio-ff6c85e1caaa5f4873638abbdb4d67763a69a07de3eeac7a9d446ce4cef7f25f WatchSource:0}: Error finding container ff6c85e1caaa5f4873638abbdb4d67763a69a07de3eeac7a9d446ce4cef7f25f: Status 404 returned error can't find the container with id ff6c85e1caaa5f4873638abbdb4d67763a69a07de3eeac7a9d446ce4cef7f25f
Sep 05 06:40:01.229842 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:01.229814    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48"
Sep 05 06:40:01.230037 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:01.229836    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48" event={"ID":"6a137bca-0b9c-41a2-b528-34a29994d622","Type":"ContainerDied","Data":"1e228a91331b93f5ddf37c05fc362df5df6933149650c54566328348eae50acd"}
Sep 05 06:40:01.230037 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:01.229908    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e228a91331b93f5ddf37c05fc362df5df6933149650c54566328348eae50acd"
Sep 05 06:40:01.231681 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:01.231652    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r" event={"ID":"b980f008-b8c3-4802-b379-e31d4b63e167","Type":"ContainerStarted","Data":"97acc3ce823cc4b909133ddd44d186091fb6ba1e1cde7f42b9e3723887616e83"}
Sep 05 06:40:01.231805 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:01.231688    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r" event={"ID":"b980f008-b8c3-4802-b379-e31d4b63e167","Type":"ContainerStarted","Data":"ff6c85e1caaa5f4873638abbdb4d67763a69a07de3eeac7a9d446ce4cef7f25f"}
Sep 05 06:40:01.253546 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:01.253491    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r" podStartSLOduration=1.2534782500000001 podStartE2EDuration="1.25347825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:40:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:40:01.250967354 +0000 UTC m=+817.841526297" watchObservedRunningTime="2026-09-05 06:40:01.25347825 +0000 UTC m=+817.844037200"
Sep 05 06:40:04.381651 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:04.381615    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48"]
Sep 05 06:40:04.387017 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:04.386993    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-2da41f75-m5d48"]
Sep 05 06:40:04.525000 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:04.524963    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"]
Sep 05 06:40:04.525234 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:04.525207    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7" podUID="ea4df8df-d892-45d3-b904-11d3191ba199" containerName="istio-proxy" containerID="cri-o://887d8771d90efee1785b476317ce755498c557cc8b4460dbea21005d45844b61" gracePeriod=30
Sep 05 06:40:05.325386 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.325339    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"]
Sep 05 06:40:05.325866 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.325840    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a137bca-0b9c-41a2-b528-34a29994d622" containerName="revoke-keys"
Sep 05 06:40:05.325866 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.325865    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a137bca-0b9c-41a2-b528-34a29994d622" containerName="revoke-keys"
Sep 05 06:40:05.326095 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.325985    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a137bca-0b9c-41a2-b528-34a29994d622" containerName="revoke-keys"
Sep 05 06:40:05.328249 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.328228    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.332100 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.332081    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-bba7714e-openshift-default-dockercfg-kh4gl\""
Sep 05 06:40:05.342701 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.342671    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"]
Sep 05 06:40:05.461087 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.461056    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-workload-socket\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.461404 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.461091    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-token\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.461404 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.461111    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-credential-socket\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.461404 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.461161    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-envoy\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.461404 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.461301    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-podinfo\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.461404 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.461336    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p48b\" (UniqueName: \"kubernetes.io/projected/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-kube-api-access-2p48b\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.461404 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.461355    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istiod-ca-cert\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.461404 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.461402    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-data\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.461627 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.461422    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-workload-certs\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.562473 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.562443    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-podinfo\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.562600 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.562480    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p48b\" (UniqueName: \"kubernetes.io/projected/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-kube-api-access-2p48b\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.562600 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.562497    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istiod-ca-cert\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.562600 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.562521    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-data\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.562728 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.562641    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-workload-certs\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.562728 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.562684    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-workload-socket\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.562728 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.562724    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-token\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.562905 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.562751    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-credential-socket\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.562905 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.562789    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-envoy\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.562905 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.562831    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-data\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.563065 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.562993    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-workload-socket\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.563164 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.563080    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-workload-certs\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.563277 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.563258    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-credential-socket\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.563311 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.563278    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istiod-ca-cert\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.564892 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.564859    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-envoy\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.565026 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.565000    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-podinfo\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.571563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.571541    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-token\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.571677 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.571614    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p48b\" (UniqueName: \"kubernetes.io/projected/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-kube-api-access-2p48b\") pod \"e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") " pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.639440 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.639383    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:05.898834 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.898746    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a137bca-0b9c-41a2-b528-34a29994d622" path="/var/lib/kubelet/pods/6a137bca-0b9c-41a2-b528-34a29994d622/volumes"
Sep 05 06:40:05.918509 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.918481    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"]
Sep 05 06:40:05.919569 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:40:05.919546    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14bd8a6e_7ab5_4f77_a38b_894be0b8d535.slice/crio-dd56bbbaf69adc737e27d6657eeb94bbfc680d7bd51594f35507de417c737b4d WatchSource:0}: Error finding container dd56bbbaf69adc737e27d6657eeb94bbfc680d7bd51594f35507de417c737b4d: Status 404 returned error can't find the container with id dd56bbbaf69adc737e27d6657eeb94bbfc680d7bd51594f35507de417c737b4d
Sep 05 06:40:05.921675 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.921647    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:40:05.921734 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.921706    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:40:05.921774 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:05.921732    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:40:06.266965 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:06.266926    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8" event={"ID":"14bd8a6e-7ab5-4f77-a38b-894be0b8d535","Type":"ContainerStarted","Data":"d2f8ad4a613aea65163174e6a419669cc88b288163e731501b5876872e43b70c"}
Sep 05 06:40:06.267118 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:06.266991    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8" event={"ID":"14bd8a6e-7ab5-4f77-a38b-894be0b8d535","Type":"ContainerStarted","Data":"dd56bbbaf69adc737e27d6657eeb94bbfc680d7bd51594f35507de417c737b4d"}
Sep 05 06:40:06.290727 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:06.290680    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8" podStartSLOduration=1.2906656189999999 podStartE2EDuration="1.290665619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:40:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:40:06.287819046 +0000 UTC m=+822.878378008" watchObservedRunningTime="2026-09-05 06:40:06.290665619 +0000 UTC m=+822.881224570"
Sep 05 06:40:06.640127 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:06.640055    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:06.644878 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:06.644856    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:07.272044 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:07.272018    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:07.272842 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:07.272821    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:09.768865 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.768845    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:40:09.902473 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.902393    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea4df8df-d892-45d3-b904-11d3191ba199-istiod-ca-cert\") pod \"ea4df8df-d892-45d3-b904-11d3191ba199\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") "
Sep 05 06:40:09.902473 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.902430    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-workload-socket\") pod \"ea4df8df-d892-45d3-b904-11d3191ba199\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") "
Sep 05 06:40:09.902473 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.902461    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea4df8df-d892-45d3-b904-11d3191ba199-istio-podinfo\") pod \"ea4df8df-d892-45d3-b904-11d3191ba199\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") "
Sep 05 06:40:09.902727 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.902518    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea4df8df-d892-45d3-b904-11d3191ba199-istio-token\") pod \"ea4df8df-d892-45d3-b904-11d3191ba199\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") "
Sep 05 06:40:09.902727 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.902577    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-istio-envoy\") pod \"ea4df8df-d892-45d3-b904-11d3191ba199\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") "
Sep 05 06:40:09.902727 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.902600    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-istio-data\") pod \"ea4df8df-d892-45d3-b904-11d3191ba199\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") "
Sep 05 06:40:09.902727 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.902627    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-workload-certs\") pod \"ea4df8df-d892-45d3-b904-11d3191ba199\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") "
Sep 05 06:40:09.902727 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.902665    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-credential-socket\") pod \"ea4df8df-d892-45d3-b904-11d3191ba199\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") "
Sep 05 06:40:09.902727 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.902688    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24tnc\" (UniqueName: \"kubernetes.io/projected/ea4df8df-d892-45d3-b904-11d3191ba199-kube-api-access-24tnc\") pod \"ea4df8df-d892-45d3-b904-11d3191ba199\" (UID: \"ea4df8df-d892-45d3-b904-11d3191ba199\") "
Sep 05 06:40:09.903041 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.902823    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea4df8df-d892-45d3-b904-11d3191ba199-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ea4df8df-d892-45d3-b904-11d3191ba199" (UID: "ea4df8df-d892-45d3-b904-11d3191ba199"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:40:09.903105 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.903047    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ea4df8df-d892-45d3-b904-11d3191ba199" (UID: "ea4df8df-d892-45d3-b904-11d3191ba199"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:09.903164 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.903153    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea4df8df-d892-45d3-b904-11d3191ba199-istiod-ca-cert\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:09.903217 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.903171    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-workload-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:09.903217 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.903184    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ea4df8df-d892-45d3-b904-11d3191ba199" (UID: "ea4df8df-d892-45d3-b904-11d3191ba199"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:09.903669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.903507    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-istio-data" (OuterVolumeSpecName: "istio-data") pod "ea4df8df-d892-45d3-b904-11d3191ba199" (UID: "ea4df8df-d892-45d3-b904-11d3191ba199"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:09.903669 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.903579    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ea4df8df-d892-45d3-b904-11d3191ba199" (UID: "ea4df8df-d892-45d3-b904-11d3191ba199"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:09.905248 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.905187    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea4df8df-d892-45d3-b904-11d3191ba199-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ea4df8df-d892-45d3-b904-11d3191ba199" (UID: "ea4df8df-d892-45d3-b904-11d3191ba199"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 06:40:09.905379 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.905321    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea4df8df-d892-45d3-b904-11d3191ba199-kube-api-access-24tnc" (OuterVolumeSpecName: "kube-api-access-24tnc") pod "ea4df8df-d892-45d3-b904-11d3191ba199" (UID: "ea4df8df-d892-45d3-b904-11d3191ba199"). InnerVolumeSpecName "kube-api-access-24tnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:40:09.905379 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.905337    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea4df8df-d892-45d3-b904-11d3191ba199-istio-token" (OuterVolumeSpecName: "istio-token") pod "ea4df8df-d892-45d3-b904-11d3191ba199" (UID: "ea4df8df-d892-45d3-b904-11d3191ba199"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:40:09.905379 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:09.905349    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ea4df8df-d892-45d3-b904-11d3191ba199" (UID: "ea4df8df-d892-45d3-b904-11d3191ba199"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:10.003969 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.003942    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea4df8df-d892-45d3-b904-11d3191ba199-istio-podinfo\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:10.003969 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.003965    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea4df8df-d892-45d3-b904-11d3191ba199-istio-token\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:10.003969 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.003975    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-istio-envoy\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:10.004162 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.003982    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-istio-data\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:10.004162 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.003991    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-workload-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:10.004162 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.004000    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4df8df-d892-45d3-b904-11d3191ba199-credential-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:10.004162 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.004010    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24tnc\" (UniqueName: \"kubernetes.io/projected/ea4df8df-d892-45d3-b904-11d3191ba199-kube-api-access-24tnc\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:10.295986 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.295950    2559 generic.go:356] "Generic (PLEG): container finished" podID="ea4df8df-d892-45d3-b904-11d3191ba199" containerID="887d8771d90efee1785b476317ce755498c557cc8b4460dbea21005d45844b61" exitCode=0
Sep 05 06:40:10.296157 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.296002    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"
Sep 05 06:40:10.296157 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.296030    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7" event={"ID":"ea4df8df-d892-45d3-b904-11d3191ba199","Type":"ContainerDied","Data":"887d8771d90efee1785b476317ce755498c557cc8b4460dbea21005d45844b61"}
Sep 05 06:40:10.296157 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.296119    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7" event={"ID":"ea4df8df-d892-45d3-b904-11d3191ba199","Type":"ContainerDied","Data":"6762b46bc4abcdf34a491ec2f195228824dc4012cfaecb348f0c18d73c428894"}
Sep 05 06:40:10.296157 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.296147    2559 scope.go:117] "RemoveContainer" containerID="887d8771d90efee1785b476317ce755498c557cc8b4460dbea21005d45844b61"
Sep 05 06:40:10.305365 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.305348    2559 scope.go:117] "RemoveContainer" containerID="3d1e6d62dec1c663b953a80970028130f2484602a36e8a3c8d8a05e2f26309f5"
Sep 05 06:40:10.313323 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.313306    2559 scope.go:117] "RemoveContainer" containerID="887d8771d90efee1785b476317ce755498c557cc8b4460dbea21005d45844b61"
Sep 05 06:40:10.313572 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:40:10.313554    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"887d8771d90efee1785b476317ce755498c557cc8b4460dbea21005d45844b61\": container with ID starting with 887d8771d90efee1785b476317ce755498c557cc8b4460dbea21005d45844b61 not found: ID does not exist" containerID="887d8771d90efee1785b476317ce755498c557cc8b4460dbea21005d45844b61"
Sep 05 06:40:10.313621 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.313587    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"887d8771d90efee1785b476317ce755498c557cc8b4460dbea21005d45844b61"} err="failed to get container status \"887d8771d90efee1785b476317ce755498c557cc8b4460dbea21005d45844b61\": rpc error: code = NotFound desc = could not find container \"887d8771d90efee1785b476317ce755498c557cc8b4460dbea21005d45844b61\": container with ID starting with 887d8771d90efee1785b476317ce755498c557cc8b4460dbea21005d45844b61 not found: ID does not exist"
Sep 05 06:40:10.313621 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.313604    2559 scope.go:117] "RemoveContainer" containerID="3d1e6d62dec1c663b953a80970028130f2484602a36e8a3c8d8a05e2f26309f5"
Sep 05 06:40:10.313827 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:40:10.313808    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d1e6d62dec1c663b953a80970028130f2484602a36e8a3c8d8a05e2f26309f5\": container with ID starting with 3d1e6d62dec1c663b953a80970028130f2484602a36e8a3c8d8a05e2f26309f5 not found: ID does not exist" containerID="3d1e6d62dec1c663b953a80970028130f2484602a36e8a3c8d8a05e2f26309f5"
Sep 05 06:40:10.313903 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.313837    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d1e6d62dec1c663b953a80970028130f2484602a36e8a3c8d8a05e2f26309f5"} err="failed to get container status \"3d1e6d62dec1c663b953a80970028130f2484602a36e8a3c8d8a05e2f26309f5\": rpc error: code = NotFound desc = could not find container \"3d1e6d62dec1c663b953a80970028130f2484602a36e8a3c8d8a05e2f26309f5\": container with ID starting with 3d1e6d62dec1c663b953a80970028130f2484602a36e8a3c8d8a05e2f26309f5 not found: ID does not exist"
Sep 05 06:40:10.321299 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.321280    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"]
Sep 05 06:40:10.326999 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.326976    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-2da41f75-gw-openshift-default-7cc6cff8b8-cnds7"]
Sep 05 06:40:10.609158 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.609082    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:10.609303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.609153    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:10.621561 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:10.621539    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:11.312837 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:11.312810    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:11.893132 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:11.893095    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea4df8df-d892-45d3-b904-11d3191ba199" path="/var/lib/kubelet/pods/ea4df8df-d892-45d3-b904-11d3191ba199/volumes"
Sep 05 06:40:17.696102 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:17.696068    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:40:28.403771 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:28.403735    2559 generic.go:356] "Generic (PLEG): container finished" podID="14bd8a6e-7ab5-4f77-a38b-894be0b8d535" containerID="d2f8ad4a613aea65163174e6a419669cc88b288163e731501b5876872e43b70c" exitCode=0
Sep 05 06:40:28.404293 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:28.403818    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8" event={"ID":"14bd8a6e-7ab5-4f77-a38b-894be0b8d535","Type":"ContainerDied","Data":"d2f8ad4a613aea65163174e6a419669cc88b288163e731501b5876872e43b70c"}
Sep 05 06:40:28.404371 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:28.404308    2559 scope.go:117] "RemoveContainer" containerID="d2f8ad4a613aea65163174e6a419669cc88b288163e731501b5876872e43b70c"
Sep 05 06:40:28.405525 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:28.405486    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:40:28.405630 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:28.405563    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:40:28.405630 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:28.405603    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:40:29.414100 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:29.414068    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8" event={"ID":"14bd8a6e-7ab5-4f77-a38b-894be0b8d535","Type":"ContainerStarted","Data":"46b263466f2d65f632acdaf15816cb90716ea12c377f227263207b1baf7e4e4c"}
Sep 05 06:40:29.640673 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:29.640575    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:29.640673 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:29.640619    2559 patch_prober.go:28] interesting pod/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 05 06:40:29.640955 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:29.640674    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8" podUID="14bd8a6e-7ab5-4f77-a38b-894be0b8d535" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 05 06:40:30.598652 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:30.598612    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"]
Sep 05 06:40:30.640479 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:30.640442    2559 patch_prober.go:28] interesting pod/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 05 06:40:30.640764 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:30.640721    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8" podUID="14bd8a6e-7ab5-4f77-a38b-894be0b8d535" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 05 06:40:30.751809 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:30.751778    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8956b5-cf296"]
Sep 05 06:40:30.752527 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:30.752449    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea4df8df-d892-45d3-b904-11d3191ba199" containerName="istio-proxy"
Sep 05 06:40:30.752527 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:30.752478    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea4df8df-d892-45d3-b904-11d3191ba199" containerName="istio-proxy"
Sep 05 06:40:30.752527 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:30.752495    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea4df8df-d892-45d3-b904-11d3191ba199" containerName="istio-proxy"
Sep 05 06:40:30.752527 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:30.752504    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea4df8df-d892-45d3-b904-11d3191ba199" containerName="istio-proxy"
Sep 05 06:40:30.752865 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:30.752595    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea4df8df-d892-45d3-b904-11d3191ba199" containerName="istio-proxy"
Sep 05 06:40:30.754992 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:30.754971    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8956b5-cf296"
Sep 05 06:40:30.757711 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:30.757692    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slfqj\""
Sep 05 06:40:30.757805 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:30.757692    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 06:40:30.766298 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:30.766277    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8956b5-cf296"]
Sep 05 06:40:30.898733 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:30.898648    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efcd8292-cf16-4153-966a-fe5def3f4df2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-8956b5-cf296\" (UID: \"efcd8292-cf16-4153-966a-fe5def3f4df2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8956b5-cf296"
Sep 05 06:40:30.999817 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:30.999772    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efcd8292-cf16-4153-966a-fe5def3f4df2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-8956b5-cf296\" (UID: \"efcd8292-cf16-4153-966a-fe5def3f4df2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8956b5-cf296"
Sep 05 06:40:31.000546 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:31.000524    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efcd8292-cf16-4153-966a-fe5def3f4df2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-8956b5-cf296\" (UID: \"efcd8292-cf16-4153-966a-fe5def3f4df2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8956b5-cf296"
Sep 05 06:40:31.065609 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:31.065529    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8956b5-cf296"
Sep 05 06:40:31.090349 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:31.090311    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"]
Sep 05 06:40:31.361911 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:31.361754    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8956b5-cf296"]
Sep 05 06:40:31.365240 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:40:31.365210    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefcd8292_cf16_4153_966a_fe5def3f4df2.slice/crio-fcadaae89ad9fc5a2340eb1c1e3e400bd5a0d1525a1ec7afe450c5ef48463c29 WatchSource:0}: Error finding container fcadaae89ad9fc5a2340eb1c1e3e400bd5a0d1525a1ec7afe450c5ef48463c29: Status 404 returned error can't find the container with id fcadaae89ad9fc5a2340eb1c1e3e400bd5a0d1525a1ec7afe450c5ef48463c29
Sep 05 06:40:31.431375 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:31.431340    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8956b5-cf296" event={"ID":"efcd8292-cf16-4153-966a-fe5def3f4df2","Type":"ContainerStarted","Data":"fcadaae89ad9fc5a2340eb1c1e3e400bd5a0d1525a1ec7afe450c5ef48463c29"}
Sep 05 06:40:31.640530 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:31.640269    2559 patch_prober.go:28] interesting pod/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 05 06:40:31.640530 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:31.640320    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8" podUID="14bd8a6e-7ab5-4f77-a38b-894be0b8d535" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 05 06:40:32.442961 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:32.442913    2559 generic.go:356] "Generic (PLEG): container finished" podID="efcd8292-cf16-4153-966a-fe5def3f4df2" containerID="3e3cccfb3bfa9b0f467877649aa48165cb920b5285a655503ca9bfad46506a85" exitCode=0
Sep 05 06:40:32.443143 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:32.443035    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8956b5-cf296" event={"ID":"efcd8292-cf16-4153-966a-fe5def3f4df2","Type":"ContainerDied","Data":"3e3cccfb3bfa9b0f467877649aa48165cb920b5285a655503ca9bfad46506a85"}
Sep 05 06:40:32.640736 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:32.640637    2559 patch_prober.go:28] interesting pod/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 05 06:40:32.640736 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:32.640706    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8" podUID="14bd8a6e-7ab5-4f77-a38b-894be0b8d535" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 05 06:40:33.605730 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:33.605709    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8956b5-cf296"
Sep 05 06:40:33.645375 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:33.644971    2559 patch_prober.go:28] interesting pod/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 05 06:40:33.645375 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:33.645032    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8" podUID="14bd8a6e-7ab5-4f77-a38b-894be0b8d535" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 05 06:40:33.733920 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:33.733797    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efcd8292-cf16-4153-966a-fe5def3f4df2-maas-api-service-ca\") pod \"efcd8292-cf16-4153-966a-fe5def3f4df2\" (UID: \"efcd8292-cf16-4153-966a-fe5def3f4df2\") "
Sep 05 06:40:33.747966 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:33.746532    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efcd8292-cf16-4153-966a-fe5def3f4df2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "efcd8292-cf16-4153-966a-fe5def3f4df2" (UID: "efcd8292-cf16-4153-966a-fe5def3f4df2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:40:33.847617 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:33.847585    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/efcd8292-cf16-4153-966a-fe5def3f4df2-maas-api-service-ca\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:34.457756 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:34.457728    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8956b5-cf296"
Sep 05 06:40:34.457963 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:34.457711    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8956b5-cf296" event={"ID":"efcd8292-cf16-4153-966a-fe5def3f4df2","Type":"ContainerDied","Data":"fcadaae89ad9fc5a2340eb1c1e3e400bd5a0d1525a1ec7afe450c5ef48463c29"}
Sep 05 06:40:34.457963 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:34.457915    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fcadaae89ad9fc5a2340eb1c1e3e400bd5a0d1525a1ec7afe450c5ef48463c29"
Sep 05 06:40:34.644170 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:34.644133    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:34.644687 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:34.644663    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:34.645334 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:34.645308    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:39.892893 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:39.892850    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:40:42.149468 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:42.149436    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8956b5-cf296"]
Sep 05 06:40:42.158692 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:42.158669    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8956b5-cf296"]
Sep 05 06:40:43.891396 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:43.891364    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efcd8292-cf16-4153-966a-fe5def3f4df2" path="/var/lib/kubelet/pods/efcd8292-cf16-4153-966a-fe5def3f4df2/volumes"
Sep 05 06:40:45.618323 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:45.618262    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r" podUID="b980f008-b8c3-4802-b379-e31d4b63e167" containerName="main" containerID="cri-o://97acc3ce823cc4b909133ddd44d186091fb6ba1e1cde7f42b9e3723887616e83" gracePeriod=45
Sep 05 06:40:45.870156 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:45.870100    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:45.941152 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:45.941119    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b980f008-b8c3-4802-b379-e31d4b63e167-tls-certs\") pod \"b980f008-b8c3-4802-b379-e31d4b63e167\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") "
Sep 05 06:40:45.941305 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:45.941169    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-dshm\") pod \"b980f008-b8c3-4802-b379-e31d4b63e167\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") "
Sep 05 06:40:45.941305 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:45.941216    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c969k\" (UniqueName: \"kubernetes.io/projected/b980f008-b8c3-4802-b379-e31d4b63e167-kube-api-access-c969k\") pod \"b980f008-b8c3-4802-b379-e31d4b63e167\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") "
Sep 05 06:40:45.941305 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:45.941242    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-home\") pod \"b980f008-b8c3-4802-b379-e31d4b63e167\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") "
Sep 05 06:40:45.941463 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:45.941319    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-model-cache\") pod \"b980f008-b8c3-4802-b379-e31d4b63e167\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") "
Sep 05 06:40:45.941463 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:45.941341    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-tmp-dir\") pod \"b980f008-b8c3-4802-b379-e31d4b63e167\" (UID: \"b980f008-b8c3-4802-b379-e31d4b63e167\") "
Sep 05 06:40:45.941629 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:45.941546    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-home" (OuterVolumeSpecName: "home") pod "b980f008-b8c3-4802-b379-e31d4b63e167" (UID: "b980f008-b8c3-4802-b379-e31d4b63e167"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:45.941809 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:45.941635    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-model-cache" (OuterVolumeSpecName: "model-cache") pod "b980f008-b8c3-4802-b379-e31d4b63e167" (UID: "b980f008-b8c3-4802-b379-e31d4b63e167"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:45.941809 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:45.941662    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b980f008-b8c3-4802-b379-e31d4b63e167" (UID: "b980f008-b8c3-4802-b379-e31d4b63e167"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:45.943284 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:45.943260    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-dshm" (OuterVolumeSpecName: "dshm") pod "b980f008-b8c3-4802-b379-e31d4b63e167" (UID: "b980f008-b8c3-4802-b379-e31d4b63e167"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:45.943374 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:45.943352    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b980f008-b8c3-4802-b379-e31d4b63e167-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b980f008-b8c3-4802-b379-e31d4b63e167" (UID: "b980f008-b8c3-4802-b379-e31d4b63e167"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:40:45.943421 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:45.943352    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b980f008-b8c3-4802-b379-e31d4b63e167-kube-api-access-c969k" (OuterVolumeSpecName: "kube-api-access-c969k") pod "b980f008-b8c3-4802-b379-e31d4b63e167" (UID: "b980f008-b8c3-4802-b379-e31d4b63e167"). InnerVolumeSpecName "kube-api-access-c969k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:40:46.042042 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.042017    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-model-cache\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:46.042042 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.042041    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-tmp-dir\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:46.042163 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.042050    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b980f008-b8c3-4802-b379-e31d4b63e167-tls-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:46.042163 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.042058    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-dshm\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:46.042163 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.042066    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c969k\" (UniqueName: \"kubernetes.io/projected/b980f008-b8c3-4802-b379-e31d4b63e167-kube-api-access-c969k\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:46.042163 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.042075    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b980f008-b8c3-4802-b379-e31d4b63e167-home\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:46.111697 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.111665    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr" podUID="967d2129-2c77-42b1-9dcf-82372113175a" containerName="main" containerID="cri-o://19a437299e2857729fe8fa680e7d55a64da3abddc27a8aa9180b68f60a4cc5de" gracePeriod=45
Sep 05 06:40:46.339103 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.339077    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:40:46.444519 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.444487    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-tmp-dir\") pod \"967d2129-2c77-42b1-9dcf-82372113175a\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") "
Sep 05 06:40:46.444684 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.444533    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-dshm\") pod \"967d2129-2c77-42b1-9dcf-82372113175a\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") "
Sep 05 06:40:46.444684 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.444554    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/967d2129-2c77-42b1-9dcf-82372113175a-tls-certs\") pod \"967d2129-2c77-42b1-9dcf-82372113175a\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") "
Sep 05 06:40:46.444684 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.444602    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-model-cache\") pod \"967d2129-2c77-42b1-9dcf-82372113175a\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") "
Sep 05 06:40:46.444684 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.444639    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-home\") pod \"967d2129-2c77-42b1-9dcf-82372113175a\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") "
Sep 05 06:40:46.444684 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.444655    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp5sm\" (UniqueName: \"kubernetes.io/projected/967d2129-2c77-42b1-9dcf-82372113175a-kube-api-access-gp5sm\") pod \"967d2129-2c77-42b1-9dcf-82372113175a\" (UID: \"967d2129-2c77-42b1-9dcf-82372113175a\") "
Sep 05 06:40:46.444978 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.444777    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "967d2129-2c77-42b1-9dcf-82372113175a" (UID: "967d2129-2c77-42b1-9dcf-82372113175a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:46.444978 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.444862    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-model-cache" (OuterVolumeSpecName: "model-cache") pod "967d2129-2c77-42b1-9dcf-82372113175a" (UID: "967d2129-2c77-42b1-9dcf-82372113175a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:46.444978 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.444878    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-home" (OuterVolumeSpecName: "home") pod "967d2129-2c77-42b1-9dcf-82372113175a" (UID: "967d2129-2c77-42b1-9dcf-82372113175a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:46.444978 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.444975    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-model-cache\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:46.445177 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.444992    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-home\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:46.445177 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.445006    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-tmp-dir\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:46.446722 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.446697    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-dshm" (OuterVolumeSpecName: "dshm") pod "967d2129-2c77-42b1-9dcf-82372113175a" (UID: "967d2129-2c77-42b1-9dcf-82372113175a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:46.446837 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.446780    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/967d2129-2c77-42b1-9dcf-82372113175a-kube-api-access-gp5sm" (OuterVolumeSpecName: "kube-api-access-gp5sm") pod "967d2129-2c77-42b1-9dcf-82372113175a" (UID: "967d2129-2c77-42b1-9dcf-82372113175a"). InnerVolumeSpecName "kube-api-access-gp5sm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:40:46.446837 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.446789    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/967d2129-2c77-42b1-9dcf-82372113175a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "967d2129-2c77-42b1-9dcf-82372113175a" (UID: "967d2129-2c77-42b1-9dcf-82372113175a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:40:46.525686 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.525616    2559 generic.go:356] "Generic (PLEG): container finished" podID="b980f008-b8c3-4802-b379-e31d4b63e167" containerID="97acc3ce823cc4b909133ddd44d186091fb6ba1e1cde7f42b9e3723887616e83" exitCode=0
Sep 05 06:40:46.525686 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.525667    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"
Sep 05 06:40:46.525854 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.525699    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r" event={"ID":"b980f008-b8c3-4802-b379-e31d4b63e167","Type":"ContainerDied","Data":"97acc3ce823cc4b909133ddd44d186091fb6ba1e1cde7f42b9e3723887616e83"}
Sep 05 06:40:46.525854 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.525740    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r" event={"ID":"b980f008-b8c3-4802-b379-e31d4b63e167","Type":"ContainerDied","Data":"ff6c85e1caaa5f4873638abbdb4d67763a69a07de3eeac7a9d446ce4cef7f25f"}
Sep 05 06:40:46.525854 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.525763    2559 scope.go:117] "RemoveContainer" containerID="97acc3ce823cc4b909133ddd44d186091fb6ba1e1cde7f42b9e3723887616e83"
Sep 05 06:40:46.527652 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.527619    2559 generic.go:356] "Generic (PLEG): container finished" podID="967d2129-2c77-42b1-9dcf-82372113175a" containerID="19a437299e2857729fe8fa680e7d55a64da3abddc27a8aa9180b68f60a4cc5de" exitCode=0
Sep 05 06:40:46.527765 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.527669    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"
Sep 05 06:40:46.527765 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.527758    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr" event={"ID":"967d2129-2c77-42b1-9dcf-82372113175a","Type":"ContainerDied","Data":"19a437299e2857729fe8fa680e7d55a64da3abddc27a8aa9180b68f60a4cc5de"}
Sep 05 06:40:46.527910 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.527791    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr" event={"ID":"967d2129-2c77-42b1-9dcf-82372113175a","Type":"ContainerDied","Data":"da81513a6d970dd25709037a31f2a61fe2908e30d80493d21a04d655e8248da2"}
Sep 05 06:40:46.535204 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.535184    2559 scope.go:117] "RemoveContainer" containerID="97acc3ce823cc4b909133ddd44d186091fb6ba1e1cde7f42b9e3723887616e83"
Sep 05 06:40:46.535554 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:40:46.535527    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97acc3ce823cc4b909133ddd44d186091fb6ba1e1cde7f42b9e3723887616e83\": container with ID starting with 97acc3ce823cc4b909133ddd44d186091fb6ba1e1cde7f42b9e3723887616e83 not found: ID does not exist" containerID="97acc3ce823cc4b909133ddd44d186091fb6ba1e1cde7f42b9e3723887616e83"
Sep 05 06:40:46.535701 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.535600    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97acc3ce823cc4b909133ddd44d186091fb6ba1e1cde7f42b9e3723887616e83"} err="failed to get container status \"97acc3ce823cc4b909133ddd44d186091fb6ba1e1cde7f42b9e3723887616e83\": rpc error: code = NotFound desc = could not find container \"97acc3ce823cc4b909133ddd44d186091fb6ba1e1cde7f42b9e3723887616e83\": container with ID starting with 97acc3ce823cc4b909133ddd44d186091fb6ba1e1cde7f42b9e3723887616e83 not found: ID does not exist"
Sep 05 06:40:46.535701 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.535636    2559 scope.go:117] "RemoveContainer" containerID="19a437299e2857729fe8fa680e7d55a64da3abddc27a8aa9180b68f60a4cc5de"
Sep 05 06:40:46.546017 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.545997    2559 scope.go:117] "RemoveContainer" containerID="19a437299e2857729fe8fa680e7d55a64da3abddc27a8aa9180b68f60a4cc5de"
Sep 05 06:40:46.546323 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.546303    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp5sm\" (UniqueName: \"kubernetes.io/projected/967d2129-2c77-42b1-9dcf-82372113175a-kube-api-access-gp5sm\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:46.546323 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.546322    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/967d2129-2c77-42b1-9dcf-82372113175a-dshm\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:46.546437 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.546331    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/967d2129-2c77-42b1-9dcf-82372113175a-tls-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:46.546437 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:40:46.546361    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19a437299e2857729fe8fa680e7d55a64da3abddc27a8aa9180b68f60a4cc5de\": container with ID starting with 19a437299e2857729fe8fa680e7d55a64da3abddc27a8aa9180b68f60a4cc5de not found: ID does not exist" containerID="19a437299e2857729fe8fa680e7d55a64da3abddc27a8aa9180b68f60a4cc5de"
Sep 05 06:40:46.546437 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.546377    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19a437299e2857729fe8fa680e7d55a64da3abddc27a8aa9180b68f60a4cc5de"} err="failed to get container status \"19a437299e2857729fe8fa680e7d55a64da3abddc27a8aa9180b68f60a4cc5de\": rpc error: code = NotFound desc = could not find container \"19a437299e2857729fe8fa680e7d55a64da3abddc27a8aa9180b68f60a4cc5de\": container with ID starting with 19a437299e2857729fe8fa680e7d55a64da3abddc27a8aa9180b68f60a4cc5de not found: ID does not exist"
Sep 05 06:40:46.551912 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.551875    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"]
Sep 05 06:40:46.554123 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.554105    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-c4e74764-kserve-8559768645-qbnqr"]
Sep 05 06:40:46.565663 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.565642    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"]
Sep 05 06:40:46.569631 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:46.569613    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-8956b5/autores-model-8956b5-kserve-7d6cdd846c-qkf2r"]
Sep 05 06:40:47.891691 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:47.891657    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="967d2129-2c77-42b1-9dcf-82372113175a" path="/var/lib/kubelet/pods/967d2129-2c77-42b1-9dcf-82372113175a/volumes"
Sep 05 06:40:47.892170 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:47.892154    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b980f008-b8c3-4802-b379-e31d4b63e167" path="/var/lib/kubelet/pods/b980f008-b8c3-4802-b379-e31d4b63e167/volumes"
Sep 05 06:40:50.170130 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.170099    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"]
Sep 05 06:40:50.170572 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.170446    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="967d2129-2c77-42b1-9dcf-82372113175a" containerName="main"
Sep 05 06:40:50.170572 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.170457    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="967d2129-2c77-42b1-9dcf-82372113175a" containerName="main"
Sep 05 06:40:50.170572 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.170471    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efcd8292-cf16-4153-966a-fe5def3f4df2" containerName="revoke-keys"
Sep 05 06:40:50.170572 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.170477    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="efcd8292-cf16-4153-966a-fe5def3f4df2" containerName="revoke-keys"
Sep 05 06:40:50.170572 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.170485    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b980f008-b8c3-4802-b379-e31d4b63e167" containerName="main"
Sep 05 06:40:50.170572 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.170491    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b980f008-b8c3-4802-b379-e31d4b63e167" containerName="main"
Sep 05 06:40:50.170572 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.170551    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="967d2129-2c77-42b1-9dcf-82372113175a" containerName="main"
Sep 05 06:40:50.170572 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.170558    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="efcd8292-cf16-4153-966a-fe5def3f4df2" containerName="revoke-keys"
Sep 05 06:40:50.170572 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.170568    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea4df8df-d892-45d3-b904-11d3191ba199" containerName="istio-proxy"
Sep 05 06:40:50.170572 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.170574    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b980f008-b8c3-4802-b379-e31d4b63e167" containerName="main"
Sep 05 06:40:50.173612 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.173597    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.176474 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.176450    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-9f698c-openshift-default-dockercfg-rfkxr\""
Sep 05 06:40:50.186836 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.186817    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"]
Sep 05 06:40:50.276477 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.276442    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-workload-socket\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.276638 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.276503    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbl6l\" (UniqueName: \"kubernetes.io/projected/031e5d01-3e72-4e83-b588-7a2c626461bd-kube-api-access-cbl6l\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.276638 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.276578    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-podinfo\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.276638 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.276619    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-workload-certs\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.276745 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.276654    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-data\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.276779 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.276741    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-credential-socket\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.276779 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.276772    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-token\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.276845 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.276791    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-envoy\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.276905 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.276864    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/031e5d01-3e72-4e83-b588-7a2c626461bd-istiod-ca-cert\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.377222 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.377193    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-workload-socket\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.377222 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.377228    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbl6l\" (UniqueName: \"kubernetes.io/projected/031e5d01-3e72-4e83-b588-7a2c626461bd-kube-api-access-cbl6l\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.377450 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.377253    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-podinfo\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.377450 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.377274    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-workload-certs\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.377450 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.377295    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-data\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.377450 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.377331    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-credential-socket\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.377450 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.377351    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-token\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.377450 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.377366    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-envoy\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.377450 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.377395    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/031e5d01-3e72-4e83-b588-7a2c626461bd-istiod-ca-cert\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.377804 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.377628    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-workload-socket\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.377804 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.377646    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-workload-certs\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.377804 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.377718    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-data\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.377804 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.377747    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-credential-socket\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.378178 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.378159    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/031e5d01-3e72-4e83-b588-7a2c626461bd-istiod-ca-cert\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.379588 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.379568    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-envoy\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.379704 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.379686    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-podinfo\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.388103 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.388075    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbl6l\" (UniqueName: \"kubernetes.io/projected/031e5d01-3e72-4e83-b588-7a2c626461bd-kube-api-access-cbl6l\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.388237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.388220    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-token\") pod \"e2e-explicit-9f698c-openshift-default-7f69567688-whchg\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") " pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.484142 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.484111    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:50.789397 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.789365    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"]
Sep 05 06:40:50.790434 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:40:50.790411    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod031e5d01_3e72_4e83_b588_7a2c626461bd.slice/crio-a6e8c5d01d81e88cdf8a9d3fa76f61e9374f9547e37d8109c6a67de4f339d1cb WatchSource:0}: Error finding container a6e8c5d01d81e88cdf8a9d3fa76f61e9374f9547e37d8109c6a67de4f339d1cb: Status 404 returned error can't find the container with id a6e8c5d01d81e88cdf8a9d3fa76f61e9374f9547e37d8109c6a67de4f339d1cb
Sep 05 06:40:50.792692 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.792660    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:40:50.792784 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.792726    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:40:50.792784 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:50.792754    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:40:51.556484 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:51.556457    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"]
Sep 05 06:40:51.556867 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:51.556650    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8" podUID="14bd8a6e-7ab5-4f77-a38b-894be0b8d535" containerName="istio-proxy" containerID="cri-o://46b263466f2d65f632acdaf15816cb90716ea12c377f227263207b1baf7e4e4c" gracePeriod=30
Sep 05 06:40:51.562288 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:51.562262    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg" event={"ID":"031e5d01-3e72-4e83-b588-7a2c626461bd","Type":"ContainerStarted","Data":"a9ad4fa04911b711b270c4ea5bbe5fa2b2b13fb3abaed2f1d042f72fa8d58c4f"}
Sep 05 06:40:51.562420 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:51.562313    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg" event={"ID":"031e5d01-3e72-4e83-b588-7a2c626461bd","Type":"ContainerStarted","Data":"a6e8c5d01d81e88cdf8a9d3fa76f61e9374f9547e37d8109c6a67de4f339d1cb"}
Sep 05 06:40:51.605796 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:51.605745    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg" podStartSLOduration=1.6057300190000001 podStartE2EDuration="1.605730019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:40:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:40:51.60441348 +0000 UTC m=+868.194972430" watchObservedRunningTime="2026-09-05 06:40:51.605730019 +0000 UTC m=+868.196288981"
Sep 05 06:40:52.374810 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.374775    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"]
Sep 05 06:40:52.378492 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.378464    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.380975 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.380952    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-13503c3c-openshift-default-dockercfg-w9gtk\""
Sep 05 06:40:52.392837 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.392815    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"]
Sep 05 06:40:52.484603 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.484574    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:52.490297 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.490273    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:52.493566 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.493542    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-token\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.493681 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.493598    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-workload-certs\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.493681 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.493626    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istiod-ca-cert\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.493759 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.493696    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-credential-socket\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.493759 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.493721    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-podinfo\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.493858 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.493838    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-workload-socket\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.493922 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.493865    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-data\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.493922 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.493916    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh6bb\" (UniqueName: \"kubernetes.io/projected/56bc7df6-bf2e-4c4d-ae61-6c9936159344-kube-api-access-gh6bb\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.493995 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.493946    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-envoy\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.567614 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.567591    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:52.568354 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.568337    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:40:52.595280 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.595253    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-token\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.595408 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.595304    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-workload-certs\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.595408 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.595335    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istiod-ca-cert\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.595408 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.595352    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-credential-socket\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.595408 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.595370    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-podinfo\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.595642 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.595479    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-workload-socket\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.595642 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.595502    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-data\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.595642 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.595538    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh6bb\" (UniqueName: \"kubernetes.io/projected/56bc7df6-bf2e-4c4d-ae61-6c9936159344-kube-api-access-gh6bb\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.595642 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.595576    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-envoy\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.595858 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.595816    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-credential-socket\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.596125 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.596099    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-workload-certs\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.596272 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.596194    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istiod-ca-cert\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.596372 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.596209    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-data\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.596469 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.596402    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-workload-socket\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.598259 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.598231    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-envoy\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.598433 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.598405    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-podinfo\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.606893 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.606856    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-token\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.607139 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.607115    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh6bb\" (UniqueName: \"kubernetes.io/projected/56bc7df6-bf2e-4c4d-ae61-6c9936159344-kube-api-access-gh6bb\") pod \"e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") " pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.688980 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.688955    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:52.969461 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.969428    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"]
Sep 05 06:40:52.970596 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:40:52.970570    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56bc7df6_bf2e_4c4d_ae61_6c9936159344.slice/crio-aa0d0cedd96c171f429a4456d91cf7be61e53b5137731e2f73f17c2a32b0fdfb WatchSource:0}: Error finding container aa0d0cedd96c171f429a4456d91cf7be61e53b5137731e2f73f17c2a32b0fdfb: Status 404 returned error can't find the container with id aa0d0cedd96c171f429a4456d91cf7be61e53b5137731e2f73f17c2a32b0fdfb
Sep 05 06:40:52.973222 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.973188    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:40:52.973308 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.973267    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:40:52.973308 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:52.973298    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:40:53.574428 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:53.574386    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd" event={"ID":"56bc7df6-bf2e-4c4d-ae61-6c9936159344","Type":"ContainerStarted","Data":"459806f42b9ce6b4a2437d05a61b1ee6765741b3ae19a84794dac812e4b90a6a"}
Sep 05 06:40:53.574800 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:53.574454    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd" event={"ID":"56bc7df6-bf2e-4c4d-ae61-6c9936159344","Type":"ContainerStarted","Data":"aa0d0cedd96c171f429a4456d91cf7be61e53b5137731e2f73f17c2a32b0fdfb"}
Sep 05 06:40:53.604251 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:53.604205    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd" podStartSLOduration=1.60418943 podStartE2EDuration="1.60418943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:40:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:40:53.603074722 +0000 UTC m=+870.193633672" watchObservedRunningTime="2026-09-05 06:40:53.60418943 +0000 UTC m=+870.194748379"
Sep 05 06:40:53.689718 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:53.689695    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:54.694449 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:54.694419    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:55.585529 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:55.585499    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:55.586560 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:55.586539    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:40:56.788043 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.788008    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:40:56.818385 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.818334    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:56.934627 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.934588    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istiod-ca-cert\") pod \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") "
Sep 05 06:40:56.934627 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.934625    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-data\") pod \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") "
Sep 05 06:40:56.934916 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.934663    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-workload-certs\") pod \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") "
Sep 05 06:40:56.934916 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.934690    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-credential-socket\") pod \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") "
Sep 05 06:40:56.934916 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.934711    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-token\") pod \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") "
Sep 05 06:40:56.934916 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.934727    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2p48b\" (UniqueName: \"kubernetes.io/projected/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-kube-api-access-2p48b\") pod \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") "
Sep 05 06:40:56.934916 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.934768    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-podinfo\") pod \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") "
Sep 05 06:40:56.934916 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.934812    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-workload-socket\") pod \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") "
Sep 05 06:40:56.934916 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.934838    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-data" (OuterVolumeSpecName: "istio-data") pod "14bd8a6e-7ab5-4f77-a38b-894be0b8d535" (UID: "14bd8a6e-7ab5-4f77-a38b-894be0b8d535"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:56.935282 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.934929    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-envoy\") pod \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\" (UID: \"14bd8a6e-7ab5-4f77-a38b-894be0b8d535\") "
Sep 05 06:40:56.935282 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.935041    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "14bd8a6e-7ab5-4f77-a38b-894be0b8d535" (UID: "14bd8a6e-7ab5-4f77-a38b-894be0b8d535"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:40:56.935282 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.935052    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "14bd8a6e-7ab5-4f77-a38b-894be0b8d535" (UID: "14bd8a6e-7ab5-4f77-a38b-894be0b8d535"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:56.935282 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.935195    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "14bd8a6e-7ab5-4f77-a38b-894be0b8d535" (UID: "14bd8a6e-7ab5-4f77-a38b-894be0b8d535"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:56.935764 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.935325    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istiod-ca-cert\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:56.935764 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.935346    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-data\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:56.935764 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.935358    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-credential-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:56.935764 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.935372    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-workload-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:56.935764 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.935580    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "14bd8a6e-7ab5-4f77-a38b-894be0b8d535" (UID: "14bd8a6e-7ab5-4f77-a38b-894be0b8d535"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:56.937013 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.936989    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-kube-api-access-2p48b" (OuterVolumeSpecName: "kube-api-access-2p48b") pod "14bd8a6e-7ab5-4f77-a38b-894be0b8d535" (UID: "14bd8a6e-7ab5-4f77-a38b-894be0b8d535"). InnerVolumeSpecName "kube-api-access-2p48b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:40:56.937122 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.937080    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-token" (OuterVolumeSpecName: "istio-token") pod "14bd8a6e-7ab5-4f77-a38b-894be0b8d535" (UID: "14bd8a6e-7ab5-4f77-a38b-894be0b8d535"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:40:56.937463 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.937440    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "14bd8a6e-7ab5-4f77-a38b-894be0b8d535" (UID: "14bd8a6e-7ab5-4f77-a38b-894be0b8d535"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:40:56.937513 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:56.937443    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "14bd8a6e-7ab5-4f77-a38b-894be0b8d535" (UID: "14bd8a6e-7ab5-4f77-a38b-894be0b8d535"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 06:40:57.036068 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.036042    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-token\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:57.036172 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.036070    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p48b\" (UniqueName: \"kubernetes.io/projected/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-kube-api-access-2p48b\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:57.036172 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.036087    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-podinfo\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:57.036172 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.036104    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-istio-envoy\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:57.036172 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.036118    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14bd8a6e-7ab5-4f77-a38b-894be0b8d535-workload-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:40:57.170624 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.170548    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"]
Sep 05 06:40:57.170982 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.170967    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14bd8a6e-7ab5-4f77-a38b-894be0b8d535" containerName="istio-proxy"
Sep 05 06:40:57.171032 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.170984    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="14bd8a6e-7ab5-4f77-a38b-894be0b8d535" containerName="istio-proxy"
Sep 05 06:40:57.171032 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.170992    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14bd8a6e-7ab5-4f77-a38b-894be0b8d535" containerName="istio-proxy"
Sep 05 06:40:57.171032 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.171000    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="14bd8a6e-7ab5-4f77-a38b-894be0b8d535" containerName="istio-proxy"
Sep 05 06:40:57.171123 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.171056    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="14bd8a6e-7ab5-4f77-a38b-894be0b8d535" containerName="istio-proxy"
Sep 05 06:40:57.171123 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.171066    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="14bd8a6e-7ab5-4f77-a38b-894be0b8d535" containerName="istio-proxy"
Sep 05 06:40:57.175303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.175282    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.177908 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.177873    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7080a4\"/\"default-dockercfg-jbnq4\""
Sep 05 06:40:57.178034 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.177928    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7080a4\"/\"openshift-service-ca.crt\""
Sep 05 06:40:57.178034 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.177971    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7080a4\"/\"kube-root-ca.crt\""
Sep 05 06:40:57.178034 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.177929    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7080a4\"/\"test-model-7080a4-kserve-self-signed-certs\""
Sep 05 06:40:57.183726 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.183689    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"]
Sep 05 06:40:57.237820 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.237795    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-home\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.237962 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.237831    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lzr7\" (UniqueName: \"kubernetes.io/projected/01122e19-e181-459a-b771-f0f327786ab9-kube-api-access-9lzr7\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.237962 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.237852    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-tmp-dir\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.238058 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.237961    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-dshm\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.238058 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.237988    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01122e19-e181-459a-b771-f0f327786ab9-tls-certs\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.238058 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.238016    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-model-cache\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.338919 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.338872    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-home\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.339067 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.338939    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lzr7\" (UniqueName: \"kubernetes.io/projected/01122e19-e181-459a-b771-f0f327786ab9-kube-api-access-9lzr7\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.339067 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.338960    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-tmp-dir\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.339067 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.339007    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-dshm\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.339067 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.339033    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01122e19-e181-459a-b771-f0f327786ab9-tls-certs\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.339293 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.339070    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-model-cache\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.339293 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.339267    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-home\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.339293 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.339286    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-tmp-dir\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.339433 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.339404    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-model-cache\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.341189 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.341169    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-dshm\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.341395 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.341377    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01122e19-e181-459a-b771-f0f327786ab9-tls-certs\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.347994 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.347972    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lzr7\" (UniqueName: \"kubernetes.io/projected/01122e19-e181-459a-b771-f0f327786ab9-kube-api-access-9lzr7\") pod \"test-model-7080a4-kserve-78cccff86f-tnm9d\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") " pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.486741 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.486719    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:40:57.602911 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.602839    2559 generic.go:356] "Generic (PLEG): container finished" podID="14bd8a6e-7ab5-4f77-a38b-894be0b8d535" containerID="46b263466f2d65f632acdaf15816cb90716ea12c377f227263207b1baf7e4e4c" exitCode=0
Sep 05 06:40:57.602911 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.602907    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"
Sep 05 06:40:57.603136 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.602920    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8" event={"ID":"14bd8a6e-7ab5-4f77-a38b-894be0b8d535","Type":"ContainerDied","Data":"46b263466f2d65f632acdaf15816cb90716ea12c377f227263207b1baf7e4e4c"}
Sep 05 06:40:57.603136 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.602970    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8" event={"ID":"14bd8a6e-7ab5-4f77-a38b-894be0b8d535","Type":"ContainerDied","Data":"dd56bbbaf69adc737e27d6657eeb94bbfc680d7bd51594f35507de417c737b4d"}
Sep 05 06:40:57.603136 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.602995    2559 scope.go:117] "RemoveContainer" containerID="46b263466f2d65f632acdaf15816cb90716ea12c377f227263207b1baf7e4e4c"
Sep 05 06:40:57.613796 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.613777    2559 scope.go:117] "RemoveContainer" containerID="d2f8ad4a613aea65163174e6a419669cc88b288163e731501b5876872e43b70c"
Sep 05 06:40:57.622658 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.622624    2559 scope.go:117] "RemoveContainer" containerID="46b263466f2d65f632acdaf15816cb90716ea12c377f227263207b1baf7e4e4c"
Sep 05 06:40:57.623156 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:40:57.623114    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46b263466f2d65f632acdaf15816cb90716ea12c377f227263207b1baf7e4e4c\": container with ID starting with 46b263466f2d65f632acdaf15816cb90716ea12c377f227263207b1baf7e4e4c not found: ID does not exist" containerID="46b263466f2d65f632acdaf15816cb90716ea12c377f227263207b1baf7e4e4c"
Sep 05 06:40:57.623264 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.623151    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46b263466f2d65f632acdaf15816cb90716ea12c377f227263207b1baf7e4e4c"} err="failed to get container status \"46b263466f2d65f632acdaf15816cb90716ea12c377f227263207b1baf7e4e4c\": rpc error: code = NotFound desc = could not find container \"46b263466f2d65f632acdaf15816cb90716ea12c377f227263207b1baf7e4e4c\": container with ID starting with 46b263466f2d65f632acdaf15816cb90716ea12c377f227263207b1baf7e4e4c not found: ID does not exist"
Sep 05 06:40:57.623264 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.623173    2559 scope.go:117] "RemoveContainer" containerID="d2f8ad4a613aea65163174e6a419669cc88b288163e731501b5876872e43b70c"
Sep 05 06:40:57.623655 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:40:57.623624    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2f8ad4a613aea65163174e6a419669cc88b288163e731501b5876872e43b70c\": container with ID starting with d2f8ad4a613aea65163174e6a419669cc88b288163e731501b5876872e43b70c not found: ID does not exist" containerID="d2f8ad4a613aea65163174e6a419669cc88b288163e731501b5876872e43b70c"
Sep 05 06:40:57.623735 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.623664    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2f8ad4a613aea65163174e6a419669cc88b288163e731501b5876872e43b70c"} err="failed to get container status \"d2f8ad4a613aea65163174e6a419669cc88b288163e731501b5876872e43b70c\": rpc error: code = NotFound desc = could not find container \"d2f8ad4a613aea65163174e6a419669cc88b288163e731501b5876872e43b70c\": container with ID starting with d2f8ad4a613aea65163174e6a419669cc88b288163e731501b5876872e43b70c not found: ID does not exist"
Sep 05 06:40:57.631730 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.631706    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"]
Sep 05 06:40:57.637168 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.637145    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-bba7714e-openshift-default-5f47d8bbf4-tp2n8"]
Sep 05 06:40:57.764704 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.764659    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"]
Sep 05 06:40:57.767124 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:40:57.767101    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01122e19_e181_459a_b771_f0f327786ab9.slice/crio-f7c72846691aa493d09e8a4dce7210a5e51eb9ce558674da2b82022d1ced0d0a WatchSource:0}: Error finding container f7c72846691aa493d09e8a4dce7210a5e51eb9ce558674da2b82022d1ced0d0a: Status 404 returned error can't find the container with id f7c72846691aa493d09e8a4dce7210a5e51eb9ce558674da2b82022d1ced0d0a
Sep 05 06:40:57.891847 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:57.891811    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14bd8a6e-7ab5-4f77-a38b-894be0b8d535" path="/var/lib/kubelet/pods/14bd8a6e-7ab5-4f77-a38b-894be0b8d535/volumes"
Sep 05 06:40:58.611695 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:58.611660    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d" event={"ID":"01122e19-e181-459a-b771-f0f327786ab9","Type":"ContainerStarted","Data":"2c593a6a348d0c63fd29ab1d0ffc3668113db0df1f47b2617640d4884ee2ced1"}
Sep 05 06:40:58.611695 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:58.611702    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d" event={"ID":"01122e19-e181-459a-b771-f0f327786ab9","Type":"ContainerStarted","Data":"f7c72846691aa493d09e8a4dce7210a5e51eb9ce558674da2b82022d1ced0d0a"}
Sep 05 06:40:58.631146 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:40:58.631098    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d" podStartSLOduration=1.631082843 podStartE2EDuration="1.631082843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:40:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:40:58.62851267 +0000 UTC m=+875.219071631" watchObservedRunningTime="2026-09-05 06:40:58.631082843 +0000 UTC m=+875.221641788"
Sep 05 06:41:07.487020 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:07.486978    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:41:07.487433 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:07.487046    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:41:07.499359 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:07.499335    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:41:07.690224 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:07.690181    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:41:11.627504 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:11.627070    2559 patch_prober.go:28] interesting pod/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 05 06:41:11.627504 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:11.627142    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd" podUID="56bc7df6-bf2e-4c4d-ae61-6c9936159344" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 05 06:41:13.714480 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:13.714449    2559 generic.go:356] "Generic (PLEG): container finished" podID="031e5d01-3e72-4e83-b588-7a2c626461bd" containerID="a9ad4fa04911b711b270c4ea5bbe5fa2b2b13fb3abaed2f1d042f72fa8d58c4f" exitCode=0
Sep 05 06:41:13.714813 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:13.714522    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg" event={"ID":"031e5d01-3e72-4e83-b588-7a2c626461bd","Type":"ContainerDied","Data":"a9ad4fa04911b711b270c4ea5bbe5fa2b2b13fb3abaed2f1d042f72fa8d58c4f"}
Sep 05 06:41:13.714926 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:13.714902    2559 scope.go:117] "RemoveContainer" containerID="a9ad4fa04911b711b270c4ea5bbe5fa2b2b13fb3abaed2f1d042f72fa8d58c4f"
Sep 05 06:41:13.715984 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:13.715956    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:41:13.716070 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:13.716033    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:41:13.716127 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:13.716072    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:41:13.716437 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:13.716416    2559 generic.go:356] "Generic (PLEG): container finished" podID="56bc7df6-bf2e-4c4d-ae61-6c9936159344" containerID="459806f42b9ce6b4a2437d05a61b1ee6765741b3ae19a84794dac812e4b90a6a" exitCode=0
Sep 05 06:41:13.716511 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:13.716491    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd" event={"ID":"56bc7df6-bf2e-4c4d-ae61-6c9936159344","Type":"ContainerDied","Data":"459806f42b9ce6b4a2437d05a61b1ee6765741b3ae19a84794dac812e4b90a6a"}
Sep 05 06:41:13.716802 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:13.716789    2559 scope.go:117] "RemoveContainer" containerID="459806f42b9ce6b4a2437d05a61b1ee6765741b3ae19a84794dac812e4b90a6a"
Sep 05 06:41:13.717710 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:13.717687    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:41:13.717754 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:13.717743    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:41:13.717787 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:13.717773    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:41:14.725204 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:14.725163    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd" event={"ID":"56bc7df6-bf2e-4c4d-ae61-6c9936159344","Type":"ContainerStarted","Data":"95dbab413cf31229b0b1fe55c609d1a7136244f77706c04541302c530e21caf6"}
Sep 05 06:41:14.727454 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:14.727426    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg" event={"ID":"031e5d01-3e72-4e83-b588-7a2c626461bd","Type":"ContainerStarted","Data":"acadcc08b04739ee486ee955d0be51c68d89e8a9bdba34e27f178920d5a4d53d"}
Sep 05 06:41:15.485292 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:15.485248    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:41:15.485488 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:15.485370    2559 patch_prober.go:28] interesting pod/e2e-explicit-9f698c-openshift-default-7f69567688-whchg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 05 06:41:15.485488 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:15.485446    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg" podUID="031e5d01-3e72-4e83-b588-7a2c626461bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 05 06:41:15.689993 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:15.689945    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:41:15.690252 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:15.690229    2559 patch_prober.go:28] interesting pod/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 05 06:41:15.690304 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:15.690285    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd" podUID="56bc7df6-bf2e-4c4d-ae61-6c9936159344" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 05 06:41:16.484652 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:16.484572    2559 patch_prober.go:28] interesting pod/e2e-explicit-9f698c-openshift-default-7f69567688-whchg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 05 06:41:16.484652 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:16.484638    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg" podUID="031e5d01-3e72-4e83-b588-7a2c626461bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 05 06:41:16.510945 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:16.510909    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:41:16.689674 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:16.689631    2559 patch_prober.go:28] interesting pod/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 05 06:41:16.689837 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:16.689693    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd" podUID="56bc7df6-bf2e-4c4d-ae61-6c9936159344" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 05 06:41:17.485134 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:17.485101    2559 patch_prober.go:28] interesting pod/e2e-explicit-9f698c-openshift-default-7f69567688-whchg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 05 06:41:17.485556 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:17.485531    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg" podUID="031e5d01-3e72-4e83-b588-7a2c626461bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 05 06:41:17.689783 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:17.689751    2559 patch_prober.go:28] interesting pod/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 05 06:41:17.690037 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:17.690013    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd" podUID="56bc7df6-bf2e-4c4d-ae61-6c9936159344" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 05 06:41:18.485020 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:18.484981    2559 patch_prober.go:28] interesting pod/e2e-explicit-9f698c-openshift-default-7f69567688-whchg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 05 06:41:18.485220 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:18.485042    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg" podUID="031e5d01-3e72-4e83-b588-7a2c626461bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 05 06:41:18.690032 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:18.689996    2559 patch_prober.go:28] interesting pod/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 05 06:41:18.690233 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:18.690055    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd" podUID="56bc7df6-bf2e-4c4d-ae61-6c9936159344" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 05 06:41:19.485360 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:19.485319    2559 patch_prober.go:28] interesting pod/e2e-explicit-9f698c-openshift-default-7f69567688-whchg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 05 06:41:19.485737 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:19.485375    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg" podUID="031e5d01-3e72-4e83-b588-7a2c626461bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 05 06:41:19.689468 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:19.689438    2559 patch_prober.go:28] interesting pod/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Sep 05 06:41:19.689684 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:19.689660    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd" podUID="56bc7df6-bf2e-4c4d-ae61-6c9936159344" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Sep 05 06:41:20.485295 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:20.485255    2559 patch_prober.go:28] interesting pod/e2e-explicit-9f698c-openshift-default-7f69567688-whchg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 05 06:41:20.485485 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:20.485315    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg" podUID="031e5d01-3e72-4e83-b588-7a2c626461bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 05 06:41:20.693044 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:20.693013    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:41:20.693429 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:20.693403    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:41:20.694132 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:20.694115    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:41:21.488658 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:21.488627    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:41:21.489052 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:21.488913    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:41:21.489532 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:21.489512    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:41:27.366644 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:27.366612    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"]
Sep 05 06:41:27.367080 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:27.366829    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd" podUID="56bc7df6-bf2e-4c4d-ae61-6c9936159344" containerName="istio-proxy" containerID="cri-o://95dbab413cf31229b0b1fe55c609d1a7136244f77706c04541302c530e21caf6" gracePeriod=30
Sep 05 06:41:30.885069 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:30.885037    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:41:32.611946 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.610192    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:41:32.673163 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.673097    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-workload-socket\") pod \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") "
Sep 05 06:41:32.673163 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.673145    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-token\") pod \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") "
Sep 05 06:41:32.673163 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.673160    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istiod-ca-cert\") pod \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") "
Sep 05 06:41:32.673416 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.673176    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-envoy\") pod \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") "
Sep 05 06:41:32.673416 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.673209    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-workload-certs\") pod \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") "
Sep 05 06:41:32.673416 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.673362    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gh6bb\" (UniqueName: \"kubernetes.io/projected/56bc7df6-bf2e-4c4d-ae61-6c9936159344-kube-api-access-gh6bb\") pod \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") "
Sep 05 06:41:32.673574 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.673419    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-credential-socket\") pod \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") "
Sep 05 06:41:32.673574 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.673447    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-podinfo\") pod \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") "
Sep 05 06:41:32.673574 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.673473    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-data\") pod \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\" (UID: \"56bc7df6-bf2e-4c4d-ae61-6c9936159344\") "
Sep 05 06:41:32.673574 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.673494    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "56bc7df6-bf2e-4c4d-ae61-6c9936159344" (UID: "56bc7df6-bf2e-4c4d-ae61-6c9936159344"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:41:32.673574 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.673539    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "56bc7df6-bf2e-4c4d-ae61-6c9936159344" (UID: "56bc7df6-bf2e-4c4d-ae61-6c9936159344"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:41:32.673841 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.673599    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "56bc7df6-bf2e-4c4d-ae61-6c9936159344" (UID: "56bc7df6-bf2e-4c4d-ae61-6c9936159344"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:41:32.673841 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.673767    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-workload-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:41:32.673841 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.673761    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "56bc7df6-bf2e-4c4d-ae61-6c9936159344" (UID: "56bc7df6-bf2e-4c4d-ae61-6c9936159344"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:41:32.673841 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.673782    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-workload-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:41:32.673841 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.673821    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istiod-ca-cert\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:41:32.673841 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.673834    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-data" (OuterVolumeSpecName: "istio-data") pod "56bc7df6-bf2e-4c4d-ae61-6c9936159344" (UID: "56bc7df6-bf2e-4c4d-ae61-6c9936159344"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:41:32.675435 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.675401    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "56bc7df6-bf2e-4c4d-ae61-6c9936159344" (UID: "56bc7df6-bf2e-4c4d-ae61-6c9936159344"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:41:32.675534 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.675514    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-token" (OuterVolumeSpecName: "istio-token") pod "56bc7df6-bf2e-4c4d-ae61-6c9936159344" (UID: "56bc7df6-bf2e-4c4d-ae61-6c9936159344"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:41:32.675947 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.675916    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "56bc7df6-bf2e-4c4d-ae61-6c9936159344" (UID: "56bc7df6-bf2e-4c4d-ae61-6c9936159344"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 06:41:32.675947 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.675919    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56bc7df6-bf2e-4c4d-ae61-6c9936159344-kube-api-access-gh6bb" (OuterVolumeSpecName: "kube-api-access-gh6bb") pod "56bc7df6-bf2e-4c4d-ae61-6c9936159344" (UID: "56bc7df6-bf2e-4c4d-ae61-6c9936159344"). InnerVolumeSpecName "kube-api-access-gh6bb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:41:32.774300 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.774273    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gh6bb\" (UniqueName: \"kubernetes.io/projected/56bc7df6-bf2e-4c4d-ae61-6c9936159344-kube-api-access-gh6bb\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:41:32.774300 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.774296    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-credential-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:41:32.774461 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.774306    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-podinfo\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:41:32.774461 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.774314    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-data\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:41:32.774461 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.774323    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-token\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:41:32.774461 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.774331    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56bc7df6-bf2e-4c4d-ae61-6c9936159344-istio-envoy\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:41:32.852423 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.852394    2559 generic.go:356] "Generic (PLEG): container finished" podID="56bc7df6-bf2e-4c4d-ae61-6c9936159344" containerID="95dbab413cf31229b0b1fe55c609d1a7136244f77706c04541302c530e21caf6" exitCode=0
Sep 05 06:41:32.852556 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.852451    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"
Sep 05 06:41:32.852556 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.852460    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd" event={"ID":"56bc7df6-bf2e-4c4d-ae61-6c9936159344","Type":"ContainerDied","Data":"95dbab413cf31229b0b1fe55c609d1a7136244f77706c04541302c530e21caf6"}
Sep 05 06:41:32.852556 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.852544    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd" event={"ID":"56bc7df6-bf2e-4c4d-ae61-6c9936159344","Type":"ContainerDied","Data":"aa0d0cedd96c171f429a4456d91cf7be61e53b5137731e2f73f17c2a32b0fdfb"}
Sep 05 06:41:32.852719 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.852574    2559 scope.go:117] "RemoveContainer" containerID="95dbab413cf31229b0b1fe55c609d1a7136244f77706c04541302c530e21caf6"
Sep 05 06:41:32.861934 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.861917    2559 scope.go:117] "RemoveContainer" containerID="459806f42b9ce6b4a2437d05a61b1ee6765741b3ae19a84794dac812e4b90a6a"
Sep 05 06:41:32.869858 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.869842    2559 scope.go:117] "RemoveContainer" containerID="95dbab413cf31229b0b1fe55c609d1a7136244f77706c04541302c530e21caf6"
Sep 05 06:41:32.870109 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:41:32.870091    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95dbab413cf31229b0b1fe55c609d1a7136244f77706c04541302c530e21caf6\": container with ID starting with 95dbab413cf31229b0b1fe55c609d1a7136244f77706c04541302c530e21caf6 not found: ID does not exist" containerID="95dbab413cf31229b0b1fe55c609d1a7136244f77706c04541302c530e21caf6"
Sep 05 06:41:32.870169 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.870117    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95dbab413cf31229b0b1fe55c609d1a7136244f77706c04541302c530e21caf6"} err="failed to get container status \"95dbab413cf31229b0b1fe55c609d1a7136244f77706c04541302c530e21caf6\": rpc error: code = NotFound desc = could not find container \"95dbab413cf31229b0b1fe55c609d1a7136244f77706c04541302c530e21caf6\": container with ID starting with 95dbab413cf31229b0b1fe55c609d1a7136244f77706c04541302c530e21caf6 not found: ID does not exist"
Sep 05 06:41:32.870169 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.870134    2559 scope.go:117] "RemoveContainer" containerID="459806f42b9ce6b4a2437d05a61b1ee6765741b3ae19a84794dac812e4b90a6a"
Sep 05 06:41:32.870385 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:41:32.870365    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"459806f42b9ce6b4a2437d05a61b1ee6765741b3ae19a84794dac812e4b90a6a\": container with ID starting with 459806f42b9ce6b4a2437d05a61b1ee6765741b3ae19a84794dac812e4b90a6a not found: ID does not exist" containerID="459806f42b9ce6b4a2437d05a61b1ee6765741b3ae19a84794dac812e4b90a6a"
Sep 05 06:41:32.870437 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.870393    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"459806f42b9ce6b4a2437d05a61b1ee6765741b3ae19a84794dac812e4b90a6a"} err="failed to get container status \"459806f42b9ce6b4a2437d05a61b1ee6765741b3ae19a84794dac812e4b90a6a\": rpc error: code = NotFound desc = could not find container \"459806f42b9ce6b4a2437d05a61b1ee6765741b3ae19a84794dac812e4b90a6a\": container with ID starting with 459806f42b9ce6b4a2437d05a61b1ee6765741b3ae19a84794dac812e4b90a6a not found: ID does not exist"
Sep 05 06:41:32.876906 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.876872    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"]
Sep 05 06:41:32.885115 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:32.885095    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-13503c3c-openshift-default-86b59d4f44-qzdtd"]
Sep 05 06:41:33.891473 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:33.891439    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56bc7df6-bf2e-4c4d-ae61-6c9936159344" path="/var/lib/kubelet/pods/56bc7df6-bf2e-4c4d-ae61-6c9936159344/volumes"
Sep 05 06:41:45.891815 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:45.891786    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:41:59.992807 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:41:59.992766    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:42:13.790691 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:13.790655    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:42:15.509563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:15.509527    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"]
Sep 05 06:42:15.509931 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:15.509771    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg" podUID="031e5d01-3e72-4e83-b588-7a2c626461bd" containerName="istio-proxy" containerID="cri-o://acadcc08b04739ee486ee955d0be51c68d89e8a9bdba34e27f178920d5a4d53d" gracePeriod=30
Sep 05 06:42:16.291539 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.291507    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"]
Sep 05 06:42:16.292097 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.292070    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56bc7df6-bf2e-4c4d-ae61-6c9936159344" containerName="istio-proxy"
Sep 05 06:42:16.292097 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.292095    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="56bc7df6-bf2e-4c4d-ae61-6c9936159344" containerName="istio-proxy"
Sep 05 06:42:16.292331 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.292179    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="56bc7df6-bf2e-4c4d-ae61-6c9936159344" containerName="istio-proxy"
Sep 05 06:42:16.292331 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.292190    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="56bc7df6-bf2e-4c4d-ae61-6c9936159344" containerName="istio-proxy"
Sep 05 06:42:16.295528 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.295507    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.298054 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.298035    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-eee7e8-openshift-default-dockercfg-csxwz\""
Sep 05 06:42:16.308132 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.308106    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"]
Sep 05 06:42:16.424243 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.424210    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62762379-6e50-4445-83ff-589cb15dcefa-istio-podinfo\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.424426 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.424267    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-workload-socket\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.424426 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.424299    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62762379-6e50-4445-83ff-589cb15dcefa-istio-token\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.424426 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.424323    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62762379-6e50-4445-83ff-589cb15dcefa-istiod-ca-cert\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.424426 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.424384    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6fss\" (UniqueName: \"kubernetes.io/projected/62762379-6e50-4445-83ff-589cb15dcefa-kube-api-access-j6fss\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.424426 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.424427    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-credential-socket\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.424688 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.424517    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-istio-envoy\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.424688 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.424548    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-istio-data\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.424688 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.424612    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-workload-certs\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.525621 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.525580    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-workload-certs\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.525621 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.525614    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62762379-6e50-4445-83ff-589cb15dcefa-istio-podinfo\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.526110 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.525637    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-workload-socket\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.526110 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.525659    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62762379-6e50-4445-83ff-589cb15dcefa-istio-token\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.526110 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.525682    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62762379-6e50-4445-83ff-589cb15dcefa-istiod-ca-cert\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.526110 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.525719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6fss\" (UniqueName: \"kubernetes.io/projected/62762379-6e50-4445-83ff-589cb15dcefa-kube-api-access-j6fss\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.526110 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.525759    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-credential-socket\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.526110 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.525842    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-istio-envoy\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.526110 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.525872    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-istio-data\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.526110 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.526053    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-workload-certs\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.526520 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.526208    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-istio-data\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.526520 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.526332    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-workload-socket\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.526726 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.526699    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62762379-6e50-4445-83ff-589cb15dcefa-istiod-ca-cert\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.526936 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.526914    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-credential-socket\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.528934 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.528912    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-istio-envoy\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.529269 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.529249    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62762379-6e50-4445-83ff-589cb15dcefa-istio-podinfo\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.536627 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.536609    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6fss\" (UniqueName: \"kubernetes.io/projected/62762379-6e50-4445-83ff-589cb15dcefa-kube-api-access-j6fss\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.538247 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.538230    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62762379-6e50-4445-83ff-589cb15dcefa-istio-token\") pod \"e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") " pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.605454 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.605408    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:16.881996 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.881967    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"]
Sep 05 06:42:16.882825 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:42:16.882803    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62762379_6e50_4445_83ff_589cb15dcefa.slice/crio-549e4e63cb3273a294549bdd86aa67705d7b8babb70fbc293a4041fac872b036 WatchSource:0}: Error finding container 549e4e63cb3273a294549bdd86aa67705d7b8babb70fbc293a4041fac872b036: Status 404 returned error can't find the container with id 549e4e63cb3273a294549bdd86aa67705d7b8babb70fbc293a4041fac872b036
Sep 05 06:42:16.884914 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.884867    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:42:16.885010 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.884941    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:42:16.885010 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:16.884971    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:42:17.098802 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:17.098766    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k" event={"ID":"62762379-6e50-4445-83ff-589cb15dcefa","Type":"ContainerStarted","Data":"7752824b263e69b236b1b310adc29180ee595842a872e736e1c45ddc85bb2bfb"}
Sep 05 06:42:17.099017 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:17.098825    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k" event={"ID":"62762379-6e50-4445-83ff-589cb15dcefa","Type":"ContainerStarted","Data":"549e4e63cb3273a294549bdd86aa67705d7b8babb70fbc293a4041fac872b036"}
Sep 05 06:42:17.606288 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:17.606257    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:17.611094 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:17.611069    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:17.633553 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:17.633506    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k" podStartSLOduration=1.633491241 podStartE2EDuration="1.633491241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:42:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:42:17.120246056 +0000 UTC m=+953.710805025" watchObservedRunningTime="2026-09-05 06:42:17.633491241 +0000 UTC m=+954.224050202"
Sep 05 06:42:18.104698 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:18.104660    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:18.105489 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:18.105467    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:20.763675 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.763641    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:42:20.861910 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.861836    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-credential-socket\") pod \"031e5d01-3e72-4e83-b588-7a2c626461bd\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") "
Sep 05 06:42:20.861910 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.861873    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-token\") pod \"031e5d01-3e72-4e83-b588-7a2c626461bd\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") "
Sep 05 06:42:20.862110 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.861941    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbl6l\" (UniqueName: \"kubernetes.io/projected/031e5d01-3e72-4e83-b588-7a2c626461bd-kube-api-access-cbl6l\") pod \"031e5d01-3e72-4e83-b588-7a2c626461bd\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") "
Sep 05 06:42:20.862110 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.861956    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/031e5d01-3e72-4e83-b588-7a2c626461bd-istiod-ca-cert\") pod \"031e5d01-3e72-4e83-b588-7a2c626461bd\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") "
Sep 05 06:42:20.862110 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.862007    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-workload-socket\") pod \"031e5d01-3e72-4e83-b588-7a2c626461bd\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") "
Sep 05 06:42:20.862110 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.862037    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-podinfo\") pod \"031e5d01-3e72-4e83-b588-7a2c626461bd\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") "
Sep 05 06:42:20.862110 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.862073    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-data\") pod \"031e5d01-3e72-4e83-b588-7a2c626461bd\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") "
Sep 05 06:42:20.862110 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.862101    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-workload-certs\") pod \"031e5d01-3e72-4e83-b588-7a2c626461bd\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") "
Sep 05 06:42:20.862408 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.862122    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-envoy\") pod \"031e5d01-3e72-4e83-b588-7a2c626461bd\" (UID: \"031e5d01-3e72-4e83-b588-7a2c626461bd\") "
Sep 05 06:42:20.862408 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.862067    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "031e5d01-3e72-4e83-b588-7a2c626461bd" (UID: "031e5d01-3e72-4e83-b588-7a2c626461bd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:42:20.862408 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.862341    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "031e5d01-3e72-4e83-b588-7a2c626461bd" (UID: "031e5d01-3e72-4e83-b588-7a2c626461bd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:42:20.862408 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.862364    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/031e5d01-3e72-4e83-b588-7a2c626461bd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "031e5d01-3e72-4e83-b588-7a2c626461bd" (UID: "031e5d01-3e72-4e83-b588-7a2c626461bd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:42:20.862596 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.862473    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "031e5d01-3e72-4e83-b588-7a2c626461bd" (UID: "031e5d01-3e72-4e83-b588-7a2c626461bd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:42:20.862596 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.862526    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-data" (OuterVolumeSpecName: "istio-data") pod "031e5d01-3e72-4e83-b588-7a2c626461bd" (UID: "031e5d01-3e72-4e83-b588-7a2c626461bd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:42:20.862596 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.862542    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-workload-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:20.862596 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.862564    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-credential-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:20.862596 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.862578    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/031e5d01-3e72-4e83-b588-7a2c626461bd-istiod-ca-cert\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:20.864237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.864213    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "031e5d01-3e72-4e83-b588-7a2c626461bd" (UID: "031e5d01-3e72-4e83-b588-7a2c626461bd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:42:20.864473 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.864444    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "031e5d01-3e72-4e83-b588-7a2c626461bd" (UID: "031e5d01-3e72-4e83-b588-7a2c626461bd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 06:42:20.864473 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.864456    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-token" (OuterVolumeSpecName: "istio-token") pod "031e5d01-3e72-4e83-b588-7a2c626461bd" (UID: "031e5d01-3e72-4e83-b588-7a2c626461bd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:42:20.864610 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.864483    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/031e5d01-3e72-4e83-b588-7a2c626461bd-kube-api-access-cbl6l" (OuterVolumeSpecName: "kube-api-access-cbl6l") pod "031e5d01-3e72-4e83-b588-7a2c626461bd" (UID: "031e5d01-3e72-4e83-b588-7a2c626461bd"). InnerVolumeSpecName "kube-api-access-cbl6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:42:20.963845 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.963815    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-token\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:20.963845 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.963842    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbl6l\" (UniqueName: \"kubernetes.io/projected/031e5d01-3e72-4e83-b588-7a2c626461bd-kube-api-access-cbl6l\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:20.964039 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.963856    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-podinfo\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:20.964039 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.963870    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-data\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:20.964039 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.963908    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-workload-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:20.964039 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:20.963922    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/031e5d01-3e72-4e83-b588-7a2c626461bd-istio-envoy\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:21.122823 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:21.122737    2559 generic.go:356] "Generic (PLEG): container finished" podID="031e5d01-3e72-4e83-b588-7a2c626461bd" containerID="acadcc08b04739ee486ee955d0be51c68d89e8a9bdba34e27f178920d5a4d53d" exitCode=0
Sep 05 06:42:21.122823 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:21.122794    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"
Sep 05 06:42:21.123042 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:21.122826    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg" event={"ID":"031e5d01-3e72-4e83-b588-7a2c626461bd","Type":"ContainerDied","Data":"acadcc08b04739ee486ee955d0be51c68d89e8a9bdba34e27f178920d5a4d53d"}
Sep 05 06:42:21.123042 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:21.122908    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg" event={"ID":"031e5d01-3e72-4e83-b588-7a2c626461bd","Type":"ContainerDied","Data":"a6e8c5d01d81e88cdf8a9d3fa76f61e9374f9547e37d8109c6a67de4f339d1cb"}
Sep 05 06:42:21.123042 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:21.122946    2559 scope.go:117] "RemoveContainer" containerID="acadcc08b04739ee486ee955d0be51c68d89e8a9bdba34e27f178920d5a4d53d"
Sep 05 06:42:21.134051 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:21.134035    2559 scope.go:117] "RemoveContainer" containerID="a9ad4fa04911b711b270c4ea5bbe5fa2b2b13fb3abaed2f1d042f72fa8d58c4f"
Sep 05 06:42:21.143976 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:21.142852    2559 scope.go:117] "RemoveContainer" containerID="acadcc08b04739ee486ee955d0be51c68d89e8a9bdba34e27f178920d5a4d53d"
Sep 05 06:42:21.143976 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:42:21.143913    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acadcc08b04739ee486ee955d0be51c68d89e8a9bdba34e27f178920d5a4d53d\": container with ID starting with acadcc08b04739ee486ee955d0be51c68d89e8a9bdba34e27f178920d5a4d53d not found: ID does not exist" containerID="acadcc08b04739ee486ee955d0be51c68d89e8a9bdba34e27f178920d5a4d53d"
Sep 05 06:42:21.143976 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:21.143941    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acadcc08b04739ee486ee955d0be51c68d89e8a9bdba34e27f178920d5a4d53d"} err="failed to get container status \"acadcc08b04739ee486ee955d0be51c68d89e8a9bdba34e27f178920d5a4d53d\": rpc error: code = NotFound desc = could not find container \"acadcc08b04739ee486ee955d0be51c68d89e8a9bdba34e27f178920d5a4d53d\": container with ID starting with acadcc08b04739ee486ee955d0be51c68d89e8a9bdba34e27f178920d5a4d53d not found: ID does not exist"
Sep 05 06:42:21.143976 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:21.143957    2559 scope.go:117] "RemoveContainer" containerID="a9ad4fa04911b711b270c4ea5bbe5fa2b2b13fb3abaed2f1d042f72fa8d58c4f"
Sep 05 06:42:21.144366 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:42:21.144340    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9ad4fa04911b711b270c4ea5bbe5fa2b2b13fb3abaed2f1d042f72fa8d58c4f\": container with ID starting with a9ad4fa04911b711b270c4ea5bbe5fa2b2b13fb3abaed2f1d042f72fa8d58c4f not found: ID does not exist" containerID="a9ad4fa04911b711b270c4ea5bbe5fa2b2b13fb3abaed2f1d042f72fa8d58c4f"
Sep 05 06:42:21.144430 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:21.144365    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9ad4fa04911b711b270c4ea5bbe5fa2b2b13fb3abaed2f1d042f72fa8d58c4f"} err="failed to get container status \"a9ad4fa04911b711b270c4ea5bbe5fa2b2b13fb3abaed2f1d042f72fa8d58c4f\": rpc error: code = NotFound desc = could not find container \"a9ad4fa04911b711b270c4ea5bbe5fa2b2b13fb3abaed2f1d042f72fa8d58c4f\": container with ID starting with a9ad4fa04911b711b270c4ea5bbe5fa2b2b13fb3abaed2f1d042f72fa8d58c4f not found: ID does not exist"
Sep 05 06:42:21.148844 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:21.148820    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"]
Sep 05 06:42:21.154158 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:21.154137    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-9f698c-openshift-default-7f69567688-whchg"]
Sep 05 06:42:21.891636 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:21.891603    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="031e5d01-3e72-4e83-b588-7a2c626461bd" path="/var/lib/kubelet/pods/031e5d01-3e72-4e83-b588-7a2c626461bd/volumes"
Sep 05 06:42:42.093503 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:42.093467    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:42:53.990030 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:53.989954    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"]
Sep 05 06:42:53.990536 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:53.990186    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k" podUID="62762379-6e50-4445-83ff-589cb15dcefa" containerName="istio-proxy" containerID="cri-o://7752824b263e69b236b1b310adc29180ee595842a872e736e1c45ddc85bb2bfb" gracePeriod=30
Sep 05 06:42:55.061642 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:55.061603    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"]
Sep 05 06:42:55.896939 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:55.896906    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:42:59.242758 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.242735    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:59.335995 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.335912    2559 generic.go:356] "Generic (PLEG): container finished" podID="62762379-6e50-4445-83ff-589cb15dcefa" containerID="7752824b263e69b236b1b310adc29180ee595842a872e736e1c45ddc85bb2bfb" exitCode=0
Sep 05 06:42:59.335995 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.335966    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"
Sep 05 06:42:59.336177 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.335958    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k" event={"ID":"62762379-6e50-4445-83ff-589cb15dcefa","Type":"ContainerDied","Data":"7752824b263e69b236b1b310adc29180ee595842a872e736e1c45ddc85bb2bfb"}
Sep 05 06:42:59.336177 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.336098    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k" event={"ID":"62762379-6e50-4445-83ff-589cb15dcefa","Type":"ContainerDied","Data":"549e4e63cb3273a294549bdd86aa67705d7b8babb70fbc293a4041fac872b036"}
Sep 05 06:42:59.336177 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.336126    2559 scope.go:117] "RemoveContainer" containerID="7752824b263e69b236b1b310adc29180ee595842a872e736e1c45ddc85bb2bfb"
Sep 05 06:42:59.344723 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.344706    2559 scope.go:117] "RemoveContainer" containerID="7752824b263e69b236b1b310adc29180ee595842a872e736e1c45ddc85bb2bfb"
Sep 05 06:42:59.345008 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:42:59.344989    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7752824b263e69b236b1b310adc29180ee595842a872e736e1c45ddc85bb2bfb\": container with ID starting with 7752824b263e69b236b1b310adc29180ee595842a872e736e1c45ddc85bb2bfb not found: ID does not exist" containerID="7752824b263e69b236b1b310adc29180ee595842a872e736e1c45ddc85bb2bfb"
Sep 05 06:42:59.345096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.345015    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7752824b263e69b236b1b310adc29180ee595842a872e736e1c45ddc85bb2bfb"} err="failed to get container status \"7752824b263e69b236b1b310adc29180ee595842a872e736e1c45ddc85bb2bfb\": rpc error: code = NotFound desc = could not find container \"7752824b263e69b236b1b310adc29180ee595842a872e736e1c45ddc85bb2bfb\": container with ID starting with 7752824b263e69b236b1b310adc29180ee595842a872e736e1c45ddc85bb2bfb not found: ID does not exist"
Sep 05 06:42:59.384598 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.384572    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-credential-socket\") pod \"62762379-6e50-4445-83ff-589cb15dcefa\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") "
Sep 05 06:42:59.384693 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.384606    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-workload-certs\") pod \"62762379-6e50-4445-83ff-589cb15dcefa\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") "
Sep 05 06:42:59.384693 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.384665    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62762379-6e50-4445-83ff-589cb15dcefa-istio-podinfo\") pod \"62762379-6e50-4445-83ff-589cb15dcefa\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") "
Sep 05 06:42:59.384693 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.384691    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-istio-data\") pod \"62762379-6e50-4445-83ff-589cb15dcefa\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") "
Sep 05 06:42:59.384868 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.384723    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62762379-6e50-4445-83ff-589cb15dcefa-istiod-ca-cert\") pod \"62762379-6e50-4445-83ff-589cb15dcefa\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") "
Sep 05 06:42:59.384868 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.384746    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-istio-envoy\") pod \"62762379-6e50-4445-83ff-589cb15dcefa\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") "
Sep 05 06:42:59.384868 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.384796    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-workload-socket\") pod \"62762379-6e50-4445-83ff-589cb15dcefa\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") "
Sep 05 06:42:59.384868 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.384820    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62762379-6e50-4445-83ff-589cb15dcefa-istio-token\") pod \"62762379-6e50-4445-83ff-589cb15dcefa\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") "
Sep 05 06:42:59.385103 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.384845    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "62762379-6e50-4445-83ff-589cb15dcefa" (UID: "62762379-6e50-4445-83ff-589cb15dcefa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:42:59.385103 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.384931    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "62762379-6e50-4445-83ff-589cb15dcefa" (UID: "62762379-6e50-4445-83ff-589cb15dcefa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:42:59.385103 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.385045    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-istio-data" (OuterVolumeSpecName: "istio-data") pod "62762379-6e50-4445-83ff-589cb15dcefa" (UID: "62762379-6e50-4445-83ff-589cb15dcefa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:42:59.385103 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.385083    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "62762379-6e50-4445-83ff-589cb15dcefa" (UID: "62762379-6e50-4445-83ff-589cb15dcefa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:42:59.385287 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.385118    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62762379-6e50-4445-83ff-589cb15dcefa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "62762379-6e50-4445-83ff-589cb15dcefa" (UID: "62762379-6e50-4445-83ff-589cb15dcefa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:42:59.385287 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.385126    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6fss\" (UniqueName: \"kubernetes.io/projected/62762379-6e50-4445-83ff-589cb15dcefa-kube-api-access-j6fss\") pod \"62762379-6e50-4445-83ff-589cb15dcefa\" (UID: \"62762379-6e50-4445-83ff-589cb15dcefa\") "
Sep 05 06:42:59.385467 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.385437    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-workload-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:59.385533 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.385467    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-credential-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:59.385533 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.385481    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-workload-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:59.385533 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.385494    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-istio-data\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:59.385533 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.385506    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62762379-6e50-4445-83ff-589cb15dcefa-istiod-ca-cert\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:59.386851 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.386820    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62762379-6e50-4445-83ff-589cb15dcefa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "62762379-6e50-4445-83ff-589cb15dcefa" (UID: "62762379-6e50-4445-83ff-589cb15dcefa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 06:42:59.386851 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.386837    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "62762379-6e50-4445-83ff-589cb15dcefa" (UID: "62762379-6e50-4445-83ff-589cb15dcefa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:42:59.386993 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.386905    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62762379-6e50-4445-83ff-589cb15dcefa-istio-token" (OuterVolumeSpecName: "istio-token") pod "62762379-6e50-4445-83ff-589cb15dcefa" (UID: "62762379-6e50-4445-83ff-589cb15dcefa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:42:59.387144 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.387129    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62762379-6e50-4445-83ff-589cb15dcefa-kube-api-access-j6fss" (OuterVolumeSpecName: "kube-api-access-j6fss") pod "62762379-6e50-4445-83ff-589cb15dcefa" (UID: "62762379-6e50-4445-83ff-589cb15dcefa"). InnerVolumeSpecName "kube-api-access-j6fss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:42:59.485891 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.485846    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62762379-6e50-4445-83ff-589cb15dcefa-istio-podinfo\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:59.485891 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.485874    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62762379-6e50-4445-83ff-589cb15dcefa-istio-envoy\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:59.486075 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.485907    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62762379-6e50-4445-83ff-589cb15dcefa-istio-token\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:59.486075 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.485916    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6fss\" (UniqueName: \"kubernetes.io/projected/62762379-6e50-4445-83ff-589cb15dcefa-kube-api-access-j6fss\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:42:59.660640 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.660602    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"]
Sep 05 06:42:59.664710 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.664686    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-eee7e8-openshift-default-6fb6fdf966-8j27k"]
Sep 05 06:42:59.892168 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:42:59.892140    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62762379-6e50-4445-83ff-589cb15dcefa" path="/var/lib/kubelet/pods/62762379-6e50-4445-83ff-589cb15dcefa/volumes"
Sep 05 06:43:06.796937 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:06.796901    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:43:10.080539 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.080495    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d" podUID="01122e19-e181-459a-b771-f0f327786ab9" containerName="main" containerID="cri-o://2c593a6a348d0c63fd29ab1d0ffc3668113db0df1f47b2617640d4884ee2ced1" gracePeriod=45
Sep 05 06:43:10.329500 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.329478    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:43:10.396996 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.396927    2559 generic.go:356] "Generic (PLEG): container finished" podID="01122e19-e181-459a-b771-f0f327786ab9" containerID="2c593a6a348d0c63fd29ab1d0ffc3668113db0df1f47b2617640d4884ee2ced1" exitCode=0
Sep 05 06:43:10.396996 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.396964    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"
Sep 05 06:43:10.397162 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.397009    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d" event={"ID":"01122e19-e181-459a-b771-f0f327786ab9","Type":"ContainerDied","Data":"2c593a6a348d0c63fd29ab1d0ffc3668113db0df1f47b2617640d4884ee2ced1"}
Sep 05 06:43:10.397162 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.397053    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d" event={"ID":"01122e19-e181-459a-b771-f0f327786ab9","Type":"ContainerDied","Data":"f7c72846691aa493d09e8a4dce7210a5e51eb9ce558674da2b82022d1ced0d0a"}
Sep 05 06:43:10.397162 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.397072    2559 scope.go:117] "RemoveContainer" containerID="2c593a6a348d0c63fd29ab1d0ffc3668113db0df1f47b2617640d4884ee2ced1"
Sep 05 06:43:10.405178 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.405162    2559 scope.go:117] "RemoveContainer" containerID="2c593a6a348d0c63fd29ab1d0ffc3668113db0df1f47b2617640d4884ee2ced1"
Sep 05 06:43:10.405431 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:43:10.405411    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c593a6a348d0c63fd29ab1d0ffc3668113db0df1f47b2617640d4884ee2ced1\": container with ID starting with 2c593a6a348d0c63fd29ab1d0ffc3668113db0df1f47b2617640d4884ee2ced1 not found: ID does not exist" containerID="2c593a6a348d0c63fd29ab1d0ffc3668113db0df1f47b2617640d4884ee2ced1"
Sep 05 06:43:10.405482 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.405437    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c593a6a348d0c63fd29ab1d0ffc3668113db0df1f47b2617640d4884ee2ced1"} err="failed to get container status \"2c593a6a348d0c63fd29ab1d0ffc3668113db0df1f47b2617640d4884ee2ced1\": rpc error: code = NotFound desc = could not find container \"2c593a6a348d0c63fd29ab1d0ffc3668113db0df1f47b2617640d4884ee2ced1\": container with ID starting with 2c593a6a348d0c63fd29ab1d0ffc3668113db0df1f47b2617640d4884ee2ced1 not found: ID does not exist"
Sep 05 06:43:10.474237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.474210    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-tmp-dir\") pod \"01122e19-e181-459a-b771-f0f327786ab9\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") "
Sep 05 06:43:10.474359 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.474268    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-dshm\") pod \"01122e19-e181-459a-b771-f0f327786ab9\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") "
Sep 05 06:43:10.474359 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.474306    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-model-cache\") pod \"01122e19-e181-459a-b771-f0f327786ab9\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") "
Sep 05 06:43:10.474359 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.474332    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-home\") pod \"01122e19-e181-459a-b771-f0f327786ab9\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") "
Sep 05 06:43:10.474359 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.474350    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lzr7\" (UniqueName: \"kubernetes.io/projected/01122e19-e181-459a-b771-f0f327786ab9-kube-api-access-9lzr7\") pod \"01122e19-e181-459a-b771-f0f327786ab9\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") "
Sep 05 06:43:10.474568 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.474384    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01122e19-e181-459a-b771-f0f327786ab9-tls-certs\") pod \"01122e19-e181-459a-b771-f0f327786ab9\" (UID: \"01122e19-e181-459a-b771-f0f327786ab9\") "
Sep 05 06:43:10.474568 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.474448    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "01122e19-e181-459a-b771-f0f327786ab9" (UID: "01122e19-e181-459a-b771-f0f327786ab9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:43:10.474667 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.474605    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-home" (OuterVolumeSpecName: "home") pod "01122e19-e181-459a-b771-f0f327786ab9" (UID: "01122e19-e181-459a-b771-f0f327786ab9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:43:10.474667 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.474613    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-model-cache" (OuterVolumeSpecName: "model-cache") pod "01122e19-e181-459a-b771-f0f327786ab9" (UID: "01122e19-e181-459a-b771-f0f327786ab9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:43:10.474743 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.474676    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-tmp-dir\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:43:10.476420 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.476394    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01122e19-e181-459a-b771-f0f327786ab9-kube-api-access-9lzr7" (OuterVolumeSpecName: "kube-api-access-9lzr7") pod "01122e19-e181-459a-b771-f0f327786ab9" (UID: "01122e19-e181-459a-b771-f0f327786ab9"). InnerVolumeSpecName "kube-api-access-9lzr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:43:10.476502 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.476426    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01122e19-e181-459a-b771-f0f327786ab9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "01122e19-e181-459a-b771-f0f327786ab9" (UID: "01122e19-e181-459a-b771-f0f327786ab9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:43:10.476502 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.476431    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-dshm" (OuterVolumeSpecName: "dshm") pod "01122e19-e181-459a-b771-f0f327786ab9" (UID: "01122e19-e181-459a-b771-f0f327786ab9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:43:10.575765 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.575740    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-dshm\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:43:10.575765 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.575764    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-model-cache\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:43:10.575921 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.575778    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01122e19-e181-459a-b771-f0f327786ab9-home\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:43:10.575921 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.575789    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lzr7\" (UniqueName: \"kubernetes.io/projected/01122e19-e181-459a-b771-f0f327786ab9-kube-api-access-9lzr7\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:43:10.575921 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.575802    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01122e19-e181-459a-b771-f0f327786ab9-tls-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:43:10.722128 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.722082    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"]
Sep 05 06:43:10.725463 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:10.725439    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-7080a4/test-model-7080a4-kserve-78cccff86f-tnm9d"]
Sep 05 06:43:11.891786 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:11.891752    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01122e19-e181-459a-b771-f0f327786ab9" path="/var/lib/kubelet/pods/01122e19-e181-459a-b771-f0f327786ab9/volumes"
Sep 05 06:43:15.390492 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:15.390450    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:43:23.595306 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:23.595273    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:43:31.895292 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:31.895247    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:43:40.496499 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:43:40.496461    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:44:01.486484 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:01.486449    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:44:09.891891 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:09.891846    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:44:29.799538 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:29.799451    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:44:40.587240 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.587209    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"]
Sep 05 06:44:40.587731 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.587715    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01122e19-e181-459a-b771-f0f327786ab9" containerName="main"
Sep 05 06:44:40.587783 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.587736    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="01122e19-e181-459a-b771-f0f327786ab9" containerName="main"
Sep 05 06:44:40.587783 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.587756    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56bc7df6-bf2e-4c4d-ae61-6c9936159344" containerName="istio-proxy"
Sep 05 06:44:40.587783 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.587765    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="56bc7df6-bf2e-4c4d-ae61-6c9936159344" containerName="istio-proxy"
Sep 05 06:44:40.587783 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.587777    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="031e5d01-3e72-4e83-b588-7a2c626461bd" containerName="istio-proxy"
Sep 05 06:44:40.587941 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.587786    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="031e5d01-3e72-4e83-b588-7a2c626461bd" containerName="istio-proxy"
Sep 05 06:44:40.587941 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.587797    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="031e5d01-3e72-4e83-b588-7a2c626461bd" containerName="istio-proxy"
Sep 05 06:44:40.587941 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.587806    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="031e5d01-3e72-4e83-b588-7a2c626461bd" containerName="istio-proxy"
Sep 05 06:44:40.587941 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.587822    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62762379-6e50-4445-83ff-589cb15dcefa" containerName="istio-proxy"
Sep 05 06:44:40.587941 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.587830    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="62762379-6e50-4445-83ff-589cb15dcefa" containerName="istio-proxy"
Sep 05 06:44:40.587941 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.587923    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="01122e19-e181-459a-b771-f0f327786ab9" containerName="main"
Sep 05 06:44:40.587941 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.587936    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="62762379-6e50-4445-83ff-589cb15dcefa" containerName="istio-proxy"
Sep 05 06:44:40.588143 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.587944    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="031e5d01-3e72-4e83-b588-7a2c626461bd" containerName="istio-proxy"
Sep 05 06:44:40.591418 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.591401    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.594270 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.594249    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-183ae6\"/\"kube-root-ca.crt\""
Sep 05 06:44:40.595342 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.595323    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-183ae6\"/\"auth-test-model-183ae6-kserve-self-signed-certs\""
Sep 05 06:44:40.595450 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.595347    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-183ae6\"/\"default-dockercfg-hnd2s\""
Sep 05 06:44:40.595450 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.595400    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-183ae6\"/\"openshift-service-ca.crt\""
Sep 05 06:44:40.600962 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.600875    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"]
Sep 05 06:44:40.610570 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.610523    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb944c32-6868-48c2-a746-38f5c1c3ff83-tls-certs\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.610570 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.610563    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knkc4\" (UniqueName: \"kubernetes.io/projected/cb944c32-6868-48c2-a746-38f5c1c3ff83-kube-api-access-knkc4\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.610725 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.610709    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-home\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.610779 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.610743    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-dshm\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.610779 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.610773    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-tmp-dir\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.610911 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.610805    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-model-cache\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.711610 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.711582    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb944c32-6868-48c2-a746-38f5c1c3ff83-tls-certs\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.711610 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.711612    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knkc4\" (UniqueName: \"kubernetes.io/projected/cb944c32-6868-48c2-a746-38f5c1c3ff83-kube-api-access-knkc4\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.711785 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.711669    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-home\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.711785 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.711684    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-dshm\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.711785 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.711701    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-tmp-dir\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.711785 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.711719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-model-cache\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.712188 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.712155    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-home\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.712276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.712216    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-tmp-dir\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.712276 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.712216    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-model-cache\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.713876 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.713846    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-dshm\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.714097 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.714080    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb944c32-6868-48c2-a746-38f5c1c3ff83-tls-certs\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.721324 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.721303    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knkc4\" (UniqueName: \"kubernetes.io/projected/cb944c32-6868-48c2-a746-38f5c1c3ff83-kube-api-access-knkc4\") pod \"auth-test-model-183ae6-kserve-746f958df9-x72zf\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:40.902591 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:40.902521    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:41.179160 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:41.179051    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"]
Sep 05 06:44:41.181598 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:44:41.181566    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb944c32_6868_48c2_a746_38f5c1c3ff83.slice/crio-8e5e370ff8547b02eeadc88e55dbd42ff4c945d7d4f397f83a1a1f239794f0cc WatchSource:0}: Error finding container 8e5e370ff8547b02eeadc88e55dbd42ff4c945d7d4f397f83a1a1f239794f0cc: Status 404 returned error can't find the container with id 8e5e370ff8547b02eeadc88e55dbd42ff4c945d7d4f397f83a1a1f239794f0cc
Sep 05 06:44:41.914832 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:41.914794    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf" event={"ID":"cb944c32-6868-48c2-a746-38f5c1c3ff83","Type":"ContainerStarted","Data":"a14bdc6c6351679e0c073812f23a4b1f35b3d4e4a539cc9bdb6fce5acf784d30"}
Sep 05 06:44:41.914832 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:41.914831    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf" event={"ID":"cb944c32-6868-48c2-a746-38f5c1c3ff83","Type":"ContainerStarted","Data":"8e5e370ff8547b02eeadc88e55dbd42ff4c945d7d4f397f83a1a1f239794f0cc"}
Sep 05 06:44:41.934180 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:41.934133    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf" podStartSLOduration=1.934118741 podStartE2EDuration="1.934118741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:44:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:44:41.933872849 +0000 UTC m=+1098.524431801" watchObservedRunningTime="2026-09-05 06:44:41.934118741 +0000 UTC m=+1098.524677702"
Sep 05 06:44:46.594431 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:46.594398    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:44:49.357329 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.357299    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"]
Sep 05 06:44:49.357779 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.357695    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="031e5d01-3e72-4e83-b588-7a2c626461bd" containerName="istio-proxy"
Sep 05 06:44:49.360993 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.360970    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.363456 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.363435    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 05 06:44:49.363456 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.363453    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-5e47e8-openshift-default-dockercfg-6rrm9\""
Sep 05 06:44:49.379734 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.379709    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"]
Sep 05 06:44:49.479147 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.479111    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-envoy\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.479147 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.479149    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hnbl\" (UniqueName: \"kubernetes.io/projected/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-kube-api-access-6hnbl\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.479321 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.479211    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-token\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.479321 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.479258    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-data\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.479321 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.479280    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istiod-ca-cert\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.479321 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.479308    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-credential-socket\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.479471 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.479355    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-workload-socket\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.479471 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.479374    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-podinfo\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.479471 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.479408    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-workload-certs\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.580383 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.580352    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-data\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.580520 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.580393    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istiod-ca-cert\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.580520 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.580424    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-credential-socket\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.580520 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.580452    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-workload-socket\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.580520 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.580482    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-podinfo\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.580701 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.580525    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-workload-certs\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.580756 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.580712    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-envoy\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.580808 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.580752    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hnbl\" (UniqueName: \"kubernetes.io/projected/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-kube-api-access-6hnbl\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.580808 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.580783    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-token\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.580808 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.580799    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-data\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.580998 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.580837    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-credential-socket\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.580998 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.580932    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-workload-socket\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.581092 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.581013    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-workload-certs\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.581272 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.581252    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istiod-ca-cert\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.582839 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.582817    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-envoy\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.583107 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.583088    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-podinfo\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.591158 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.591139    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-token\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.591438 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.591418    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hnbl\" (UniqueName: \"kubernetes.io/projected/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-kube-api-access-6hnbl\") pod \"e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") " pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.671722 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.671664    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:49.963894 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.963855    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"]
Sep 05 06:44:49.964360 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:44:49.964337    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0fcf165_bddc_4260_b8e0_2cb4ffeba155.slice/crio-2a58664dad55151fdcd3189f3ce90b3ec9cf72927a92efa0863a9647afa91369 WatchSource:0}: Error finding container 2a58664dad55151fdcd3189f3ce90b3ec9cf72927a92efa0863a9647afa91369: Status 404 returned error can't find the container with id 2a58664dad55151fdcd3189f3ce90b3ec9cf72927a92efa0863a9647afa91369
Sep 05 06:44:49.966140 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.966106    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:44:49.966229 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.966169    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:44:49.966229 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:49.966195    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:44:50.903242 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:50.903204    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:50.903666 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:50.903251    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:50.915028 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:50.915000    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:50.963168 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:50.963133    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk" event={"ID":"d0fcf165-bddc-4260-b8e0-2cb4ffeba155","Type":"ContainerStarted","Data":"3facc81ab65157e1b06a3000e799c2859c0536441614e4f33463621373718806"}
Sep 05 06:44:50.963316 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:50.963188    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk" event={"ID":"d0fcf165-bddc-4260-b8e0-2cb4ffeba155","Type":"ContainerStarted","Data":"2a58664dad55151fdcd3189f3ce90b3ec9cf72927a92efa0863a9647afa91369"}
Sep 05 06:44:50.974155 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:50.974128    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:44:50.987271 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:50.987232    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk" podStartSLOduration=1.987218326 podStartE2EDuration="1.987218326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:44:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:44:50.986022466 +0000 UTC m=+1107.576581448" watchObservedRunningTime="2026-09-05 06:44:50.987218326 +0000 UTC m=+1107.577777347"
Sep 05 06:44:51.672733 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:51.672696    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:51.677794 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:51.677766    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:51.969304 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:51.969276    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:51.970105 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:51.970086    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:44:57.717927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:44:57.717879    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:45:00.147064 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:00.147017    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-39047b-29809845-l54dr"]
Sep 05 06:45:00.151395 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:00.151367    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-39047b-29809845-l54dr"
Sep 05 06:45:00.153140 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:00.153122    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809845-2q6dp"]
Sep 05 06:45:00.154204 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:00.154174    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slfqj\""
Sep 05 06:45:00.157009 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:00.156979    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-362c80-29809845-9f9fz"]
Sep 05 06:45:00.157135 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:00.157110    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809845-2q6dp"
Sep 05 06:45:00.164990 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:00.164969    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-39047b-29809845-l54dr"]
Sep 05 06:45:00.165237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:00.165220    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-362c80-29809845-9f9fz"
Sep 05 06:45:00.166142 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:00.166124    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809845-2q6dp"]
Sep 05 06:45:00.167877 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:00.167859    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-39047b-29809845-l54dr"
Sep 05 06:45:00.168952 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:00.168931    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-362c80-29809845-9f9fz"]
Sep 05 06:45:00.179060 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:00.178857    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809845-2q6dp"
Sep 05 06:45:00.216448 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:00.216416    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-362c80-29809845-9f9fz"
Sep 05 06:45:00.478818 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:00.478790    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-39047b-29809845-l54dr"]
Sep 05 06:45:00.480030 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:45:00.480003    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc9cd7f3_c024_4362_88c5_5ce6863721ff.slice/crio-b613fe7435e13aa91d95ded35bc93588f887581cfb3754995a8bc05b36a1beae WatchSource:0}: Error finding container b613fe7435e13aa91d95ded35bc93588f887581cfb3754995a8bc05b36a1beae: Status 404 returned error can't find the container with id b613fe7435e13aa91d95ded35bc93588f887581cfb3754995a8bc05b36a1beae
Sep 05 06:45:00.507561 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:00.507534    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809845-2q6dp"]
Sep 05 06:45:00.511005 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:45:00.510980    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08176018_d9cd_4592_80d4_aa6fc035d729.slice/crio-954df562ed7f9814237475679573e575fc9bfa089d68b4a57b2c4a8401e3d794 WatchSource:0}: Error finding container 954df562ed7f9814237475679573e575fc9bfa089d68b4a57b2c4a8401e3d794: Status 404 returned error can't find the container with id 954df562ed7f9814237475679573e575fc9bfa089d68b4a57b2c4a8401e3d794
Sep 05 06:45:00.542171 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:00.540959    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-362c80-29809845-9f9fz"]
Sep 05 06:45:01.026129 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:01.026026    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-39047b-29809845-l54dr" event={"ID":"fc9cd7f3-c024-4362-88c5-5ce6863721ff","Type":"ContainerStarted","Data":"7403dd2d642bfd6f324e1f4d7b209e9b77d2ec8ba4c96e250e94059135eba0cf"}
Sep 05 06:45:01.026129 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:01.026095    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-39047b-29809845-l54dr" event={"ID":"fc9cd7f3-c024-4362-88c5-5ce6863721ff","Type":"ContainerStarted","Data":"b613fe7435e13aa91d95ded35bc93588f887581cfb3754995a8bc05b36a1beae"}
Sep 05 06:45:01.029018 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:01.028984    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809845-2q6dp" event={"ID":"08176018-d9cd-4592-80d4-aa6fc035d729","Type":"ContainerStarted","Data":"55729a704fcd6b21273c64cd23fcc5995673d79f8f8e027e8989122896ed0a6f"}
Sep 05 06:45:01.029155 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:01.029043    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809845-2q6dp" event={"ID":"08176018-d9cd-4592-80d4-aa6fc035d729","Type":"ContainerStarted","Data":"954df562ed7f9814237475679573e575fc9bfa089d68b4a57b2c4a8401e3d794"}
Sep 05 06:45:01.035319 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:01.035288    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-362c80-29809845-9f9fz" event={"ID":"8a0a67d0-eb07-4aa3-a8ae-e6201c822379","Type":"ContainerStarted","Data":"17b2cc2d7aab554a29ac0aa4f820ee1401c489efb5bcd63c96a59de3b76bf0a1"}
Sep 05 06:45:01.035468 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:01.035346    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-362c80-29809845-9f9fz" event={"ID":"8a0a67d0-eb07-4aa3-a8ae-e6201c822379","Type":"ContainerStarted","Data":"4ad5bc758b6c473c07b8e3909df4f58d8094a11b949232b5d3c9c38cffbaecf8"}
Sep 05 06:45:01.048495 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:01.048432    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-39047b-29809845-l54dr" podStartSLOduration=1.048413878 podStartE2EDuration="1.048413878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:45:01.043788894 +0000 UTC m=+1117.634347846" watchObservedRunningTime="2026-09-05 06:45:01.048413878 +0000 UTC m=+1117.638972832"
Sep 05 06:45:01.062878 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:01.062827    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809845-2q6dp" podStartSLOduration=1.062814741 podStartE2EDuration="1.062814741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:45:01.060171198 +0000 UTC m=+1117.650730148" watchObservedRunningTime="2026-09-05 06:45:01.062814741 +0000 UTC m=+1117.653373689"
Sep 05 06:45:01.077192 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:01.077154    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-362c80-29809845-9f9fz" podStartSLOduration=1.077141956 podStartE2EDuration="1.077141956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:45:01.076993319 +0000 UTC m=+1117.667552267" watchObservedRunningTime="2026-09-05 06:45:01.077141956 +0000 UTC m=+1117.667700903"
Sep 05 06:45:02.049916 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:02.046152    2559 generic.go:356] "Generic (PLEG): container finished" podID="fc9cd7f3-c024-4362-88c5-5ce6863721ff" containerID="7403dd2d642bfd6f324e1f4d7b209e9b77d2ec8ba4c96e250e94059135eba0cf" exitCode=0
Sep 05 06:45:02.049916 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:02.046284    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-39047b-29809845-l54dr" event={"ID":"fc9cd7f3-c024-4362-88c5-5ce6863721ff","Type":"ContainerDied","Data":"7403dd2d642bfd6f324e1f4d7b209e9b77d2ec8ba4c96e250e94059135eba0cf"}
Sep 05 06:45:02.053943 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:02.053913    2559 generic.go:356] "Generic (PLEG): container finished" podID="08176018-d9cd-4592-80d4-aa6fc035d729" containerID="55729a704fcd6b21273c64cd23fcc5995673d79f8f8e027e8989122896ed0a6f" exitCode=0
Sep 05 06:45:02.054076 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:02.054017    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809845-2q6dp" event={"ID":"08176018-d9cd-4592-80d4-aa6fc035d729","Type":"ContainerDied","Data":"55729a704fcd6b21273c64cd23fcc5995673d79f8f8e027e8989122896ed0a6f"}
Sep 05 06:45:02.055726 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:02.055691    2559 generic.go:356] "Generic (PLEG): container finished" podID="8a0a67d0-eb07-4aa3-a8ae-e6201c822379" containerID="17b2cc2d7aab554a29ac0aa4f820ee1401c489efb5bcd63c96a59de3b76bf0a1" exitCode=0
Sep 05 06:45:02.055835 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:02.055751    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-362c80-29809845-9f9fz" event={"ID":"8a0a67d0-eb07-4aa3-a8ae-e6201c822379","Type":"ContainerDied","Data":"17b2cc2d7aab554a29ac0aa4f820ee1401c489efb5bcd63c96a59de3b76bf0a1"}
Sep 05 06:45:03.234916 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:03.233038    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-362c80-29809845-9f9fz"
Sep 05 06:45:03.282778 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:03.282726    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809845-2q6dp"
Sep 05 06:45:03.285776 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:03.285754    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-39047b-29809845-l54dr"
Sep 05 06:45:04.069568 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:04.069532    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-39047b-29809845-l54dr"
Sep 05 06:45:04.069568 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:04.069567    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-39047b-29809845-l54dr" event={"ID":"fc9cd7f3-c024-4362-88c5-5ce6863721ff","Type":"ContainerDied","Data":"b613fe7435e13aa91d95ded35bc93588f887581cfb3754995a8bc05b36a1beae"}
Sep 05 06:45:04.069793 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:04.069617    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b613fe7435e13aa91d95ded35bc93588f887581cfb3754995a8bc05b36a1beae"
Sep 05 06:45:04.070998 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:04.070980    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809845-2q6dp"
Sep 05 06:45:04.071129 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:04.071009    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809845-2q6dp" event={"ID":"08176018-d9cd-4592-80d4-aa6fc035d729","Type":"ContainerDied","Data":"954df562ed7f9814237475679573e575fc9bfa089d68b4a57b2c4a8401e3d794"}
Sep 05 06:45:04.071129 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:04.071053    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="954df562ed7f9814237475679573e575fc9bfa089d68b4a57b2c4a8401e3d794"
Sep 05 06:45:04.072469 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:04.072454    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-362c80-29809845-9f9fz"
Sep 05 06:45:04.072573 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:04.072500    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-362c80-29809845-9f9fz" event={"ID":"8a0a67d0-eb07-4aa3-a8ae-e6201c822379","Type":"ContainerDied","Data":"4ad5bc758b6c473c07b8e3909df4f58d8094a11b949232b5d3c9c38cffbaecf8"}
Sep 05 06:45:04.072573 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:04.072546    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ad5bc758b6c473c07b8e3909df4f58d8094a11b949232b5d3c9c38cffbaecf8"
Sep 05 06:45:08.013091 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:08.013048    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 05 06:45:08.013538 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:08.013131    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk" podUID="d0fcf165-bddc-4260-b8e0-2cb4ffeba155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 05 06:45:09.102924 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:09.102876    2559 generic.go:356] "Generic (PLEG): container finished" podID="d0fcf165-bddc-4260-b8e0-2cb4ffeba155" containerID="3facc81ab65157e1b06a3000e799c2859c0536441614e4f33463621373718806" exitCode=0
Sep 05 06:45:09.102924 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:09.102907    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk" event={"ID":"d0fcf165-bddc-4260-b8e0-2cb4ffeba155","Type":"ContainerDied","Data":"3facc81ab65157e1b06a3000e799c2859c0536441614e4f33463621373718806"}
Sep 05 06:45:09.103348 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:09.103335    2559 scope.go:117] "RemoveContainer" containerID="3facc81ab65157e1b06a3000e799c2859c0536441614e4f33463621373718806"
Sep 05 06:45:09.104496 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:09.104468    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:45:09.104584 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:09.104540    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:45:09.104584 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:09.104572    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:45:10.110875 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:10.110843    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk" event={"ID":"d0fcf165-bddc-4260-b8e0-2cb4ffeba155","Type":"ContainerStarted","Data":"cab2cf8f70c51ffcb7b57b577227a07743a49612c19d0c9ed6acac188a661b76"}
Sep 05 06:45:10.672594 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:10.672555    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:45:10.672830 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:10.672800    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 05 06:45:10.672904 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:10.672850    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk" podUID="d0fcf165-bddc-4260-b8e0-2cb4ffeba155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 05 06:45:11.672587 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:11.672543    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 05 06:45:11.672990 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:11.672622    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk" podUID="d0fcf165-bddc-4260-b8e0-2cb4ffeba155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 05 06:45:12.672804 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:12.672766    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 05 06:45:12.673195 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:12.672824    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk" podUID="d0fcf165-bddc-4260-b8e0-2cb4ffeba155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 05 06:45:13.672927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:13.672866    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 05 06:45:13.673311 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:13.672957    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk" podUID="d0fcf165-bddc-4260-b8e0-2cb4ffeba155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 05 06:45:14.672192 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:14.672152    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Sep 05 06:45:14.672370 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:14.672220    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk" podUID="d0fcf165-bddc-4260-b8e0-2cb4ffeba155" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Sep 05 06:45:15.675783 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:15.675752    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:45:15.676268 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:15.676154    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:45:15.677055 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:15.677030    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:45:18.732183 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:18.732150    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7738-139832200631904"]
Sep 05 06:45:18.732534 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:18.732523    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a0a67d0-eb07-4aa3-a8ae-e6201c822379" containerName="cleanup"
Sep 05 06:45:18.732576 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:18.732536    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a0a67d0-eb07-4aa3-a8ae-e6201c822379" containerName="cleanup"
Sep 05 06:45:18.732576 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:18.732545    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc9cd7f3-c024-4362-88c5-5ce6863721ff" containerName="cleanup"
Sep 05 06:45:18.732576 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:18.732550    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc9cd7f3-c024-4362-88c5-5ce6863721ff" containerName="cleanup"
Sep 05 06:45:18.732576 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:18.732560    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08176018-d9cd-4592-80d4-aa6fc035d729" containerName="cleanup"
Sep 05 06:45:18.732576 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:18.732565    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="08176018-d9cd-4592-80d4-aa6fc035d729" containerName="cleanup"
Sep 05 06:45:18.732727 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:18.732616    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a0a67d0-eb07-4aa3-a8ae-e6201c822379" containerName="cleanup"
Sep 05 06:45:18.732727 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:18.732626    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc9cd7f3-c024-4362-88c5-5ce6863721ff" containerName="cleanup"
Sep 05 06:45:18.732727 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:18.732632    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="08176018-d9cd-4592-80d4-aa6fc035d729" containerName="cleanup"
Sep 05 06:45:18.735955 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:18.735929    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7738-139832200631904"
Sep 05 06:45:18.738594 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:18.738573    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wwpm9\""
Sep 05 06:45:18.750988 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:18.745286    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7738-139832200631904"]
Sep 05 06:45:18.845767 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:18.845728    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9n9j\" (UniqueName: \"kubernetes.io/projected/30ee6389-5282-4fb2-b10b-cf209ca6e32c-kube-api-access-f9n9j\") pod \"test-curl-7738-139832200631904\" (UID: \"30ee6389-5282-4fb2-b10b-cf209ca6e32c\") " pod="opendatahub/test-curl-7738-139832200631904"
Sep 05 06:45:18.946965 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:18.946931    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9n9j\" (UniqueName: \"kubernetes.io/projected/30ee6389-5282-4fb2-b10b-cf209ca6e32c-kube-api-access-f9n9j\") pod \"test-curl-7738-139832200631904\" (UID: \"30ee6389-5282-4fb2-b10b-cf209ca6e32c\") " pod="opendatahub/test-curl-7738-139832200631904"
Sep 05 06:45:18.956538 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:18.956507    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9n9j\" (UniqueName: \"kubernetes.io/projected/30ee6389-5282-4fb2-b10b-cf209ca6e32c-kube-api-access-f9n9j\") pod \"test-curl-7738-139832200631904\" (UID: \"30ee6389-5282-4fb2-b10b-cf209ca6e32c\") " pod="opendatahub/test-curl-7738-139832200631904"
Sep 05 06:45:19.049538 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:19.049476    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7738-139832200631904"
Sep 05 06:45:19.318470 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:19.318436    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7738-139832200631904"]
Sep 05 06:45:19.319691 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:45:19.319666    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30ee6389_5282_4fb2_b10b_cf209ca6e32c.slice/crio-64cb9b6dd36b339934cf1db1596a2ed1ac2d2cf3e0fb31514441d0c9cab71789 WatchSource:0}: Error finding container 64cb9b6dd36b339934cf1db1596a2ed1ac2d2cf3e0fb31514441d0c9cab71789: Status 404 returned error can't find the container with id 64cb9b6dd36b339934cf1db1596a2ed1ac2d2cf3e0fb31514441d0c9cab71789
Sep 05 06:45:19.321186 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:19.321165    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 06:45:20.178441 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:20.178405    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7738-139832200631904" event={"ID":"30ee6389-5282-4fb2-b10b-cf209ca6e32c","Type":"ContainerStarted","Data":"64cb9b6dd36b339934cf1db1596a2ed1ac2d2cf3e0fb31514441d0c9cab71789"}
Sep 05 06:45:22.196273 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:22.196240    2559 generic.go:356] "Generic (PLEG): container finished" podID="30ee6389-5282-4fb2-b10b-cf209ca6e32c" containerID="4ea34d4cde91b6668d1c0a43dcd71737f4f61c219fca89f0ef16c0e9631c423d" exitCode=0
Sep 05 06:45:22.196642 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:22.196306    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7738-139832200631904" event={"ID":"30ee6389-5282-4fb2-b10b-cf209ca6e32c","Type":"ContainerDied","Data":"4ea34d4cde91b6668d1c0a43dcd71737f4f61c219fca89f0ef16c0e9631c423d"}
Sep 05 06:45:23.331646 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.331621    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7738-139832200631904"
Sep 05 06:45:23.370015 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.369986    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7738-139832200631904_30ee6389-5282-4fb2-b10b-cf209ca6e32c/test-curl-7738-139832200631904/0.log"
Sep 05 06:45:23.396969 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.396935    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7738-139832200631904"]
Sep 05 06:45:23.401395 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.401370    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7738-139832200631904"]
Sep 05 06:45:23.488194 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.488103    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9n9j\" (UniqueName: \"kubernetes.io/projected/30ee6389-5282-4fb2-b10b-cf209ca6e32c-kube-api-access-f9n9j\") pod \"30ee6389-5282-4fb2-b10b-cf209ca6e32c\" (UID: \"30ee6389-5282-4fb2-b10b-cf209ca6e32c\") "
Sep 05 06:45:23.490210 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.490184    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30ee6389-5282-4fb2-b10b-cf209ca6e32c-kube-api-access-f9n9j" (OuterVolumeSpecName: "kube-api-access-f9n9j") pod "30ee6389-5282-4fb2-b10b-cf209ca6e32c" (UID: "30ee6389-5282-4fb2-b10b-cf209ca6e32c"). InnerVolumeSpecName "kube-api-access-f9n9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:45:23.520391 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.520363    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7738-139832200631904"]
Sep 05 06:45:23.520719 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.520706    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30ee6389-5282-4fb2-b10b-cf209ca6e32c" containerName="test-curl-7738-139832200631904"
Sep 05 06:45:23.520772 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.520721    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="30ee6389-5282-4fb2-b10b-cf209ca6e32c" containerName="test-curl-7738-139832200631904"
Sep 05 06:45:23.520813 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.520786    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="30ee6389-5282-4fb2-b10b-cf209ca6e32c" containerName="test-curl-7738-139832200631904"
Sep 05 06:45:23.523624 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.523610    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7738-139832200631904"
Sep 05 06:45:23.528874 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.528852    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7738-139832200631904"]
Sep 05 06:45:23.589653 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.589616    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9n9j\" (UniqueName: \"kubernetes.io/projected/30ee6389-5282-4fb2-b10b-cf209ca6e32c-kube-api-access-f9n9j\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:45:23.690209 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.690173    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lqcr\" (UniqueName: \"kubernetes.io/projected/df44fad8-b9c4-40b2-b082-12b9bf898b13-kube-api-access-5lqcr\") pod \"test-curl-7738-139832200631904\" (UID: \"df44fad8-b9c4-40b2-b082-12b9bf898b13\") " pod="opendatahub/test-curl-7738-139832200631904"
Sep 05 06:45:23.790666 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.790575    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lqcr\" (UniqueName: \"kubernetes.io/projected/df44fad8-b9c4-40b2-b082-12b9bf898b13-kube-api-access-5lqcr\") pod \"test-curl-7738-139832200631904\" (UID: \"df44fad8-b9c4-40b2-b082-12b9bf898b13\") " pod="opendatahub/test-curl-7738-139832200631904"
Sep 05 06:45:23.800414 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.800389    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lqcr\" (UniqueName: \"kubernetes.io/projected/df44fad8-b9c4-40b2-b082-12b9bf898b13-kube-api-access-5lqcr\") pod \"test-curl-7738-139832200631904\" (UID: \"df44fad8-b9c4-40b2-b082-12b9bf898b13\") " pod="opendatahub/test-curl-7738-139832200631904"
Sep 05 06:45:23.834429 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.834404    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7738-139832200631904"
Sep 05 06:45:23.895980 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.895854    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30ee6389-5282-4fb2-b10b-cf209ca6e32c" path="/var/lib/kubelet/pods/30ee6389-5282-4fb2-b10b-cf209ca6e32c/volumes"
Sep 05 06:45:23.966342 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:23.966313    2559 scope.go:117] "RemoveContainer" containerID="56f779f12e7412541fb192e0ac9a516dbdfe88f3bf8ea1b47480db21a56b3efe"
Sep 05 06:45:24.107583 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:24.107557    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7738-139832200631904"]
Sep 05 06:45:24.109745 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:45:24.109714    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf44fad8_b9c4_40b2_b082_12b9bf898b13.slice/crio-6afe20d531ce069f03a5ff37d23b30eb69d44433c67fa2128f474ce63e2e31c6 WatchSource:0}: Error finding container 6afe20d531ce069f03a5ff37d23b30eb69d44433c67fa2128f474ce63e2e31c6: Status 404 returned error can't find the container with id 6afe20d531ce069f03a5ff37d23b30eb69d44433c67fa2128f474ce63e2e31c6
Sep 05 06:45:24.208850 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:24.208824    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7738-139832200631904"
Sep 05 06:45:24.209024 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:24.208861    2559 scope.go:117] "RemoveContainer" containerID="4ea34d4cde91b6668d1c0a43dcd71737f4f61c219fca89f0ef16c0e9631c423d"
Sep 05 06:45:24.210367 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:24.210341    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7738-139832200631904" event={"ID":"df44fad8-b9c4-40b2-b082-12b9bf898b13","Type":"ContainerStarted","Data":"6afe20d531ce069f03a5ff37d23b30eb69d44433c67fa2128f474ce63e2e31c6"}
Sep 05 06:45:25.218355 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:25.218317    2559 generic.go:356] "Generic (PLEG): container finished" podID="df44fad8-b9c4-40b2-b082-12b9bf898b13" containerID="1edbab3a9638ee6abebf492029e08a5e28ad011425d22369dba2eb9959e44a8c" exitCode=0
Sep 05 06:45:25.218719 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:25.218394    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7738-139832200631904" event={"ID":"df44fad8-b9c4-40b2-b082-12b9bf898b13","Type":"ContainerDied","Data":"1edbab3a9638ee6abebf492029e08a5e28ad011425d22369dba2eb9959e44a8c"}
Sep 05 06:45:26.353524 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:26.353500    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7738-139832200631904"
Sep 05 06:45:26.379915 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:26.379828    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7738-139832200631904_df44fad8-b9c4-40b2-b082-12b9bf898b13/test-curl-7738-139832200631904/0.log"
Sep 05 06:45:26.406605 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:26.406577    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7738-139832200631904"]
Sep 05 06:45:26.410769 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:26.410743    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7738-139832200631904"]
Sep 05 06:45:26.514196 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:26.514127    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lqcr\" (UniqueName: \"kubernetes.io/projected/df44fad8-b9c4-40b2-b082-12b9bf898b13-kube-api-access-5lqcr\") pod \"df44fad8-b9c4-40b2-b082-12b9bf898b13\" (UID: \"df44fad8-b9c4-40b2-b082-12b9bf898b13\") "
Sep 05 06:45:26.516018 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:26.515996    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df44fad8-b9c4-40b2-b082-12b9bf898b13-kube-api-access-5lqcr" (OuterVolumeSpecName: "kube-api-access-5lqcr") pod "df44fad8-b9c4-40b2-b082-12b9bf898b13" (UID: "df44fad8-b9c4-40b2-b082-12b9bf898b13"). InnerVolumeSpecName "kube-api-access-5lqcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:45:26.545719 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:26.545683    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7738-139832200632480"]
Sep 05 06:45:26.546135 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:26.546116    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df44fad8-b9c4-40b2-b082-12b9bf898b13" containerName="test-curl-7738-139832200631904"
Sep 05 06:45:26.546224 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:26.546137    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="df44fad8-b9c4-40b2-b082-12b9bf898b13" containerName="test-curl-7738-139832200631904"
Sep 05 06:45:26.546280 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:26.546232    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="df44fad8-b9c4-40b2-b082-12b9bf898b13" containerName="test-curl-7738-139832200631904"
Sep 05 06:45:26.549166 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:26.549147    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7738-139832200632480"
Sep 05 06:45:26.555469 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:26.555448    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7738-139832200632480"]
Sep 05 06:45:26.615525 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:26.615506    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lqcr\" (UniqueName: \"kubernetes.io/projected/df44fad8-b9c4-40b2-b082-12b9bf898b13-kube-api-access-5lqcr\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:45:26.716597 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:26.716571    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhtvl\" (UniqueName: \"kubernetes.io/projected/44b99092-e726-4fb2-82f0-592b36721d0c-kube-api-access-hhtvl\") pod \"test-curl-7738-139832200632480\" (UID: \"44b99092-e726-4fb2-82f0-592b36721d0c\") " pod="opendatahub/test-curl-7738-139832200632480"
Sep 05 06:45:26.817707 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:26.817629    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhtvl\" (UniqueName: \"kubernetes.io/projected/44b99092-e726-4fb2-82f0-592b36721d0c-kube-api-access-hhtvl\") pod \"test-curl-7738-139832200632480\" (UID: \"44b99092-e726-4fb2-82f0-592b36721d0c\") " pod="opendatahub/test-curl-7738-139832200632480"
Sep 05 06:45:26.827399 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:26.827366    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhtvl\" (UniqueName: \"kubernetes.io/projected/44b99092-e726-4fb2-82f0-592b36721d0c-kube-api-access-hhtvl\") pod \"test-curl-7738-139832200632480\" (UID: \"44b99092-e726-4fb2-82f0-592b36721d0c\") " pod="opendatahub/test-curl-7738-139832200632480"
Sep 05 06:45:26.859768 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:26.859745    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7738-139832200632480"
Sep 05 06:45:27.130022 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:27.129985    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7738-139832200632480"]
Sep 05 06:45:27.130755 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:45:27.130730    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44b99092_e726_4fb2_82f0_592b36721d0c.slice/crio-0c6609a3ab1f73679d113b6f00775655450cf0ff43f1897f965f33acb701998c WatchSource:0}: Error finding container 0c6609a3ab1f73679d113b6f00775655450cf0ff43f1897f965f33acb701998c: Status 404 returned error can't find the container with id 0c6609a3ab1f73679d113b6f00775655450cf0ff43f1897f965f33acb701998c
Sep 05 06:45:27.231969 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:27.231946    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7738-139832200631904"
Sep 05 06:45:27.232090 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:27.232002    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6afe20d531ce069f03a5ff37d23b30eb69d44433c67fa2128f474ce63e2e31c6"
Sep 05 06:45:27.233401 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:27.233378    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7738-139832200632480" event={"ID":"44b99092-e726-4fb2-82f0-592b36721d0c","Type":"ContainerStarted","Data":"0c6609a3ab1f73679d113b6f00775655450cf0ff43f1897f965f33acb701998c"}
Sep 05 06:45:27.247206 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:45:27.247175    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7738-139832200631904\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="df44fad8-b9c4-40b2-b082-12b9bf898b13" pod="opendatahub/test-curl-7738-139832200631904"
Sep 05 06:45:27.891983 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:27.891951    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df44fad8-b9c4-40b2-b082-12b9bf898b13" path="/var/lib/kubelet/pods/df44fad8-b9c4-40b2-b082-12b9bf898b13/volumes"
Sep 05 06:45:28.240044 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:28.240011    2559 generic.go:356] "Generic (PLEG): container finished" podID="44b99092-e726-4fb2-82f0-592b36721d0c" containerID="76cf217c2c590ca5f51feeec0c4afcef3884cb5fbf40242d8e9be58b49ba1938" exitCode=0
Sep 05 06:45:28.240190 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:28.240104    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7738-139832200632480" event={"ID":"44b99092-e726-4fb2-82f0-592b36721d0c","Type":"ContainerDied","Data":"76cf217c2c590ca5f51feeec0c4afcef3884cb5fbf40242d8e9be58b49ba1938"}
Sep 05 06:45:29.370868 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.370843    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7738-139832200632480"
Sep 05 06:45:29.393316 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.393294    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7738-139832200632480_44b99092-e726-4fb2-82f0-592b36721d0c/test-curl-7738-139832200632480/0.log"
Sep 05 06:45:29.420294 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.420271    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7738-139832200632480"]
Sep 05 06:45:29.422448 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.422428    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7738-139832200632480"]
Sep 05 06:45:29.539863 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.539793    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhtvl\" (UniqueName: \"kubernetes.io/projected/44b99092-e726-4fb2-82f0-592b36721d0c-kube-api-access-hhtvl\") pod \"44b99092-e726-4fb2-82f0-592b36721d0c\" (UID: \"44b99092-e726-4fb2-82f0-592b36721d0c\") "
Sep 05 06:45:29.541725 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.541701    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44b99092-e726-4fb2-82f0-592b36721d0c-kube-api-access-hhtvl" (OuterVolumeSpecName: "kube-api-access-hhtvl") pod "44b99092-e726-4fb2-82f0-592b36721d0c" (UID: "44b99092-e726-4fb2-82f0-592b36721d0c"). InnerVolumeSpecName "kube-api-access-hhtvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:45:29.549034 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.549008    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7738-139832200632480"]
Sep 05 06:45:29.549377 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.549365    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44b99092-e726-4fb2-82f0-592b36721d0c" containerName="test-curl-7738-139832200632480"
Sep 05 06:45:29.549424 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.549379    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="44b99092-e726-4fb2-82f0-592b36721d0c" containerName="test-curl-7738-139832200632480"
Sep 05 06:45:29.549460 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.549436    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="44b99092-e726-4fb2-82f0-592b36721d0c" containerName="test-curl-7738-139832200632480"
Sep 05 06:45:29.552420 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.552405    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7738-139832200632480"
Sep 05 06:45:29.559853 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.559835    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7738-139832200632480"]
Sep 05 06:45:29.641176 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.641155    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhtvl\" (UniqueName: \"kubernetes.io/projected/44b99092-e726-4fb2-82f0-592b36721d0c-kube-api-access-hhtvl\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:45:29.742458 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.742429    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q9ct\" (UniqueName: \"kubernetes.io/projected/51e8a9e9-beff-42f8-b0b1-97f68c418998-kube-api-access-7q9ct\") pod \"test-curl-7738-139832200632480\" (UID: \"51e8a9e9-beff-42f8-b0b1-97f68c418998\") " pod="opendatahub/test-curl-7738-139832200632480"
Sep 05 06:45:29.843742 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.843668    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q9ct\" (UniqueName: \"kubernetes.io/projected/51e8a9e9-beff-42f8-b0b1-97f68c418998-kube-api-access-7q9ct\") pod \"test-curl-7738-139832200632480\" (UID: \"51e8a9e9-beff-42f8-b0b1-97f68c418998\") " pod="opendatahub/test-curl-7738-139832200632480"
Sep 05 06:45:29.854363 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.854338    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q9ct\" (UniqueName: \"kubernetes.io/projected/51e8a9e9-beff-42f8-b0b1-97f68c418998-kube-api-access-7q9ct\") pod \"test-curl-7738-139832200632480\" (UID: \"51e8a9e9-beff-42f8-b0b1-97f68c418998\") " pod="opendatahub/test-curl-7738-139832200632480"
Sep 05 06:45:29.862102 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.862082    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7738-139832200632480"
Sep 05 06:45:29.893775 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:29.893750    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44b99092-e726-4fb2-82f0-592b36721d0c" path="/var/lib/kubelet/pods/44b99092-e726-4fb2-82f0-592b36721d0c/volumes"
Sep 05 06:45:30.134385 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:30.134363    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7738-139832200632480"]
Sep 05 06:45:30.135492 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:45:30.135469    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51e8a9e9_beff_42f8_b0b1_97f68c418998.slice/crio-f1c0233c2a3066bbec4b82a83bca1d13b7653d71c8e203da6ebb039173222448 WatchSource:0}: Error finding container f1c0233c2a3066bbec4b82a83bca1d13b7653d71c8e203da6ebb039173222448: Status 404 returned error can't find the container with id f1c0233c2a3066bbec4b82a83bca1d13b7653d71c8e203da6ebb039173222448
Sep 05 06:45:30.252814 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:30.252778    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7738-139832200632480" event={"ID":"51e8a9e9-beff-42f8-b0b1-97f68c418998","Type":"ContainerStarted","Data":"f1c0233c2a3066bbec4b82a83bca1d13b7653d71c8e203da6ebb039173222448"}
Sep 05 06:45:30.254221 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:30.254197    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7738-139832200632480"
Sep 05 06:45:30.254360 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:30.254230    2559 scope.go:117] "RemoveContainer" containerID="76cf217c2c590ca5f51feeec0c4afcef3884cb5fbf40242d8e9be58b49ba1938"
Sep 05 06:45:31.261343 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:31.261310    2559 generic.go:356] "Generic (PLEG): container finished" podID="51e8a9e9-beff-42f8-b0b1-97f68c418998" containerID="632c38c5d308f849656d7d91ca40de006896083a6c6482eba0a2fc8cac1bb9f7" exitCode=0
Sep 05 06:45:31.261847 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:31.261395    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7738-139832200632480" event={"ID":"51e8a9e9-beff-42f8-b0b1-97f68c418998","Type":"ContainerDied","Data":"632c38c5d308f849656d7d91ca40de006896083a6c6482eba0a2fc8cac1bb9f7"}
Sep 05 06:45:32.399772 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:32.399747    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7738-139832200632480"
Sep 05 06:45:32.428458 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:32.428411    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7738-139832200632480_51e8a9e9-beff-42f8-b0b1-97f68c418998/test-curl-7738-139832200632480/0.log"
Sep 05 06:45:32.454407 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:32.454378    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7738-139832200632480"]
Sep 05 06:45:32.462299 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:32.462277    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7738-139832200632480"]
Sep 05 06:45:32.462556 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:32.462537    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q9ct\" (UniqueName: \"kubernetes.io/projected/51e8a9e9-beff-42f8-b0b1-97f68c418998-kube-api-access-7q9ct\") pod \"51e8a9e9-beff-42f8-b0b1-97f68c418998\" (UID: \"51e8a9e9-beff-42f8-b0b1-97f68c418998\") "
Sep 05 06:45:32.464499 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:32.464475    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51e8a9e9-beff-42f8-b0b1-97f68c418998-kube-api-access-7q9ct" (OuterVolumeSpecName: "kube-api-access-7q9ct") pod "51e8a9e9-beff-42f8-b0b1-97f68c418998" (UID: "51e8a9e9-beff-42f8-b0b1-97f68c418998"). InnerVolumeSpecName "kube-api-access-7q9ct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:45:32.563783 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:32.563725    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q9ct\" (UniqueName: \"kubernetes.io/projected/51e8a9e9-beff-42f8-b0b1-97f68c418998-kube-api-access-7q9ct\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:45:33.274728 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.274698    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1c0233c2a3066bbec4b82a83bca1d13b7653d71c8e203da6ebb039173222448"
Sep 05 06:45:33.274728 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.274700    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7738-139832200632480"
Sep 05 06:45:33.284456 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:45:33.284429    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7738-139832200632480\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="51e8a9e9-beff-42f8-b0b1-97f68c418998" pod="opendatahub/test-curl-7738-139832200632480"
Sep 05 06:45:33.400213 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.400185    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"]
Sep 05 06:45:33.400604 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.400586    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51e8a9e9-beff-42f8-b0b1-97f68c418998" containerName="test-curl-7738-139832200632480"
Sep 05 06:45:33.400654 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.400608    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="51e8a9e9-beff-42f8-b0b1-97f68c418998" containerName="test-curl-7738-139832200632480"
Sep 05 06:45:33.400714 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.400705    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="51e8a9e9-beff-42f8-b0b1-97f68c418998" containerName="test-curl-7738-139832200632480"
Sep 05 06:45:33.403775 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.403760    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.406343 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:45:33.406312    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7738-139832200632480\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="51e8a9e9-beff-42f8-b0b1-97f68c418998" pod="opendatahub/test-curl-7738-139832200632480"
Sep 05 06:45:33.406343 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.406336    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-87058ae4-openshift-default-dockercfg-9k5hm\""
Sep 05 06:45:33.413352 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.413326    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"]
Sep 05 06:45:33.467850 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.467820    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istiod-ca-cert\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.468009 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.467904    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-262zt\" (UniqueName: \"kubernetes.io/projected/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-kube-api-access-262zt\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.468009 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.467955    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-credential-socket\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.468009 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.467978    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-podinfo\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.468009 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.468006    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-workload-socket\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.468171 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.468031    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-workload-certs\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.468171 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.468087    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-envoy\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.468171 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.468126    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-data\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.468171 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.468157    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-token\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.569031 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.568964    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-262zt\" (UniqueName: \"kubernetes.io/projected/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-kube-api-access-262zt\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.569165 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.569014    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-credential-socket\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.569165 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.569084    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-podinfo\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.569165 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.569111    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-workload-socket\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.569165 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.569130    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-workload-certs\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.569165 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.569149    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-envoy\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.569445 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.569187    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-data\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.569445 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.569222    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-token\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.569445 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.569263    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istiod-ca-cert\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.569445 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.569371    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-credential-socket\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.569445 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.569439    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-workload-socket\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.569667 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.569513    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-workload-certs\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.569755 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.569734    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-data\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.569902 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.569867    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istiod-ca-cert\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.571500 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.571481    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-envoy\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.571862 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.571846    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-podinfo\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.579762 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.579738    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-token\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.579920 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.579898    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-262zt\" (UniqueName: \"kubernetes.io/projected/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-kube-api-access-262zt\") pod \"e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") " pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.714311 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.714287    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:33.891427 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:45:33.891349    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7738-139832200632480\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="51e8a9e9-beff-42f8-b0b1-97f68c418998" pod="opendatahub/test-curl-7738-139832200632480"
Sep 05 06:45:33.893512 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.893491    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51e8a9e9-beff-42f8-b0b1-97f68c418998" path="/var/lib/kubelet/pods/51e8a9e9-beff-42f8-b0b1-97f68c418998/volumes"
Sep 05 06:45:33.995730 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.995701    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"]
Sep 05 06:45:33.997092 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:45:33.997065    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b3f89ff_708c_4679_9a49_1e00adc4e4a2.slice/crio-e7896fe917f01f3ec5548b73800e15340e52d263f8a7fa9eac0adbc032cc0a32 WatchSource:0}: Error finding container e7896fe917f01f3ec5548b73800e15340e52d263f8a7fa9eac0adbc032cc0a32: Status 404 returned error can't find the container with id e7896fe917f01f3ec5548b73800e15340e52d263f8a7fa9eac0adbc032cc0a32
Sep 05 06:45:33.999419 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.999388    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:45:33.999472 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.999455    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:45:33.999512 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:33.999485    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:45:34.289356 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.289320    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j" event={"ID":"2b3f89ff-708c-4679-9a49-1e00adc4e4a2","Type":"ContainerStarted","Data":"4d1f75ac6bea35948e3a28e9993e1fe7e58cd45bbce46ace4e3564e97432c76d"}
Sep 05 06:45:34.289564 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.289399    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j" event={"ID":"2b3f89ff-708c-4679-9a49-1e00adc4e4a2","Type":"ContainerStarted","Data":"e7896fe917f01f3ec5548b73800e15340e52d263f8a7fa9eac0adbc032cc0a32"}
Sep 05 06:45:34.313492 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.313445    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j" podStartSLOduration=1.313428219 podStartE2EDuration="1.313428219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:45:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:45:34.312216218 +0000 UTC m=+1150.902775168" watchObservedRunningTime="2026-09-05 06:45:34.313428219 +0000 UTC m=+1150.903987170"
Sep 05 06:45:34.466866 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.466830    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"]
Sep 05 06:45:34.470324 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.470302    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.473157 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.473132    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-39047b\"/\"openshift-service-ca.crt\""
Sep 05 06:45:34.473260 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.473162    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-39047b\"/\"default-dockercfg-cskxb\""
Sep 05 06:45:34.473260 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.473174    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-39047b\"/\"kube-root-ca.crt\""
Sep 05 06:45:34.473260 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.473162    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-39047b\"/\"auth-test-model-39047b-kserve-self-signed-certs\""
Sep 05 06:45:34.476093 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.476071    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9t7k\" (UniqueName: \"kubernetes.io/projected/0435b1df-61d7-4726-abac-1810595333d9-kube-api-access-f9t7k\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.476205 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.476110    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-dshm\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.476205 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.476156    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-home\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.476322 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.476223    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-tmp-dir\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.476322 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.476288    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-model-cache\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.476430 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.476328    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0435b1df-61d7-4726-abac-1810595333d9-tls-certs\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.478632 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.478592    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"]
Sep 05 06:45:34.577143 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.577069    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9t7k\" (UniqueName: \"kubernetes.io/projected/0435b1df-61d7-4726-abac-1810595333d9-kube-api-access-f9t7k\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.577143 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.577119    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-dshm\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.577347 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.577166    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-home\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.577347 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.577243    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-tmp-dir\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.577347 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.577321    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-model-cache\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.577347 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.577345    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0435b1df-61d7-4726-abac-1810595333d9-tls-certs\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.577620 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.577593    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-home\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.577691 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.577643    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-model-cache\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.577829 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.577803    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-tmp-dir\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.579513 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.579490    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-dshm\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.580098 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.580078    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0435b1df-61d7-4726-abac-1810595333d9-tls-certs\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.586265 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.586243    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9t7k\" (UniqueName: \"kubernetes.io/projected/0435b1df-61d7-4726-abac-1810595333d9-kube-api-access-f9t7k\") pod \"auth-test-model-39047b-kserve-6f997ff987-9dktc\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") " pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:34.714904 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.714856    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:34.781879 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:34.781846    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:35.056131 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:35.056106    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"]
Sep 05 06:45:35.057247 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:45:35.057224    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0435b1df_61d7_4726_abac_1810595333d9.slice/crio-c4eecf9e2c151720c7995362c885f8c026ce7c9eab58135ba25c28c598415b9c WatchSource:0}: Error finding container c4eecf9e2c151720c7995362c885f8c026ce7c9eab58135ba25c28c598415b9c: Status 404 returned error can't find the container with id c4eecf9e2c151720c7995362c885f8c026ce7c9eab58135ba25c28c598415b9c
Sep 05 06:45:35.298211 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:35.298179    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc" event={"ID":"0435b1df-61d7-4726-abac-1810595333d9","Type":"ContainerStarted","Data":"d5ef3487737a67ecc3d5b6a96b2401d8577d91761e527fd8caadcb2a7113847a"}
Sep 05 06:45:35.298402 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:35.298222    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc" event={"ID":"0435b1df-61d7-4726-abac-1810595333d9","Type":"ContainerStarted","Data":"c4eecf9e2c151720c7995362c885f8c026ce7c9eab58135ba25c28c598415b9c"}
Sep 05 06:45:35.317980 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:35.317927    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc" podStartSLOduration=1.317909348 podStartE2EDuration="1.317909348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:45:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:45:35.31746653 +0000 UTC m=+1151.908025482" watchObservedRunningTime="2026-09-05 06:45:35.317909348 +0000 UTC m=+1151.908468299"
Sep 05 06:45:35.720130 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:35.720096    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:36.304579 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:36.304541    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:36.305251 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:36.305232    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:40.092970 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:40.092932    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:45:44.782613 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:44.782577    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:44.783031 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:44.782625    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:44.794550 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:44.794531    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:45.381320 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:45.381291    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:45:55.810307 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:55.810265    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:45:58.447695 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:58.447651    2559 generic.go:356] "Generic (PLEG): container finished" podID="2b3f89ff-708c-4679-9a49-1e00adc4e4a2" containerID="4d1f75ac6bea35948e3a28e9993e1fe7e58cd45bbce46ace4e3564e97432c76d" exitCode=0
Sep 05 06:45:58.448111 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:58.447725    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j" event={"ID":"2b3f89ff-708c-4679-9a49-1e00adc4e4a2","Type":"ContainerDied","Data":"4d1f75ac6bea35948e3a28e9993e1fe7e58cd45bbce46ace4e3564e97432c76d"}
Sep 05 06:45:58.448111 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:58.448056    2559 scope.go:117] "RemoveContainer" containerID="4d1f75ac6bea35948e3a28e9993e1fe7e58cd45bbce46ace4e3564e97432c76d"
Sep 05 06:45:58.449037 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:58.449011    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:45:58.449097 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:58.449074    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:45:58.449147 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:58.449104    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:45:59.455649 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:59.455607    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j" event={"ID":"2b3f89ff-708c-4679-9a49-1e00adc4e4a2","Type":"ContainerStarted","Data":"768f3071c3a6804dc5bc3e1a495b61fdee7687877daa62ebdd05ab689e461dcb"}
Sep 05 06:45:59.715270 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:59.715182    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:45:59.715429 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:59.715405    2559 patch_prober.go:28] interesting pod/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 05 06:45:59.715478 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:45:59.715455    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j" podUID="2b3f89ff-708c-4679-9a49-1e00adc4e4a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 05 06:46:00.714799 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:00.714765    2559 patch_prober.go:28] interesting pod/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 05 06:46:00.715203 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:00.714833    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j" podUID="2b3f89ff-708c-4679-9a49-1e00adc4e4a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 05 06:46:01.715126 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:01.715093    2559 patch_prober.go:28] interesting pod/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 05 06:46:01.715506 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:01.715151    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j" podUID="2b3f89ff-708c-4679-9a49-1e00adc4e4a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 05 06:46:02.714809 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:02.714767    2559 patch_prober.go:28] interesting pod/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 05 06:46:02.715085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:02.714827    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j" podUID="2b3f89ff-708c-4679-9a49-1e00adc4e4a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 05 06:46:03.714963 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:03.714925    2559 patch_prober.go:28] interesting pod/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 05 06:46:03.715337 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:03.714983    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j" podUID="2b3f89ff-708c-4679-9a49-1e00adc4e4a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 05 06:46:04.715529 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:04.715489    2559 patch_prober.go:28] interesting pod/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 05 06:46:04.715931 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:04.715561    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j" podUID="2b3f89ff-708c-4679-9a49-1e00adc4e4a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 05 06:46:05.718464 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:05.718435    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:46:05.718904 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:05.718722    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:46:05.719384 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:05.719366    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:46:11.583222 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:11.583191    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:46:23.987704 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:23.987674    2559 scope.go:117] "RemoveContainer" containerID="7318916fcb8f06969bd86f0f62b3c115a66ace7e4b57a0af67bbd9bae547b2d1"
Sep 05 06:46:26.790065 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:26.790031    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:46:29.683610 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.683569    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"]
Sep 05 06:46:29.687113 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.687093    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.690198 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.690180    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-87058ae4\"/\"kube-root-ca.crt\""
Sep 05 06:46:29.690682 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.690666    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-87058ae4\"/\"openshift-service-ca.crt\""
Sep 05 06:46:29.691357 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.691339    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-87058ae4\"/\"default-dockercfg-wbhch\""
Sep 05 06:46:29.691466 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.691406    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-87058ae4\"/\"e2e-lifecycle-model-87058ae4-kserve-self-signed-certs\""
Sep 05 06:46:29.701125 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.701100    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"]
Sep 05 06:46:29.773157 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.773130    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-tls-certs\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.773285 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.773182    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-model-cache\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.773285 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.773233    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-dshm\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.773285 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.773277    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhm5m\" (UniqueName: \"kubernetes.io/projected/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-kube-api-access-vhm5m\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.773413 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.773353    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-tmp-dir\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.773413 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.773391    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-home\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.874389 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.874359    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-model-cache\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.874389 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.874395    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-dshm\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.874594 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.874428    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhm5m\" (UniqueName: \"kubernetes.io/projected/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-kube-api-access-vhm5m\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.874594 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.874475    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-tmp-dir\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.874594 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.874521    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-home\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.874594 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.874574    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-tls-certs\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.874826 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.874805    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-model-cache\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.874966 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.874948    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-tmp-dir\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.875037 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.875015    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-home\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.876564 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.876545    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-dshm\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.876923 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.876906    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-tls-certs\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.883770 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.883750    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhm5m\" (UniqueName: \"kubernetes.io/projected/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-kube-api-access-vhm5m\") pod \"e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") " pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:29.996776 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:29.996750    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:30.272295 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:30.272266    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"]
Sep 05 06:46:30.272988 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:46:30.272958    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod574350d8_5fb0_4972_9fb1_5dcb68ac06d9.slice/crio-110e4c3df2a4bb47f1a27a78c5d57925e7760277aa0d20fb42d4f859783432ac WatchSource:0}: Error finding container 110e4c3df2a4bb47f1a27a78c5d57925e7760277aa0d20fb42d4f859783432ac: Status 404 returned error can't find the container with id 110e4c3df2a4bb47f1a27a78c5d57925e7760277aa0d20fb42d4f859783432ac
Sep 05 06:46:30.641665 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:30.641560    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b" event={"ID":"574350d8-5fb0-4972-9fb1-5dcb68ac06d9","Type":"ContainerStarted","Data":"f0bbd181fe8f63d63b2c37c819c96a6e4984a7d543bc60546c7dd1f126d331ef"}
Sep 05 06:46:30.641665 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:30.641627    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b" event={"ID":"574350d8-5fb0-4972-9fb1-5dcb68ac06d9","Type":"ContainerStarted","Data":"110e4c3df2a4bb47f1a27a78c5d57925e7760277aa0d20fb42d4f859783432ac"}
Sep 05 06:46:30.660150 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:30.660098    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b" podStartSLOduration=1.660085246 podStartE2EDuration="1.660085246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:46:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:46:30.659812418 +0000 UTC m=+1207.250371369" watchObservedRunningTime="2026-09-05 06:46:30.660085246 +0000 UTC m=+1207.250644196"
Sep 05 06:46:39.997059 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:39.997024    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:39.997496 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:39.997356    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:40.009253 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:40.009230    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:40.710907 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:40.710859    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:46:45.090509 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:46:45.090470    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:47:02.602844 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:02.602809    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:47:21.296163 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:21.296084    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:47:24.016713 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:24.016683    2559 scope.go:117] "RemoveContainer" containerID="3e3cccfb3bfa9b0f467877649aa48165cb920b5285a655503ca9bfad46506a85"
Sep 05 06:47:26.103553 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:26.103518    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"]
Sep 05 06:47:38.501263 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:38.501227    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:47:41.122983 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.122943    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b" podUID="574350d8-5fb0-4972-9fb1-5dcb68ac06d9" containerName="main" containerID="cri-o://f0bbd181fe8f63d63b2c37c819c96a6e4984a7d543bc60546c7dd1f126d331ef" gracePeriod=45
Sep 05 06:47:41.360815 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.360792    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:47:41.495195 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.495168    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-model-cache\") pod \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") "
Sep 05 06:47:41.495367 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.495210    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhm5m\" (UniqueName: \"kubernetes.io/projected/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-kube-api-access-vhm5m\") pod \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") "
Sep 05 06:47:41.495367 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.495251    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-dshm\") pod \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") "
Sep 05 06:47:41.495367 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.495284    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-home\") pod \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") "
Sep 05 06:47:41.495367 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.495330    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-tls-certs\") pod \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") "
Sep 05 06:47:41.495367 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.495358    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-tmp-dir\") pod \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\" (UID: \"574350d8-5fb0-4972-9fb1-5dcb68ac06d9\") "
Sep 05 06:47:41.495627 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.495471    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "574350d8-5fb0-4972-9fb1-5dcb68ac06d9" (UID: "574350d8-5fb0-4972-9fb1-5dcb68ac06d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:47:41.495627 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.495534    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-home" (OuterVolumeSpecName: "home") pod "574350d8-5fb0-4972-9fb1-5dcb68ac06d9" (UID: "574350d8-5fb0-4972-9fb1-5dcb68ac06d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:47:41.495762 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.495676    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "574350d8-5fb0-4972-9fb1-5dcb68ac06d9" (UID: "574350d8-5fb0-4972-9fb1-5dcb68ac06d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:47:41.495762 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.495692    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-model-cache\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:47:41.495762 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.495709    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-home\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:47:41.497505 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.497479    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-kube-api-access-vhm5m" (OuterVolumeSpecName: "kube-api-access-vhm5m") pod "574350d8-5fb0-4972-9fb1-5dcb68ac06d9" (UID: "574350d8-5fb0-4972-9fb1-5dcb68ac06d9"). InnerVolumeSpecName "kube-api-access-vhm5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:47:41.497628 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.497507    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "574350d8-5fb0-4972-9fb1-5dcb68ac06d9" (UID: "574350d8-5fb0-4972-9fb1-5dcb68ac06d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:47:41.497628 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.497514    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-dshm" (OuterVolumeSpecName: "dshm") pod "574350d8-5fb0-4972-9fb1-5dcb68ac06d9" (UID: "574350d8-5fb0-4972-9fb1-5dcb68ac06d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:47:41.596288 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.596265    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-tls-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:47:41.596288 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.596285    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-tmp-dir\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:47:41.596435 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.596295    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhm5m\" (UniqueName: \"kubernetes.io/projected/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-kube-api-access-vhm5m\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:47:41.596435 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:41.596305    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/574350d8-5fb0-4972-9fb1-5dcb68ac06d9-dshm\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:47:42.054086 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:42.054050    2559 generic.go:356] "Generic (PLEG): container finished" podID="574350d8-5fb0-4972-9fb1-5dcb68ac06d9" containerID="f0bbd181fe8f63d63b2c37c819c96a6e4984a7d543bc60546c7dd1f126d331ef" exitCode=0
Sep 05 06:47:42.054259 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:42.054090    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"
Sep 05 06:47:42.054259 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:42.054130    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b" event={"ID":"574350d8-5fb0-4972-9fb1-5dcb68ac06d9","Type":"ContainerDied","Data":"f0bbd181fe8f63d63b2c37c819c96a6e4984a7d543bc60546c7dd1f126d331ef"}
Sep 05 06:47:42.054259 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:42.054171    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b" event={"ID":"574350d8-5fb0-4972-9fb1-5dcb68ac06d9","Type":"ContainerDied","Data":"110e4c3df2a4bb47f1a27a78c5d57925e7760277aa0d20fb42d4f859783432ac"}
Sep 05 06:47:42.054259 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:42.054188    2559 scope.go:117] "RemoveContainer" containerID="f0bbd181fe8f63d63b2c37c819c96a6e4984a7d543bc60546c7dd1f126d331ef"
Sep 05 06:47:42.063323 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:42.063306    2559 scope.go:117] "RemoveContainer" containerID="f0bbd181fe8f63d63b2c37c819c96a6e4984a7d543bc60546c7dd1f126d331ef"
Sep 05 06:47:42.063574 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:47:42.063555    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0bbd181fe8f63d63b2c37c819c96a6e4984a7d543bc60546c7dd1f126d331ef\": container with ID starting with f0bbd181fe8f63d63b2c37c819c96a6e4984a7d543bc60546c7dd1f126d331ef not found: ID does not exist" containerID="f0bbd181fe8f63d63b2c37c819c96a6e4984a7d543bc60546c7dd1f126d331ef"
Sep 05 06:47:42.063627 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:42.063582    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0bbd181fe8f63d63b2c37c819c96a6e4984a7d543bc60546c7dd1f126d331ef"} err="failed to get container status \"f0bbd181fe8f63d63b2c37c819c96a6e4984a7d543bc60546c7dd1f126d331ef\": rpc error: code = NotFound desc = could not find container \"f0bbd181fe8f63d63b2c37c819c96a6e4984a7d543bc60546c7dd1f126d331ef\": container with ID starting with f0bbd181fe8f63d63b2c37c819c96a6e4984a7d543bc60546c7dd1f126d331ef not found: ID does not exist"
Sep 05 06:47:42.072795 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:42.072777    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"]
Sep 05 06:47:42.076783 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:42.076763    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-87058ae4/e2e-lifecycle-model-87058ae4-kserve-6bfb55dbb7-tcp4b"]
Sep 05 06:47:43.892364 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:43.892329    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="574350d8-5fb0-4972-9fb1-5dcb68ac06d9" path="/var/lib/kubelet/pods/574350d8-5fb0-4972-9fb1-5dcb68ac06d9/volumes"
Sep 05 06:47:45.316533 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:45.316503    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"]
Sep 05 06:47:46.470906 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:46.470841    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"]
Sep 05 06:47:56.794105 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:47:56.794067    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:48:00.335983 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.335921    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf" podUID="cb944c32-6868-48c2-a746-38f5c1c3ff83" containerName="main" containerID="cri-o://a14bdc6c6351679e0c073812f23a4b1f35b3d4e4a539cc9bdb6fce5acf784d30" gracePeriod=45
Sep 05 06:48:00.582484 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.582456    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:48:00.661010 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.660933    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb944c32-6868-48c2-a746-38f5c1c3ff83-tls-certs\") pod \"cb944c32-6868-48c2-a746-38f5c1c3ff83\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") "
Sep 05 06:48:00.661010 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.660971    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-tmp-dir\") pod \"cb944c32-6868-48c2-a746-38f5c1c3ff83\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") "
Sep 05 06:48:00.661010 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.661007    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knkc4\" (UniqueName: \"kubernetes.io/projected/cb944c32-6868-48c2-a746-38f5c1c3ff83-kube-api-access-knkc4\") pod \"cb944c32-6868-48c2-a746-38f5c1c3ff83\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") "
Sep 05 06:48:00.661263 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.661029    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-model-cache\") pod \"cb944c32-6868-48c2-a746-38f5c1c3ff83\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") "
Sep 05 06:48:00.661263 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.661051    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-home\") pod \"cb944c32-6868-48c2-a746-38f5c1c3ff83\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") "
Sep 05 06:48:00.661263 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.661113    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-dshm\") pod \"cb944c32-6868-48c2-a746-38f5c1c3ff83\" (UID: \"cb944c32-6868-48c2-a746-38f5c1c3ff83\") "
Sep 05 06:48:00.661428 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.661311    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-model-cache" (OuterVolumeSpecName: "model-cache") pod "cb944c32-6868-48c2-a746-38f5c1c3ff83" (UID: "cb944c32-6868-48c2-a746-38f5c1c3ff83"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:48:00.661428 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.661334    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cb944c32-6868-48c2-a746-38f5c1c3ff83" (UID: "cb944c32-6868-48c2-a746-38f5c1c3ff83"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:48:00.661428 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.661371    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-home" (OuterVolumeSpecName: "home") pod "cb944c32-6868-48c2-a746-38f5c1c3ff83" (UID: "cb944c32-6868-48c2-a746-38f5c1c3ff83"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:48:00.663214 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.663185    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb944c32-6868-48c2-a746-38f5c1c3ff83-kube-api-access-knkc4" (OuterVolumeSpecName: "kube-api-access-knkc4") pod "cb944c32-6868-48c2-a746-38f5c1c3ff83" (UID: "cb944c32-6868-48c2-a746-38f5c1c3ff83"). InnerVolumeSpecName "kube-api-access-knkc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:48:00.663328 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.663227    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-dshm" (OuterVolumeSpecName: "dshm") pod "cb944c32-6868-48c2-a746-38f5c1c3ff83" (UID: "cb944c32-6868-48c2-a746-38f5c1c3ff83"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:48:00.663328 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.663233    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb944c32-6868-48c2-a746-38f5c1c3ff83-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cb944c32-6868-48c2-a746-38f5c1c3ff83" (UID: "cb944c32-6868-48c2-a746-38f5c1c3ff83"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:48:00.761929 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.761875    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knkc4\" (UniqueName: \"kubernetes.io/projected/cb944c32-6868-48c2-a746-38f5c1c3ff83-kube-api-access-knkc4\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:00.761929 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.761927    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-model-cache\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:00.762071 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.761937    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-home\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:00.762071 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.761944    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-dshm\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:00.762071 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.761951    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb944c32-6868-48c2-a746-38f5c1c3ff83-tls-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:00.762071 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:00.761958    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb944c32-6868-48c2-a746-38f5c1c3ff83-tmp-dir\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:01.170684 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.170652    2559 generic.go:356] "Generic (PLEG): container finished" podID="cb944c32-6868-48c2-a746-38f5c1c3ff83" containerID="a14bdc6c6351679e0c073812f23a4b1f35b3d4e4a539cc9bdb6fce5acf784d30" exitCode=0
Sep 05 06:48:01.170834 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.170740    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"
Sep 05 06:48:01.170834 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.170732    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf" event={"ID":"cb944c32-6868-48c2-a746-38f5c1c3ff83","Type":"ContainerDied","Data":"a14bdc6c6351679e0c073812f23a4b1f35b3d4e4a539cc9bdb6fce5acf784d30"}
Sep 05 06:48:01.170934 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.170860    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf" event={"ID":"cb944c32-6868-48c2-a746-38f5c1c3ff83","Type":"ContainerDied","Data":"8e5e370ff8547b02eeadc88e55dbd42ff4c945d7d4f397f83a1a1f239794f0cc"}
Sep 05 06:48:01.170934 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.170896    2559 scope.go:117] "RemoveContainer" containerID="a14bdc6c6351679e0c073812f23a4b1f35b3d4e4a539cc9bdb6fce5acf784d30"
Sep 05 06:48:01.180322 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.180303    2559 scope.go:117] "RemoveContainer" containerID="a14bdc6c6351679e0c073812f23a4b1f35b3d4e4a539cc9bdb6fce5acf784d30"
Sep 05 06:48:01.180624 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:48:01.180597    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a14bdc6c6351679e0c073812f23a4b1f35b3d4e4a539cc9bdb6fce5acf784d30\": container with ID starting with a14bdc6c6351679e0c073812f23a4b1f35b3d4e4a539cc9bdb6fce5acf784d30 not found: ID does not exist" containerID="a14bdc6c6351679e0c073812f23a4b1f35b3d4e4a539cc9bdb6fce5acf784d30"
Sep 05 06:48:01.180666 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.180637    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a14bdc6c6351679e0c073812f23a4b1f35b3d4e4a539cc9bdb6fce5acf784d30"} err="failed to get container status \"a14bdc6c6351679e0c073812f23a4b1f35b3d4e4a539cc9bdb6fce5acf784d30\": rpc error: code = NotFound desc = could not find container \"a14bdc6c6351679e0c073812f23a4b1f35b3d4e4a539cc9bdb6fce5acf784d30\": container with ID starting with a14bdc6c6351679e0c073812f23a4b1f35b3d4e4a539cc9bdb6fce5acf784d30 not found: ID does not exist"
Sep 05 06:48:01.197505 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.197482    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"]
Sep 05 06:48:01.202249 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.202228    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/auth-test-model-183ae6-kserve-746f958df9-x72zf"]
Sep 05 06:48:01.490124 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.490089    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc" podUID="0435b1df-61d7-4726-abac-1810595333d9" containerName="main" containerID="cri-o://d5ef3487737a67ecc3d5b6a96b2401d8577d91761e527fd8caadcb2a7113847a" gracePeriod=45
Sep 05 06:48:01.733279 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.733256    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:48:01.766923 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.766844    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-home\") pod \"0435b1df-61d7-4726-abac-1810595333d9\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") "
Sep 05 06:48:01.766923 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.766876    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-tmp-dir\") pod \"0435b1df-61d7-4726-abac-1810595333d9\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") "
Sep 05 06:48:01.767125 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.766933    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9t7k\" (UniqueName: \"kubernetes.io/projected/0435b1df-61d7-4726-abac-1810595333d9-kube-api-access-f9t7k\") pod \"0435b1df-61d7-4726-abac-1810595333d9\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") "
Sep 05 06:48:01.767125 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.767017    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-model-cache\") pod \"0435b1df-61d7-4726-abac-1810595333d9\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") "
Sep 05 06:48:01.767125 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.767044    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0435b1df-61d7-4726-abac-1810595333d9-tls-certs\") pod \"0435b1df-61d7-4726-abac-1810595333d9\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") "
Sep 05 06:48:01.767125 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.767082    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-dshm\") pod \"0435b1df-61d7-4726-abac-1810595333d9\" (UID: \"0435b1df-61d7-4726-abac-1810595333d9\") "
Sep 05 06:48:01.767342 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.767139    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-home" (OuterVolumeSpecName: "home") pod "0435b1df-61d7-4726-abac-1810595333d9" (UID: "0435b1df-61d7-4726-abac-1810595333d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:48:01.767342 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.767176    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0435b1df-61d7-4726-abac-1810595333d9" (UID: "0435b1df-61d7-4726-abac-1810595333d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:48:01.767342 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.767312    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "0435b1df-61d7-4726-abac-1810595333d9" (UID: "0435b1df-61d7-4726-abac-1810595333d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:48:01.767516 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.767437    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-model-cache\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:01.767516 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.767455    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-home\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:01.767516 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.767469    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-tmp-dir\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:01.769300 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.769271    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0435b1df-61d7-4726-abac-1810595333d9-kube-api-access-f9t7k" (OuterVolumeSpecName: "kube-api-access-f9t7k") pod "0435b1df-61d7-4726-abac-1810595333d9" (UID: "0435b1df-61d7-4726-abac-1810595333d9"). InnerVolumeSpecName "kube-api-access-f9t7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:48:01.769300 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.769281    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0435b1df-61d7-4726-abac-1810595333d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0435b1df-61d7-4726-abac-1810595333d9" (UID: "0435b1df-61d7-4726-abac-1810595333d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:48:01.769442 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.769363    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-dshm" (OuterVolumeSpecName: "dshm") pod "0435b1df-61d7-4726-abac-1810595333d9" (UID: "0435b1df-61d7-4726-abac-1810595333d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:48:01.868222 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.868199    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0435b1df-61d7-4726-abac-1810595333d9-tls-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:01.868222 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.868222    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0435b1df-61d7-4726-abac-1810595333d9-dshm\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:01.868365 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.868231    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9t7k\" (UniqueName: \"kubernetes.io/projected/0435b1df-61d7-4726-abac-1810595333d9-kube-api-access-f9t7k\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:01.895709 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:01.895682    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb944c32-6868-48c2-a746-38f5c1c3ff83" path="/var/lib/kubelet/pods/cb944c32-6868-48c2-a746-38f5c1c3ff83/volumes"
Sep 05 06:48:02.177854 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:02.177824    2559 generic.go:356] "Generic (PLEG): container finished" podID="0435b1df-61d7-4726-abac-1810595333d9" containerID="d5ef3487737a67ecc3d5b6a96b2401d8577d91761e527fd8caadcb2a7113847a" exitCode=0
Sep 05 06:48:02.178029 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:02.177862    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"
Sep 05 06:48:02.178029 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:02.177910    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc" event={"ID":"0435b1df-61d7-4726-abac-1810595333d9","Type":"ContainerDied","Data":"d5ef3487737a67ecc3d5b6a96b2401d8577d91761e527fd8caadcb2a7113847a"}
Sep 05 06:48:02.178029 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:02.177957    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc" event={"ID":"0435b1df-61d7-4726-abac-1810595333d9","Type":"ContainerDied","Data":"c4eecf9e2c151720c7995362c885f8c026ce7c9eab58135ba25c28c598415b9c"}
Sep 05 06:48:02.178029 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:02.177979    2559 scope.go:117] "RemoveContainer" containerID="d5ef3487737a67ecc3d5b6a96b2401d8577d91761e527fd8caadcb2a7113847a"
Sep 05 06:48:02.191920 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:02.191814    2559 scope.go:117] "RemoveContainer" containerID="d5ef3487737a67ecc3d5b6a96b2401d8577d91761e527fd8caadcb2a7113847a"
Sep 05 06:48:02.192420 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:48:02.192347    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5ef3487737a67ecc3d5b6a96b2401d8577d91761e527fd8caadcb2a7113847a\": container with ID starting with d5ef3487737a67ecc3d5b6a96b2401d8577d91761e527fd8caadcb2a7113847a not found: ID does not exist" containerID="d5ef3487737a67ecc3d5b6a96b2401d8577d91761e527fd8caadcb2a7113847a"
Sep 05 06:48:02.192420 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:02.192382    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5ef3487737a67ecc3d5b6a96b2401d8577d91761e527fd8caadcb2a7113847a"} err="failed to get container status \"d5ef3487737a67ecc3d5b6a96b2401d8577d91761e527fd8caadcb2a7113847a\": rpc error: code = NotFound desc = could not find container \"d5ef3487737a67ecc3d5b6a96b2401d8577d91761e527fd8caadcb2a7113847a\": container with ID starting with d5ef3487737a67ecc3d5b6a96b2401d8577d91761e527fd8caadcb2a7113847a not found: ID does not exist"
Sep 05 06:48:02.200006 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:02.199983    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"]
Sep 05 06:48:02.203709 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:02.203685    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/auth-test-model-39047b-kserve-6f997ff987-9dktc"]
Sep 05 06:48:03.892494 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:03.892449    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0435b1df-61d7-4726-abac-1810595333d9" path="/var/lib/kubelet/pods/0435b1df-61d7-4726-abac-1810595333d9/volumes"
Sep 05 06:48:18.201211 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.201175    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"]
Sep 05 06:48:18.201610 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.201555    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0435b1df-61d7-4726-abac-1810595333d9" containerName="main"
Sep 05 06:48:18.201610 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.201570    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0435b1df-61d7-4726-abac-1810595333d9" containerName="main"
Sep 05 06:48:18.201610 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.201589    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="574350d8-5fb0-4972-9fb1-5dcb68ac06d9" containerName="main"
Sep 05 06:48:18.201610 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.201597    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="574350d8-5fb0-4972-9fb1-5dcb68ac06d9" containerName="main"
Sep 05 06:48:18.201610 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.201611    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb944c32-6868-48c2-a746-38f5c1c3ff83" containerName="main"
Sep 05 06:48:18.201786 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.201618    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb944c32-6868-48c2-a746-38f5c1c3ff83" containerName="main"
Sep 05 06:48:18.201786 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.201677    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb944c32-6868-48c2-a746-38f5c1c3ff83" containerName="main"
Sep 05 06:48:18.201786 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.201688    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0435b1df-61d7-4726-abac-1810595333d9" containerName="main"
Sep 05 06:48:18.201786 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.201694    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="574350d8-5fb0-4972-9fb1-5dcb68ac06d9" containerName="main"
Sep 05 06:48:18.206195 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.206174    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.209779 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.209751    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-183ae6\"/\"openshift-service-ca.crt\""
Sep 05 06:48:18.209937 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.209815    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-183ae6\"/\"sub-test-model-183ae6-kserve-self-signed-certs\""
Sep 05 06:48:18.209937 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.209829    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-183ae6\"/\"kube-root-ca.crt\""
Sep 05 06:48:18.209937 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.209756    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-183ae6\"/\"default-dockercfg-hnd2s\""
Sep 05 06:48:18.214540 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.214511    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"]
Sep 05 06:48:18.301342 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.301307    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-tls-certs\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.301512 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.301410    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-model-cache\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.301512 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.301444    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-home\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.301512 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.301479    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm8q8\" (UniqueName: \"kubernetes.io/projected/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-kube-api-access-lm8q8\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.301512 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.301499    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-tmp-dir\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.301656 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.301526    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-dshm\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.402727 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.402690    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-tls-certs\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.402946 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.402799    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-model-cache\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.402946 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.402828    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-home\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.402946 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.402856    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm8q8\" (UniqueName: \"kubernetes.io/projected/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-kube-api-access-lm8q8\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.402946 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.402894    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-tmp-dir\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.402946 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.402937    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-dshm\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.403325 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.403299    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-model-cache\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.403325 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.403316    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-tmp-dir\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.403460 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.403363    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-home\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.405231 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.405202    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-dshm\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.405231 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.405219    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-tls-certs\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.411733 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.411707    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm8q8\" (UniqueName: \"kubernetes.io/projected/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-kube-api-access-lm8q8\") pod \"sub-test-model-183ae6-kserve-6747fb9ff-76q2q\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.518834 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.518755    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:18.798661 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:18.798632    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"]
Sep 05 06:48:18.800110 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:48:18.800081    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b47ad64_5c22_4c36_8aee_47bdee25ab4c.slice/crio-0a3ce3e13e943d3b8ac8d0c6187371d7e744007d09f02acae2948fd512c654b6 WatchSource:0}: Error finding container 0a3ce3e13e943d3b8ac8d0c6187371d7e744007d09f02acae2948fd512c654b6: Status 404 returned error can't find the container with id 0a3ce3e13e943d3b8ac8d0c6187371d7e744007d09f02acae2948fd512c654b6
Sep 05 06:48:19.280479 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:19.280445    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q" event={"ID":"1b47ad64-5c22-4c36-8aee-47bdee25ab4c","Type":"ContainerStarted","Data":"196d58c2ceb185436c7fa0ac7f6436b47d579e63fdc63dfba4cb446a4f7e275f"}
Sep 05 06:48:19.280861 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:19.280488    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q" event={"ID":"1b47ad64-5c22-4c36-8aee-47bdee25ab4c","Type":"ContainerStarted","Data":"0a3ce3e13e943d3b8ac8d0c6187371d7e744007d09f02acae2948fd512c654b6"}
Sep 05 06:48:19.304755 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:19.304706    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q" podStartSLOduration=1.304690526 podStartE2EDuration="1.304690526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:48:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:48:19.303775156 +0000 UTC m=+1315.894334100" watchObservedRunningTime="2026-09-05 06:48:19.304690526 +0000 UTC m=+1315.895249481"
Sep 05 06:48:23.795037 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:23.795005    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:48:24.333903 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:24.333801    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"]
Sep 05 06:48:24.334358 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:24.334316    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j" podUID="2b3f89ff-708c-4679-9a49-1e00adc4e4a2" containerName="istio-proxy" containerID="cri-o://768f3071c3a6804dc5bc3e1a495b61fdee7687877daa62ebdd05ab689e461dcb" gracePeriod=30
Sep 05 06:48:28.519854 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:28.519803    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:28.519854 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:28.519867    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:28.533214 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:28.533185    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:29.353411 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.353380    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:48:29.584196 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.584171    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:48:29.710976 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.710945    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-podinfo\") pod \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") "
Sep 05 06:48:29.711172 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.711010    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istiod-ca-cert\") pod \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") "
Sep 05 06:48:29.711172 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.711040    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-workload-socket\") pod \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") "
Sep 05 06:48:29.711172 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.711078    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-envoy\") pod \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") "
Sep 05 06:48:29.711172 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.711109    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-262zt\" (UniqueName: \"kubernetes.io/projected/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-kube-api-access-262zt\") pod \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") "
Sep 05 06:48:29.711172 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.711161    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-data\") pod \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") "
Sep 05 06:48:29.711434 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.711222    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-workload-certs\") pod \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") "
Sep 05 06:48:29.711434 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.711247    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-credential-socket\") pod \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") "
Sep 05 06:48:29.711434 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.711283    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-token\") pod \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\" (UID: \"2b3f89ff-708c-4679-9a49-1e00adc4e4a2\") "
Sep 05 06:48:29.711434 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.711382    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2b3f89ff-708c-4679-9a49-1e00adc4e4a2" (UID: "2b3f89ff-708c-4679-9a49-1e00adc4e4a2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:48:29.711647 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.711452    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-data" (OuterVolumeSpecName: "istio-data") pod "2b3f89ff-708c-4679-9a49-1e00adc4e4a2" (UID: "2b3f89ff-708c-4679-9a49-1e00adc4e4a2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:48:29.711647 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.711537    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2b3f89ff-708c-4679-9a49-1e00adc4e4a2" (UID: "2b3f89ff-708c-4679-9a49-1e00adc4e4a2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:48:29.711782 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.711712    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2b3f89ff-708c-4679-9a49-1e00adc4e4a2" (UID: "2b3f89ff-708c-4679-9a49-1e00adc4e4a2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:48:29.711782 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.711726    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-data\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:29.711782 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.711748    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istiod-ca-cert\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:29.711782 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.711764    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-workload-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:29.711782 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.711725    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2b3f89ff-708c-4679-9a49-1e00adc4e4a2" (UID: "2b3f89ff-708c-4679-9a49-1e00adc4e4a2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:48:29.713269 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.713237    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2b3f89ff-708c-4679-9a49-1e00adc4e4a2" (UID: "2b3f89ff-708c-4679-9a49-1e00adc4e4a2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:48:29.713386 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.713289    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2b3f89ff-708c-4679-9a49-1e00adc4e4a2" (UID: "2b3f89ff-708c-4679-9a49-1e00adc4e4a2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 06:48:29.713441 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.713380    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-kube-api-access-262zt" (OuterVolumeSpecName: "kube-api-access-262zt") pod "2b3f89ff-708c-4679-9a49-1e00adc4e4a2" (UID: "2b3f89ff-708c-4679-9a49-1e00adc4e4a2"). InnerVolumeSpecName "kube-api-access-262zt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:48:29.713441 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.713398    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-token" (OuterVolumeSpecName: "istio-token") pod "2b3f89ff-708c-4679-9a49-1e00adc4e4a2" (UID: "2b3f89ff-708c-4679-9a49-1e00adc4e4a2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:48:29.813123 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.813092    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-workload-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:29.813123 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.813117    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-credential-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:29.813123 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.813127    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-token\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:29.813330 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.813135    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-podinfo\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:29.813330 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.813143    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-istio-envoy\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:29.813330 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:29.813151    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-262zt\" (UniqueName: \"kubernetes.io/projected/2b3f89ff-708c-4679-9a49-1e00adc4e4a2-kube-api-access-262zt\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:48:30.348249 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:30.348217    2559 generic.go:356] "Generic (PLEG): container finished" podID="2b3f89ff-708c-4679-9a49-1e00adc4e4a2" containerID="768f3071c3a6804dc5bc3e1a495b61fdee7687877daa62ebdd05ab689e461dcb" exitCode=0
Sep 05 06:48:30.348432 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:30.348264    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"
Sep 05 06:48:30.348432 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:30.348300    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j" event={"ID":"2b3f89ff-708c-4679-9a49-1e00adc4e4a2","Type":"ContainerDied","Data":"768f3071c3a6804dc5bc3e1a495b61fdee7687877daa62ebdd05ab689e461dcb"}
Sep 05 06:48:30.348432 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:30.348351    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j" event={"ID":"2b3f89ff-708c-4679-9a49-1e00adc4e4a2","Type":"ContainerDied","Data":"e7896fe917f01f3ec5548b73800e15340e52d263f8a7fa9eac0adbc032cc0a32"}
Sep 05 06:48:30.348432 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:30.348373    2559 scope.go:117] "RemoveContainer" containerID="768f3071c3a6804dc5bc3e1a495b61fdee7687877daa62ebdd05ab689e461dcb"
Sep 05 06:48:30.357654 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:30.357635    2559 scope.go:117] "RemoveContainer" containerID="4d1f75ac6bea35948e3a28e9993e1fe7e58cd45bbce46ace4e3564e97432c76d"
Sep 05 06:48:30.366098 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:30.366080    2559 scope.go:117] "RemoveContainer" containerID="768f3071c3a6804dc5bc3e1a495b61fdee7687877daa62ebdd05ab689e461dcb"
Sep 05 06:48:30.366376 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:48:30.366356    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"768f3071c3a6804dc5bc3e1a495b61fdee7687877daa62ebdd05ab689e461dcb\": container with ID starting with 768f3071c3a6804dc5bc3e1a495b61fdee7687877daa62ebdd05ab689e461dcb not found: ID does not exist" containerID="768f3071c3a6804dc5bc3e1a495b61fdee7687877daa62ebdd05ab689e461dcb"
Sep 05 06:48:30.366425 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:30.366384    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"768f3071c3a6804dc5bc3e1a495b61fdee7687877daa62ebdd05ab689e461dcb"} err="failed to get container status \"768f3071c3a6804dc5bc3e1a495b61fdee7687877daa62ebdd05ab689e461dcb\": rpc error: code = NotFound desc = could not find container \"768f3071c3a6804dc5bc3e1a495b61fdee7687877daa62ebdd05ab689e461dcb\": container with ID starting with 768f3071c3a6804dc5bc3e1a495b61fdee7687877daa62ebdd05ab689e461dcb not found: ID does not exist"
Sep 05 06:48:30.366425 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:30.366403    2559 scope.go:117] "RemoveContainer" containerID="4d1f75ac6bea35948e3a28e9993e1fe7e58cd45bbce46ace4e3564e97432c76d"
Sep 05 06:48:30.366684 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:48:30.366664    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d1f75ac6bea35948e3a28e9993e1fe7e58cd45bbce46ace4e3564e97432c76d\": container with ID starting with 4d1f75ac6bea35948e3a28e9993e1fe7e58cd45bbce46ace4e3564e97432c76d not found: ID does not exist" containerID="4d1f75ac6bea35948e3a28e9993e1fe7e58cd45bbce46ace4e3564e97432c76d"
Sep 05 06:48:30.366724 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:30.366691    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d1f75ac6bea35948e3a28e9993e1fe7e58cd45bbce46ace4e3564e97432c76d"} err="failed to get container status \"4d1f75ac6bea35948e3a28e9993e1fe7e58cd45bbce46ace4e3564e97432c76d\": rpc error: code = NotFound desc = could not find container \"4d1f75ac6bea35948e3a28e9993e1fe7e58cd45bbce46ace4e3564e97432c76d\": container with ID starting with 4d1f75ac6bea35948e3a28e9993e1fe7e58cd45bbce46ace4e3564e97432c76d not found: ID does not exist"
Sep 05 06:48:30.371023 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:30.370997    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"]
Sep 05 06:48:30.375472 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:30.375450    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-87058ae4-openshift-default-dbdf69cc6-n9g6j"]
Sep 05 06:48:31.891810 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:31.891777    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b3f89ff-708c-4679-9a49-1e00adc4e4a2" path="/var/lib/kubelet/pods/2b3f89ff-708c-4679-9a49-1e00adc4e4a2/volumes"
Sep 05 06:48:38.683348 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.683314    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"]
Sep 05 06:48:38.683924 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.683873    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b3f89ff-708c-4679-9a49-1e00adc4e4a2" containerName="istio-proxy"
Sep 05 06:48:38.683924 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.683911    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b3f89ff-708c-4679-9a49-1e00adc4e4a2" containerName="istio-proxy"
Sep 05 06:48:38.684056 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.683937    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b3f89ff-708c-4679-9a49-1e00adc4e4a2" containerName="istio-proxy"
Sep 05 06:48:38.684056 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.683946    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b3f89ff-708c-4679-9a49-1e00adc4e4a2" containerName="istio-proxy"
Sep 05 06:48:38.684161 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.684064    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b3f89ff-708c-4679-9a49-1e00adc4e4a2" containerName="istio-proxy"
Sep 05 06:48:38.684161 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.684078    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b3f89ff-708c-4679-9a49-1e00adc4e4a2" containerName="istio-proxy"
Sep 05 06:48:38.688474 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.688455    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.692268 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.692242    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-39047b\"/\"kube-root-ca.crt\""
Sep 05 06:48:38.692376 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.692301    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-39047b\"/\"default-dockercfg-cskxb\""
Sep 05 06:48:38.692376 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.692331    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-39047b\"/\"openshift-service-ca.crt\""
Sep 05 06:48:38.692506 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.692329    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-39047b\"/\"sub-test-model-39047b-kserve-self-signed-certs\""
Sep 05 06:48:38.697613 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.697588    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"]
Sep 05 06:48:38.791234 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.791209    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-dshm\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.791383 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.791241    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-tmp-dir\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.791383 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.791276    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-home\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.791383 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.791311    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-model-cache\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.791383 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.791345    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sgsn\" (UniqueName: \"kubernetes.io/projected/86456493-9bf4-4263-8150-950154705a85-kube-api-access-4sgsn\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.791525 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.791388    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86456493-9bf4-4263-8150-950154705a85-tls-certs\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.891972 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.891946    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-dshm\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.892135 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.891978    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-tmp-dir\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.892135 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.892009    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-home\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.892135 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.892031    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-model-cache\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.892135 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.892053    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sgsn\" (UniqueName: \"kubernetes.io/projected/86456493-9bf4-4263-8150-950154705a85-kube-api-access-4sgsn\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.892135 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.892103    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86456493-9bf4-4263-8150-950154705a85-tls-certs\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.892441 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.892408    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-home\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.892502 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.892446    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-model-cache\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.892502 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.892457    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-tmp-dir\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.894381 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.894356    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-dshm\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.894632 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.894615    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86456493-9bf4-4263-8150-950154705a85-tls-certs\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.902061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.902038    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sgsn\" (UniqueName: \"kubernetes.io/projected/86456493-9bf4-4263-8150-950154705a85-kube-api-access-4sgsn\") pod \"sub-test-model-39047b-kserve-67564f488f-2frpp\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") " pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:38.998789 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:38.998760    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:39.274588 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:39.274559    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"]
Sep 05 06:48:39.276126 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:48:39.276094    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86456493_9bf4_4263_8150_950154705a85.slice/crio-44501be26c80b0ee097c83dd9784adee4a8a750e4e8693e4bf7fd1c462c06ea2 WatchSource:0}: Error finding container 44501be26c80b0ee097c83dd9784adee4a8a750e4e8693e4bf7fd1c462c06ea2: Status 404 returned error can't find the container with id 44501be26c80b0ee097c83dd9784adee4a8a750e4e8693e4bf7fd1c462c06ea2
Sep 05 06:48:39.401876 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:39.401839    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp" event={"ID":"86456493-9bf4-4263-8150-950154705a85","Type":"ContainerStarted","Data":"09bc8cc026528e5bbc51615d2d3df2cb0ac3c62cb2afa446b3d551bcfd5d6c9e"}
Sep 05 06:48:39.402070 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:39.401916    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp" event={"ID":"86456493-9bf4-4263-8150-950154705a85","Type":"ContainerStarted","Data":"44501be26c80b0ee097c83dd9784adee4a8a750e4e8693e4bf7fd1c462c06ea2"}
Sep 05 06:48:39.424289 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:39.424241    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp" podStartSLOduration=1.424225647 podStartE2EDuration="1.424225647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:48:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:48:39.421834969 +0000 UTC m=+1336.012393922" watchObservedRunningTime="2026-09-05 06:48:39.424225647 +0000 UTC m=+1336.014784599"
Sep 05 06:48:47.590212 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:47.590126    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:48:48.999476 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:48.999426    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:48.999948 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:48.999498    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:49.012156 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:49.012130    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:48:49.473983 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:48:49.473953    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:49:16.387766 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:16.387731    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:49:20.438348 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.438306    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"]
Sep 05 06:49:20.441818 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.441797    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.444204 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.444182    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-96d352-openshift-default-dockercfg-mclvn\""
Sep 05 06:49:20.455096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.455069    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"]
Sep 05 06:49:20.561743 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.561708    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-data\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.561743 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.561746    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p76hn\" (UniqueName: \"kubernetes.io/projected/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-kube-api-access-p76hn\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.561965 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.561794    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-envoy\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.561965 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.561828    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-token\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.561965 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.561919    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-credential-socket\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.561965 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.561956    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-workload-socket\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.562100 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.561972    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-podinfo\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.562100 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.561992    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istiod-ca-cert\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.562167 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.562090    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-workload-certs\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.663288 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.663261    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-workload-certs\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.663475 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.663306    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-data\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.663475 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.663327    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p76hn\" (UniqueName: \"kubernetes.io/projected/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-kube-api-access-p76hn\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.663475 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.663369    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-envoy\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.663475 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.663395    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-token\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.663475 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.663434    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-credential-socket\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.663475 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.663462    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-workload-socket\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.663808 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.663486    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-podinfo\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.663808 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.663517    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istiod-ca-cert\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.663808 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.663721    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-workload-certs\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.664009 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.663824    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-credential-socket\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.664009 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.663871    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-workload-socket\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.664115 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.664062    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-data\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.664258 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.664240    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istiod-ca-cert\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.665825 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.665806    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-envoy\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.666052 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.666034    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-podinfo\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.673248 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.673223    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-token\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.673435 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.673410    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p76hn\" (UniqueName: \"kubernetes.io/projected/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-kube-api-access-p76hn\") pod \"e2e-api-a-96d352-openshift-default-66c4489854-p8g24\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") " pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:20.753342 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:20.753271    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:21.031441 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:21.031411    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"]
Sep 05 06:49:21.033549 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:49:21.033518    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e8e7b38_bf46_47f7_b65b_4ce31ad05581.slice/crio-57360353c1b5a8c571956692673a830e117213801df451605a670ca1e0fce98b WatchSource:0}: Error finding container 57360353c1b5a8c571956692673a830e117213801df451605a670ca1e0fce98b: Status 404 returned error can't find the container with id 57360353c1b5a8c571956692673a830e117213801df451605a670ca1e0fce98b
Sep 05 06:49:21.036299 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:21.036260    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:49:21.036412 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:21.036339    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:49:21.036412 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:21.036377    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:49:21.660641 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:21.660604    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24" event={"ID":"6e8e7b38-bf46-47f7-b65b-4ce31ad05581","Type":"ContainerStarted","Data":"b3ee28a0c499c273ce015ed704938095e39447711fe922e9e2a3d370fbf6dbfc"}
Sep 05 06:49:21.661056 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:21.660688    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24" event={"ID":"6e8e7b38-bf46-47f7-b65b-4ce31ad05581","Type":"ContainerStarted","Data":"57360353c1b5a8c571956692673a830e117213801df451605a670ca1e0fce98b"}
Sep 05 06:49:21.682189 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:21.682139    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24" podStartSLOduration=1.682124955 podStartE2EDuration="1.682124955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:49:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:49:21.679205099 +0000 UTC m=+1378.269764051" watchObservedRunningTime="2026-09-05 06:49:21.682124955 +0000 UTC m=+1378.272683906"
Sep 05 06:49:21.753900 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:21.753851    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:22.759576 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:22.759545    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:23.672873 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:23.672842    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:23.673830 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:23.673807    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:34.305311 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:34.305275    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:49:37.754349 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:37.754313    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7741-14422b"]
Sep 05 06:49:37.759407 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:37.759377    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7741-14422b"
Sep 05 06:49:37.761761 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:37.761739    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-wwpm9\""
Sep 05 06:49:37.766366 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:37.766306    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7741-14422b"]
Sep 05 06:49:37.920364 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:37.920332    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx427\" (UniqueName: \"kubernetes.io/projected/22f9147a-8dac-4211-b2af-1225dbbfd2ae-kube-api-access-tx427\") pod \"mt-curl-7741-14422b\" (UID: \"22f9147a-8dac-4211-b2af-1225dbbfd2ae\") " pod="opendatahub/mt-curl-7741-14422b"
Sep 05 06:49:38.021769 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:38.021683    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx427\" (UniqueName: \"kubernetes.io/projected/22f9147a-8dac-4211-b2af-1225dbbfd2ae-kube-api-access-tx427\") pod \"mt-curl-7741-14422b\" (UID: \"22f9147a-8dac-4211-b2af-1225dbbfd2ae\") " pod="opendatahub/mt-curl-7741-14422b"
Sep 05 06:49:38.032042 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:38.032013    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx427\" (UniqueName: \"kubernetes.io/projected/22f9147a-8dac-4211-b2af-1225dbbfd2ae-kube-api-access-tx427\") pod \"mt-curl-7741-14422b\" (UID: \"22f9147a-8dac-4211-b2af-1225dbbfd2ae\") " pod="opendatahub/mt-curl-7741-14422b"
Sep 05 06:49:38.071726 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:38.071694    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7741-14422b"
Sep 05 06:49:38.364764 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:38.364731    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7741-14422b"]
Sep 05 06:49:38.365332 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:49:38.365297    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22f9147a_8dac_4211_b2af_1225dbbfd2ae.slice/crio-decd1e75d2f5bf6c8eee575a2542229b7bf997b144fcdf6e3437abb346d287b9 WatchSource:0}: Error finding container decd1e75d2f5bf6c8eee575a2542229b7bf997b144fcdf6e3437abb346d287b9: Status 404 returned error can't find the container with id decd1e75d2f5bf6c8eee575a2542229b7bf997b144fcdf6e3437abb346d287b9
Sep 05 06:49:38.772319 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:38.772279    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7741-14422b" event={"ID":"22f9147a-8dac-4211-b2af-1225dbbfd2ae","Type":"ContainerStarted","Data":"decd1e75d2f5bf6c8eee575a2542229b7bf997b144fcdf6e3437abb346d287b9"}
Sep 05 06:49:39.779872 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:39.779829    2559 generic.go:356] "Generic (PLEG): container finished" podID="22f9147a-8dac-4211-b2af-1225dbbfd2ae" containerID="1e912877917b41564271d6ccab90f722e9509e948bdd8c246dfb0c18fc56b6b1" exitCode=0
Sep 05 06:49:39.780335 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:39.779918    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7741-14422b" event={"ID":"22f9147a-8dac-4211-b2af-1225dbbfd2ae","Type":"ContainerDied","Data":"1e912877917b41564271d6ccab90f722e9509e948bdd8c246dfb0c18fc56b6b1"}
Sep 05 06:49:40.933981 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:40.933955    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7741-14422b"
Sep 05 06:49:40.963108 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:40.963082    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tx427\" (UniqueName: \"kubernetes.io/projected/22f9147a-8dac-4211-b2af-1225dbbfd2ae-kube-api-access-tx427\") pod \"22f9147a-8dac-4211-b2af-1225dbbfd2ae\" (UID: \"22f9147a-8dac-4211-b2af-1225dbbfd2ae\") "
Sep 05 06:49:40.965314 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:40.965290    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22f9147a-8dac-4211-b2af-1225dbbfd2ae-kube-api-access-tx427" (OuterVolumeSpecName: "kube-api-access-tx427") pod "22f9147a-8dac-4211-b2af-1225dbbfd2ae" (UID: "22f9147a-8dac-4211-b2af-1225dbbfd2ae"). InnerVolumeSpecName "kube-api-access-tx427". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:49:40.968810 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:40.968786    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7741-14422b_22f9147a-8dac-4211-b2af-1225dbbfd2ae/mt-curl-7741-14422b/0.log"
Sep 05 06:49:40.996425 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:40.996401    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7741-14422b"]
Sep 05 06:49:41.000392 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:41.000361    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7741-14422b"]
Sep 05 06:49:41.064558 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:41.064496    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tx427\" (UniqueName: \"kubernetes.io/projected/22f9147a-8dac-4211-b2af-1225dbbfd2ae-kube-api-access-tx427\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:49:41.119396 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:41.119366    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7741-0b458f"]
Sep 05 06:49:41.119793 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:41.119775    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22f9147a-8dac-4211-b2af-1225dbbfd2ae" containerName="mt-curl-7741-14422b"
Sep 05 06:49:41.119870 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:41.119796    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="22f9147a-8dac-4211-b2af-1225dbbfd2ae" containerName="mt-curl-7741-14422b"
Sep 05 06:49:41.119953 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:41.119931    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="22f9147a-8dac-4211-b2af-1225dbbfd2ae" containerName="mt-curl-7741-14422b"
Sep 05 06:49:41.123007 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:41.122989    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7741-0b458f"
Sep 05 06:49:41.133979 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:41.133947    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7741-0b458f"]
Sep 05 06:49:41.164953 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:41.164924    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5lm4\" (UniqueName: \"kubernetes.io/projected/52d433df-4b84-411e-b74a-48130c50ac7a-kube-api-access-b5lm4\") pod \"mt-curl-7741-0b458f\" (UID: \"52d433df-4b84-411e-b74a-48130c50ac7a\") " pod="opendatahub/mt-curl-7741-0b458f"
Sep 05 06:49:41.266238 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:41.266211    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5lm4\" (UniqueName: \"kubernetes.io/projected/52d433df-4b84-411e-b74a-48130c50ac7a-kube-api-access-b5lm4\") pod \"mt-curl-7741-0b458f\" (UID: \"52d433df-4b84-411e-b74a-48130c50ac7a\") " pod="opendatahub/mt-curl-7741-0b458f"
Sep 05 06:49:41.278021 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:41.277997    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5lm4\" (UniqueName: \"kubernetes.io/projected/52d433df-4b84-411e-b74a-48130c50ac7a-kube-api-access-b5lm4\") pod \"mt-curl-7741-0b458f\" (UID: \"52d433df-4b84-411e-b74a-48130c50ac7a\") " pod="opendatahub/mt-curl-7741-0b458f"
Sep 05 06:49:41.435624 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:41.435564    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7741-0b458f"
Sep 05 06:49:41.709548 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:41.709517    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7741-0b458f"]
Sep 05 06:49:41.710130 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:49:41.710100    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52d433df_4b84_411e_b74a_48130c50ac7a.slice/crio-7a1e1bd885fd919f1bc6bcc749c12908559ce811740284726f5a2855a99897c7 WatchSource:0}: Error finding container 7a1e1bd885fd919f1bc6bcc749c12908559ce811740284726f5a2855a99897c7: Status 404 returned error can't find the container with id 7a1e1bd885fd919f1bc6bcc749c12908559ce811740284726f5a2855a99897c7
Sep 05 06:49:41.794422 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:41.794391    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7741-14422b"
Sep 05 06:49:41.794422 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:41.794418    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="decd1e75d2f5bf6c8eee575a2542229b7bf997b144fcdf6e3437abb346d287b9"
Sep 05 06:49:41.795835 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:41.795796    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7741-0b458f" event={"ID":"52d433df-4b84-411e-b74a-48130c50ac7a","Type":"ContainerStarted","Data":"7a1e1bd885fd919f1bc6bcc749c12908559ce811740284726f5a2855a99897c7"}
Sep 05 06:49:41.806009 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:49:41.805976    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7741-14422b\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="22f9147a-8dac-4211-b2af-1225dbbfd2ae" pod="opendatahub/mt-curl-7741-14422b"
Sep 05 06:49:41.892923 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:41.892862    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22f9147a-8dac-4211-b2af-1225dbbfd2ae" path="/var/lib/kubelet/pods/22f9147a-8dac-4211-b2af-1225dbbfd2ae/volumes"
Sep 05 06:49:42.810280 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:42.810242    2559 generic.go:356] "Generic (PLEG): container finished" podID="52d433df-4b84-411e-b74a-48130c50ac7a" containerID="d282909a774cfdbcd2ad4880b0e82fe84ab40fe8217185bdd863086bee1163f5" exitCode=0
Sep 05 06:49:42.810654 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:42.810323    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7741-0b458f" event={"ID":"52d433df-4b84-411e-b74a-48130c50ac7a","Type":"ContainerDied","Data":"d282909a774cfdbcd2ad4880b0e82fe84ab40fe8217185bdd863086bee1163f5"}
Sep 05 06:49:43.968452 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:43.968424    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7741-0b458f"
Sep 05 06:49:43.987200 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:43.987180    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5lm4\" (UniqueName: \"kubernetes.io/projected/52d433df-4b84-411e-b74a-48130c50ac7a-kube-api-access-b5lm4\") pod \"52d433df-4b84-411e-b74a-48130c50ac7a\" (UID: \"52d433df-4b84-411e-b74a-48130c50ac7a\") "
Sep 05 06:49:43.989595 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:43.989563    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52d433df-4b84-411e-b74a-48130c50ac7a-kube-api-access-b5lm4" (OuterVolumeSpecName: "kube-api-access-b5lm4") pod "52d433df-4b84-411e-b74a-48130c50ac7a" (UID: "52d433df-4b84-411e-b74a-48130c50ac7a"). InnerVolumeSpecName "kube-api-access-b5lm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:49:43.991989 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:43.991970    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7741-0b458f_52d433df-4b84-411e-b74a-48130c50ac7a/mt-curl-7741-0b458f/0.log"
Sep 05 06:49:44.017396 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:44.017360    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7741-0b458f"]
Sep 05 06:49:44.021757 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:44.021732    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7741-0b458f"]
Sep 05 06:49:44.088158 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:44.088092    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5lm4\" (UniqueName: \"kubernetes.io/projected/52d433df-4b84-411e-b74a-48130c50ac7a-kube-api-access-b5lm4\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:49:44.826034 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:44.826003    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7741-0b458f"
Sep 05 06:49:44.826232 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:44.826056    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a1e1bd885fd919f1bc6bcc749c12908559ce811740284726f5a2855a99897c7"
Sep 05 06:49:44.828360 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:44.828323    2559 generic.go:356] "Generic (PLEG): container finished" podID="6e8e7b38-bf46-47f7-b65b-4ce31ad05581" containerID="b3ee28a0c499c273ce015ed704938095e39447711fe922e9e2a3d370fbf6dbfc" exitCode=0
Sep 05 06:49:44.828517 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:44.828382    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24" event={"ID":"6e8e7b38-bf46-47f7-b65b-4ce31ad05581","Type":"ContainerDied","Data":"b3ee28a0c499c273ce015ed704938095e39447711fe922e9e2a3d370fbf6dbfc"}
Sep 05 06:49:44.828865 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:44.828847    2559 scope.go:117] "RemoveContainer" containerID="b3ee28a0c499c273ce015ed704938095e39447711fe922e9e2a3d370fbf6dbfc"
Sep 05 06:49:44.830152 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:44.830120    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:49:44.830359 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:44.830190    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:49:44.830359 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:44.830224    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:49:44.851856 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:49:44.851827    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7741-0b458f\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="52d433df-4b84-411e-b74a-48130c50ac7a" pod="opendatahub/mt-curl-7741-0b458f"
Sep 05 06:49:45.844121 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:45.844077    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24" event={"ID":"6e8e7b38-bf46-47f7-b65b-4ce31ad05581","Type":"ContainerStarted","Data":"b9e8ba3ac842a4a3e90c27925912d8bd99c280637fed3d31b303e7a55b06b5e9"}
Sep 05 06:49:45.846573 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:49:45.846537    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7741-0b458f\" is forbidden: User \"system:node:ip-10-0-130-170.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-170.ec2.internal' and this object" podUID="52d433df-4b84-411e-b74a-48130c50ac7a" pod="opendatahub/mt-curl-7741-0b458f"
Sep 05 06:49:45.892033 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:45.892005    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52d433df-4b84-411e-b74a-48130c50ac7a" path="/var/lib/kubelet/pods/52d433df-4b84-411e-b74a-48130c50ac7a/volumes"
Sep 05 06:49:46.284815 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:46.284784    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"]
Sep 05 06:49:46.754093 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:46.754054    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:46.754290 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:46.754264    2559 patch_prober.go:28] interesting pod/e2e-api-a-96d352-openshift-default-66c4489854-p8g24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 05 06:49:46.754378 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:46.754336    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24" podUID="6e8e7b38-bf46-47f7-b65b-4ce31ad05581" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 05 06:49:47.595006 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:47.594972    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"]
Sep 05 06:49:47.754702 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:47.754668    2559 patch_prober.go:28] interesting pod/e2e-api-a-96d352-openshift-default-66c4489854-p8g24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 05 06:49:47.754861 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:47.754726    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24" podUID="6e8e7b38-bf46-47f7-b65b-4ce31ad05581" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 05 06:49:48.754310 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:48.754271    2559 patch_prober.go:28] interesting pod/e2e-api-a-96d352-openshift-default-66c4489854-p8g24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 05 06:49:48.754686 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:48.754337    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24" podUID="6e8e7b38-bf46-47f7-b65b-4ce31ad05581" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 05 06:49:49.303774 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.303741    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"]
Sep 05 06:49:49.304214 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.304199    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52d433df-4b84-411e-b74a-48130c50ac7a" containerName="mt-curl-7741-0b458f"
Sep 05 06:49:49.304291 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.304217    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="52d433df-4b84-411e-b74a-48130c50ac7a" containerName="mt-curl-7741-0b458f"
Sep 05 06:49:49.304336 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.304324    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="52d433df-4b84-411e-b74a-48130c50ac7a" containerName="mt-curl-7741-0b458f"
Sep 05 06:49:49.307782 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.307759    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.310423 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.310407    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a9301f-openshift-default-dockercfg-g5t68\""
Sep 05 06:49:49.318961 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.318928    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"]
Sep 05 06:49:49.356018 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.355987    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-workload-socket\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.356273 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.356248    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-workload-certs\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.356430 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.356413    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-data\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.356537 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.356524    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-token\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.356634 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.356621    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istiod-ca-cert\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.356735 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.356723    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-podinfo\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.356835 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.356820    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rxzh\" (UniqueName: \"kubernetes.io/projected/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-kube-api-access-7rxzh\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.357511 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.357213    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-credential-socket\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.357511 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.357274    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-envoy\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.458348 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.458307    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-workload-socket\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.458511 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.458405    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-workload-certs\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.458877 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.458854    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-workload-certs\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.458945 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.458927    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-workload-socket\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.459044 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.459026    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-data\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.459336 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.459318    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-token\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.459425 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.459410    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-data\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.459462 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.459451    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istiod-ca-cert\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.459503 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.459487    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-podinfo\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.459628 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.459612    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rxzh\" (UniqueName: \"kubernetes.io/projected/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-kube-api-access-7rxzh\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.459800 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.459784    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-credential-socket\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.460032 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.460007    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-envoy\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.460383 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.460360    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-credential-socket\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.460726 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.460702    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istiod-ca-cert\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.461902 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.461867    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-podinfo\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.462537 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.462519    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-envoy\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.467460 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.467420    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-token\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.468780 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.468760    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rxzh\" (UniqueName: \"kubernetes.io/projected/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-kube-api-access-7rxzh\") pod \"e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") " pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.618536 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.618330    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:49.754299 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.754266    2559 patch_prober.go:28] interesting pod/e2e-api-a-96d352-openshift-default-66c4489854-p8g24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 05 06:49:49.754465 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.754322    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24" podUID="6e8e7b38-bf46-47f7-b65b-4ce31ad05581" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 05 06:49:49.922139 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.922107    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"]
Sep 05 06:49:49.938620 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.938570    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:49:49.938761 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.938673    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:49:49.938761 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:49.938722    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:49:50.754433 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:50.754393    2559 patch_prober.go:28] interesting pod/e2e-api-a-96d352-openshift-default-66c4489854-p8g24 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Sep 05 06:49:50.754641 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:50.754456    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24" podUID="6e8e7b38-bf46-47f7-b65b-4ce31ad05581" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Sep 05 06:49:50.880114 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:50.880076    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs" event={"ID":"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a","Type":"ContainerStarted","Data":"3da03b1ffcb46eb3079475bffd81e4a04114722cca529ce6b81fbfef7256083a"}
Sep 05 06:49:50.880458 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:50.880177    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs" event={"ID":"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a","Type":"ContainerStarted","Data":"b0ae8a2153418a13caba951425c712eeca0106512c9d8c4fde650952b7b6d2a1"}
Sep 05 06:49:50.902835 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:50.902775    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs" podStartSLOduration=1.9027574280000001 podStartE2EDuration="1.902757428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:49:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:49:50.90056134 +0000 UTC m=+1407.491120340" watchObservedRunningTime="2026-09-05 06:49:50.902757428 +0000 UTC m=+1407.493316450"
Sep 05 06:49:51.619583 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:51.619536    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:51.625820 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:51.625789    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:51.758503 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:51.758471    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:51.758896 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:51.758845    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:51.759639 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:51.759621    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:49:51.892290 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:51.892215    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:51.892466 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:51.892293    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:49:53.420185 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:49:53.420145    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:50:01.304399 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.304331    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q" podUID="1b47ad64-5c22-4c36-8aee-47bdee25ab4c" containerName="main" containerID="cri-o://196d58c2ceb185436c7fa0ac7f6436b47d579e63fdc63dfba4cb446a4f7e275f" gracePeriod=45
Sep 05 06:50:01.548828 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.548802    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:50:01.677958 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.677858    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-tmp-dir\") pod \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") "
Sep 05 06:50:01.677958 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.677942    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-model-cache\") pod \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") "
Sep 05 06:50:01.678199 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.677979    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-home\") pod \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") "
Sep 05 06:50:01.678199 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.678096    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lm8q8\" (UniqueName: \"kubernetes.io/projected/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-kube-api-access-lm8q8\") pod \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") "
Sep 05 06:50:01.678199 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.678126    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-dshm\") pod \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") "
Sep 05 06:50:01.678199 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.678134    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1b47ad64-5c22-4c36-8aee-47bdee25ab4c" (UID: "1b47ad64-5c22-4c36-8aee-47bdee25ab4c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:50:01.678199 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.678156    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-tls-certs\") pod \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\" (UID: \"1b47ad64-5c22-4c36-8aee-47bdee25ab4c\") "
Sep 05 06:50:01.678443 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.678228    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-home" (OuterVolumeSpecName: "home") pod "1b47ad64-5c22-4c36-8aee-47bdee25ab4c" (UID: "1b47ad64-5c22-4c36-8aee-47bdee25ab4c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:50:01.678443 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.678243    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-model-cache" (OuterVolumeSpecName: "model-cache") pod "1b47ad64-5c22-4c36-8aee-47bdee25ab4c" (UID: "1b47ad64-5c22-4c36-8aee-47bdee25ab4c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:50:01.678552 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.678497    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-tmp-dir\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:01.678552 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.678516    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-model-cache\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:01.678552 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.678531    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-home\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:01.680286 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.680250    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1b47ad64-5c22-4c36-8aee-47bdee25ab4c" (UID: "1b47ad64-5c22-4c36-8aee-47bdee25ab4c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:50:01.680286 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.680274    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-kube-api-access-lm8q8" (OuterVolumeSpecName: "kube-api-access-lm8q8") pod "1b47ad64-5c22-4c36-8aee-47bdee25ab4c" (UID: "1b47ad64-5c22-4c36-8aee-47bdee25ab4c"). InnerVolumeSpecName "kube-api-access-lm8q8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:50:01.680446 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.680257    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-dshm" (OuterVolumeSpecName: "dshm") pod "1b47ad64-5c22-4c36-8aee-47bdee25ab4c" (UID: "1b47ad64-5c22-4c36-8aee-47bdee25ab4c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:50:01.779764 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.779719    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lm8q8\" (UniqueName: \"kubernetes.io/projected/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-kube-api-access-lm8q8\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:01.779764 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.779753    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-dshm\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:01.779764 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.779763    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b47ad64-5c22-4c36-8aee-47bdee25ab4c-tls-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:01.946957 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.946914    2559 generic.go:356] "Generic (PLEG): container finished" podID="1b47ad64-5c22-4c36-8aee-47bdee25ab4c" containerID="196d58c2ceb185436c7fa0ac7f6436b47d579e63fdc63dfba4cb446a4f7e275f" exitCode=0
Sep 05 06:50:01.947134 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.946962    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"
Sep 05 06:50:01.947134 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.947002    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q" event={"ID":"1b47ad64-5c22-4c36-8aee-47bdee25ab4c","Type":"ContainerDied","Data":"196d58c2ceb185436c7fa0ac7f6436b47d579e63fdc63dfba4cb446a4f7e275f"}
Sep 05 06:50:01.947134 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.947053    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q" event={"ID":"1b47ad64-5c22-4c36-8aee-47bdee25ab4c","Type":"ContainerDied","Data":"0a3ce3e13e943d3b8ac8d0c6187371d7e744007d09f02acae2948fd512c654b6"}
Sep 05 06:50:01.947134 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.947071    2559 scope.go:117] "RemoveContainer" containerID="196d58c2ceb185436c7fa0ac7f6436b47d579e63fdc63dfba4cb446a4f7e275f"
Sep 05 06:50:01.959519 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.959495    2559 scope.go:117] "RemoveContainer" containerID="196d58c2ceb185436c7fa0ac7f6436b47d579e63fdc63dfba4cb446a4f7e275f"
Sep 05 06:50:01.959876 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:50:01.959856    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"196d58c2ceb185436c7fa0ac7f6436b47d579e63fdc63dfba4cb446a4f7e275f\": container with ID starting with 196d58c2ceb185436c7fa0ac7f6436b47d579e63fdc63dfba4cb446a4f7e275f not found: ID does not exist" containerID="196d58c2ceb185436c7fa0ac7f6436b47d579e63fdc63dfba4cb446a4f7e275f"
Sep 05 06:50:01.959985 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.959903    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"196d58c2ceb185436c7fa0ac7f6436b47d579e63fdc63dfba4cb446a4f7e275f"} err="failed to get container status \"196d58c2ceb185436c7fa0ac7f6436b47d579e63fdc63dfba4cb446a4f7e275f\": rpc error: code = NotFound desc = could not find container \"196d58c2ceb185436c7fa0ac7f6436b47d579e63fdc63dfba4cb446a4f7e275f\": container with ID starting with 196d58c2ceb185436c7fa0ac7f6436b47d579e63fdc63dfba4cb446a4f7e275f not found: ID does not exist"
Sep 05 06:50:01.965293 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.965263    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"]
Sep 05 06:50:01.969451 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:01.969423    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/sub-test-model-183ae6-kserve-6747fb9ff-76q2q"]
Sep 05 06:50:02.614563 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.614514    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp" podUID="86456493-9bf4-4263-8150-950154705a85" containerName="main" containerID="cri-o://09bc8cc026528e5bbc51615d2d3df2cb0ac3c62cb2afa446b3d551bcfd5d6c9e" gracePeriod=45
Sep 05 06:50:02.880814 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.880789    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:50:02.954437 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.954405    2559 generic.go:356] "Generic (PLEG): container finished" podID="86456493-9bf4-4263-8150-950154705a85" containerID="09bc8cc026528e5bbc51615d2d3df2cb0ac3c62cb2afa446b3d551bcfd5d6c9e" exitCode=0
Sep 05 06:50:02.954581 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.954445    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"
Sep 05 06:50:02.954581 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.954489    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp" event={"ID":"86456493-9bf4-4263-8150-950154705a85","Type":"ContainerDied","Data":"09bc8cc026528e5bbc51615d2d3df2cb0ac3c62cb2afa446b3d551bcfd5d6c9e"}
Sep 05 06:50:02.954581 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.954536    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp" event={"ID":"86456493-9bf4-4263-8150-950154705a85","Type":"ContainerDied","Data":"44501be26c80b0ee097c83dd9784adee4a8a750e4e8693e4bf7fd1c462c06ea2"}
Sep 05 06:50:02.954581 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.954561    2559 scope.go:117] "RemoveContainer" containerID="09bc8cc026528e5bbc51615d2d3df2cb0ac3c62cb2afa446b3d551bcfd5d6c9e"
Sep 05 06:50:02.963538 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.963522    2559 scope.go:117] "RemoveContainer" containerID="09bc8cc026528e5bbc51615d2d3df2cb0ac3c62cb2afa446b3d551bcfd5d6c9e"
Sep 05 06:50:02.963822 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:50:02.963804    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09bc8cc026528e5bbc51615d2d3df2cb0ac3c62cb2afa446b3d551bcfd5d6c9e\": container with ID starting with 09bc8cc026528e5bbc51615d2d3df2cb0ac3c62cb2afa446b3d551bcfd5d6c9e not found: ID does not exist" containerID="09bc8cc026528e5bbc51615d2d3df2cb0ac3c62cb2afa446b3d551bcfd5d6c9e"
Sep 05 06:50:02.963872 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.963832    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09bc8cc026528e5bbc51615d2d3df2cb0ac3c62cb2afa446b3d551bcfd5d6c9e"} err="failed to get container status \"09bc8cc026528e5bbc51615d2d3df2cb0ac3c62cb2afa446b3d551bcfd5d6c9e\": rpc error: code = NotFound desc = could not find container \"09bc8cc026528e5bbc51615d2d3df2cb0ac3c62cb2afa446b3d551bcfd5d6c9e\": container with ID starting with 09bc8cc026528e5bbc51615d2d3df2cb0ac3c62cb2afa446b3d551bcfd5d6c9e not found: ID does not exist"
Sep 05 06:50:02.991621 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.991598    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86456493-9bf4-4263-8150-950154705a85-tls-certs\") pod \"86456493-9bf4-4263-8150-950154705a85\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") "
Sep 05 06:50:02.991725 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.991686    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-tmp-dir\") pod \"86456493-9bf4-4263-8150-950154705a85\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") "
Sep 05 06:50:02.991773 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.991747    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4sgsn\" (UniqueName: \"kubernetes.io/projected/86456493-9bf4-4263-8150-950154705a85-kube-api-access-4sgsn\") pod \"86456493-9bf4-4263-8150-950154705a85\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") "
Sep 05 06:50:02.991811 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.991772    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-home\") pod \"86456493-9bf4-4263-8150-950154705a85\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") "
Sep 05 06:50:02.991857 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.991828    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-dshm\") pod \"86456493-9bf4-4263-8150-950154705a85\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") "
Sep 05 06:50:02.991912 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.991853    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-model-cache\") pod \"86456493-9bf4-4263-8150-950154705a85\" (UID: \"86456493-9bf4-4263-8150-950154705a85\") "
Sep 05 06:50:02.992026 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.991977    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "86456493-9bf4-4263-8150-950154705a85" (UID: "86456493-9bf4-4263-8150-950154705a85"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:50:02.992102 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.992047    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-home" (OuterVolumeSpecName: "home") pod "86456493-9bf4-4263-8150-950154705a85" (UID: "86456493-9bf4-4263-8150-950154705a85"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:50:02.992227 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.992203    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-model-cache" (OuterVolumeSpecName: "model-cache") pod "86456493-9bf4-4263-8150-950154705a85" (UID: "86456493-9bf4-4263-8150-950154705a85"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:50:02.992337 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.992256    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-tmp-dir\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:02.992337 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.992279    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-home\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:02.993731 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.993712    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86456493-9bf4-4263-8150-950154705a85-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "86456493-9bf4-4263-8150-950154705a85" (UID: "86456493-9bf4-4263-8150-950154705a85"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:50:02.993804 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.993726    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86456493-9bf4-4263-8150-950154705a85-kube-api-access-4sgsn" (OuterVolumeSpecName: "kube-api-access-4sgsn") pod "86456493-9bf4-4263-8150-950154705a85" (UID: "86456493-9bf4-4263-8150-950154705a85"). InnerVolumeSpecName "kube-api-access-4sgsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:50:02.993843 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:02.993806    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-dshm" (OuterVolumeSpecName: "dshm") pod "86456493-9bf4-4263-8150-950154705a85" (UID: "86456493-9bf4-4263-8150-950154705a85"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:50:03.093525 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.093485    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-dshm\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:03.093525 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.093521    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86456493-9bf4-4263-8150-950154705a85-model-cache\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:03.093525 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.093531    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86456493-9bf4-4263-8150-950154705a85-tls-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:03.093776 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.093540    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4sgsn\" (UniqueName: \"kubernetes.io/projected/86456493-9bf4-4263-8150-950154705a85-kube-api-access-4sgsn\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:03.277191 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.277155    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"]
Sep 05 06:50:03.281215 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.281184    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/sub-test-model-39047b-kserve-67564f488f-2frpp"]
Sep 05 06:50:03.481529 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.481497    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"]
Sep 05 06:50:03.482014 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.481994    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86456493-9bf4-4263-8150-950154705a85" containerName="main"
Sep 05 06:50:03.482075 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.482017    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="86456493-9bf4-4263-8150-950154705a85" containerName="main"
Sep 05 06:50:03.482075 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.482042    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b47ad64-5c22-4c36-8aee-47bdee25ab4c" containerName="main"
Sep 05 06:50:03.482075 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.482048    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b47ad64-5c22-4c36-8aee-47bdee25ab4c" containerName="main"
Sep 05 06:50:03.482167 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.482119    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="86456493-9bf4-4263-8150-950154705a85" containerName="main"
Sep 05 06:50:03.482167 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.482128    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b47ad64-5c22-4c36-8aee-47bdee25ab4c" containerName="main"
Sep 05 06:50:03.486736 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.486716    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.490136 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.490013    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-183ae6\"/\"kube-root-ca.crt\""
Sep 05 06:50:03.490136 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.490111    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-183ae6\"/\"openshift-service-ca.crt\""
Sep 05 06:50:03.491109 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.491081    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-183ae6\"/\"rate-test-model-183ae6-kserve-self-signed-certs\""
Sep 05 06:50:03.491772 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.491380    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-183ae6\"/\"default-dockercfg-hnd2s\""
Sep 05 06:50:03.493411 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.493370    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"]
Sep 05 06:50:03.598711 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.598640    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-home\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.598711 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.598670    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-tls-certs\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.598711 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.598702    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-model-cache\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.599018 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.598828    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52vvn\" (UniqueName: \"kubernetes.io/projected/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-kube-api-access-52vvn\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.599018 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.598931    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-tmp-dir\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.599018 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.598963    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-dshm\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.700474 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.700440    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52vvn\" (UniqueName: \"kubernetes.io/projected/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-kube-api-access-52vvn\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.700928 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.700621    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-tmp-dir\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.700928 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.700655    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-dshm\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.700928 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.700691    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-home\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.700928 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.700716    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-tls-certs\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.700928 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.700765    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-model-cache\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.701146 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.701097    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-tmp-dir\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.701841 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.701786    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-model-cache\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.701841 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.701809    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-home\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.703894 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.703846    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-tls-certs\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.704095 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.704070    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-dshm\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.710424 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.710399    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52vvn\" (UniqueName: \"kubernetes.io/projected/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-kube-api-access-52vvn\") pod \"rate-test-model-183ae6-kserve-777ddb9475-w4plj\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") " pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.799915 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.799868    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:03.894078 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.894052    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b47ad64-5c22-4c36-8aee-47bdee25ab4c" path="/var/lib/kubelet/pods/1b47ad64-5c22-4c36-8aee-47bdee25ab4c/volumes"
Sep 05 06:50:03.894457 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:03.894445    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86456493-9bf4-4263-8150-950154705a85" path="/var/lib/kubelet/pods/86456493-9bf4-4263-8150-950154705a85/volumes"
Sep 05 06:50:04.074943 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:04.074910    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"]
Sep 05 06:50:04.076599 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:50:04.076567    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9b9c158_ad01_4a7b_a0b7_aac13f87b2a7.slice/crio-043b798221415ae47641e8546737a27ad6ac6da777863c1d3729ed3518e42122 WatchSource:0}: Error finding container 043b798221415ae47641e8546737a27ad6ac6da777863c1d3729ed3518e42122: Status 404 returned error can't find the container with id 043b798221415ae47641e8546737a27ad6ac6da777863c1d3729ed3518e42122
Sep 05 06:50:04.970025 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:04.969990    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj" event={"ID":"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7","Type":"ContainerStarted","Data":"35d6eb2b03c5a32c8204605e89a8dee262a6f7567596d8b50bc9fe2d44a5d551"}
Sep 05 06:50:04.970025 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:04.970033    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj" event={"ID":"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7","Type":"ContainerStarted","Data":"043b798221415ae47641e8546737a27ad6ac6da777863c1d3729ed3518e42122"}
Sep 05 06:50:04.988829 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:04.988780    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj" podStartSLOduration=1.9887639030000002 podStartE2EDuration="1.988763903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:50:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:50:04.988120473 +0000 UTC m=+1421.578679438" watchObservedRunningTime="2026-09-05 06:50:04.988763903 +0000 UTC m=+1421.579322855"
Sep 05 06:50:11.001946 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:11.001902    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:50:13.028416 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:13.028378    2559 generic.go:356] "Generic (PLEG): container finished" podID="c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" containerID="3da03b1ffcb46eb3079475bffd81e4a04114722cca529ce6b81fbfef7256083a" exitCode=0
Sep 05 06:50:13.028973 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:13.028445    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs" event={"ID":"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a","Type":"ContainerDied","Data":"3da03b1ffcb46eb3079475bffd81e4a04114722cca529ce6b81fbfef7256083a"}
Sep 05 06:50:13.028973 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:13.028829    2559 scope.go:117] "RemoveContainer" containerID="3da03b1ffcb46eb3079475bffd81e4a04114722cca529ce6b81fbfef7256083a"
Sep 05 06:50:13.030085 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:13.030050    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:50:13.030154 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:13.030125    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:50:13.030208 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:13.030159    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:50:13.800522 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:13.800478    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:13.800707 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:13.800545    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:13.813119 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:13.813090    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:14.042316 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:14.042272    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs" event={"ID":"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a","Type":"ContainerStarted","Data":"022703f583f159b016ba455fd197a3ed6cff59d8558a842359825cf86b5a2653"}
Sep 05 06:50:14.063141 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:14.063014    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:50:14.618922 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:14.618890    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:50:14.619124 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:14.619100    2559 patch_prober.go:28] interesting pod/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 05 06:50:14.619173 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:14.619159    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs" podUID="c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 05 06:50:15.619450 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:15.619413    2559 patch_prober.go:28] interesting pod/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 05 06:50:15.619852 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:15.619483    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs" podUID="c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 05 06:50:16.619720 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:16.619632    2559 patch_prober.go:28] interesting pod/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 05 06:50:16.619720 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:16.619707    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs" podUID="c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 05 06:50:17.619233 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:17.619196    2559 patch_prober.go:28] interesting pod/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 05 06:50:17.619411 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:17.619254    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs" podUID="c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 05 06:50:18.618767 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:18.618728    2559 patch_prober.go:28] interesting pod/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Sep 05 06:50:18.619174 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:18.618785    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs" podUID="c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Sep 05 06:50:19.623219 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:19.623184    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:50:19.623652 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:19.623591    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:50:19.624358 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:19.624336    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:50:20.706737 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:20.706703    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-96d352-8c6ll"]
Sep 05 06:50:20.709270 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:20.709250    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-96d352-8c6ll"
Sep 05 06:50:20.711627 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:20.711611    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slfqj\""
Sep 05 06:50:20.711748 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:20.711646    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 06:50:20.718508 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:20.718483    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-96d352-8c6ll"]
Sep 05 06:50:20.780379 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:20.780351    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7006f88d-75fa-4056-bf72-2397ce4bf93e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-96d352-8c6ll\" (UID: \"7006f88d-75fa-4056-bf72-2397ce4bf93e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-96d352-8c6ll"
Sep 05 06:50:20.881329 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:20.881298    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7006f88d-75fa-4056-bf72-2397ce4bf93e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-96d352-8c6ll\" (UID: \"7006f88d-75fa-4056-bf72-2397ce4bf93e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-96d352-8c6ll"
Sep 05 06:50:20.881921 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:20.881875    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7006f88d-75fa-4056-bf72-2397ce4bf93e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-96d352-8c6ll\" (UID: \"7006f88d-75fa-4056-bf72-2397ce4bf93e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-96d352-8c6ll"
Sep 05 06:50:21.022054 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:21.021970    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-96d352-8c6ll"
Sep 05 06:50:21.294277 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:21.294248    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-96d352-8c6ll"]
Sep 05 06:50:21.296565 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:50:21.296537    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7006f88d_75fa_4056_bf72_2397ce4bf93e.slice/crio-de6bdb89b0c1d68803893a10d639c3d4ee15a124c0201a1764de66e98abaaa0a WatchSource:0}: Error finding container de6bdb89b0c1d68803893a10d639c3d4ee15a124c0201a1764de66e98abaaa0a: Status 404 returned error can't find the container with id de6bdb89b0c1d68803893a10d639c3d4ee15a124c0201a1764de66e98abaaa0a
Sep 05 06:50:22.099500 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:22.099469    2559 generic.go:356] "Generic (PLEG): container finished" podID="7006f88d-75fa-4056-bf72-2397ce4bf93e" containerID="b3d4b8c480aab2ab8e0650ab585cbcac38bb1f8e7c96aeaa9ee0c54016dac5f8" exitCode=0
Sep 05 06:50:22.099852 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:22.099562    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-96d352-8c6ll" event={"ID":"7006f88d-75fa-4056-bf72-2397ce4bf93e","Type":"ContainerDied","Data":"b3d4b8c480aab2ab8e0650ab585cbcac38bb1f8e7c96aeaa9ee0c54016dac5f8"}
Sep 05 06:50:22.099852 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:22.099621    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-96d352-8c6ll" event={"ID":"7006f88d-75fa-4056-bf72-2397ce4bf93e","Type":"ContainerStarted","Data":"de6bdb89b0c1d68803893a10d639c3d4ee15a124c0201a1764de66e98abaaa0a"}
Sep 05 06:50:23.238798 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:23.238766    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-96d352-8c6ll"
Sep 05 06:50:23.303058 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:23.303030    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7006f88d-75fa-4056-bf72-2397ce4bf93e-maas-api-service-ca\") pod \"7006f88d-75fa-4056-bf72-2397ce4bf93e\" (UID: \"7006f88d-75fa-4056-bf72-2397ce4bf93e\") "
Sep 05 06:50:23.303367 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:23.303346    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7006f88d-75fa-4056-bf72-2397ce4bf93e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7006f88d-75fa-4056-bf72-2397ce4bf93e" (UID: "7006f88d-75fa-4056-bf72-2397ce4bf93e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:50:23.404682 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:23.404603    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7006f88d-75fa-4056-bf72-2397ce4bf93e-maas-api-service-ca\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:24.112756 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:24.112726    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-96d352-8c6ll"
Sep 05 06:50:24.112756 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:24.112747    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-96d352-8c6ll" event={"ID":"7006f88d-75fa-4056-bf72-2397ce4bf93e","Type":"ContainerDied","Data":"de6bdb89b0c1d68803893a10d639c3d4ee15a124c0201a1764de66e98abaaa0a"}
Sep 05 06:50:24.112933 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:24.112788    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de6bdb89b0c1d68803893a10d639c3d4ee15a124c0201a1764de66e98abaaa0a"
Sep 05 06:50:36.634437 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:36.634402    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-96d352-8c6ll"]
Sep 05 06:50:36.637057 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:36.637031    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-96d352-8c6ll"]
Sep 05 06:50:37.898995 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:37.898961    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7006f88d-75fa-4056-bf72-2397ce4bf93e" path="/var/lib/kubelet/pods/7006f88d-75fa-4056-bf72-2397ce4bf93e/volumes"
Sep 05 06:50:42.385090 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.385053    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"]
Sep 05 06:50:42.385549 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.385494    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7006f88d-75fa-4056-bf72-2397ce4bf93e" containerName="revoke-keys"
Sep 05 06:50:42.385549 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.385508    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7006f88d-75fa-4056-bf72-2397ce4bf93e" containerName="revoke-keys"
Sep 05 06:50:42.385660 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.385591    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7006f88d-75fa-4056-bf72-2397ce4bf93e" containerName="revoke-keys"
Sep 05 06:50:42.388448 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.388423    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.391407 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.391373    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-39047b\"/\"openshift-service-ca.crt\""
Sep 05 06:50:42.391556 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.391410    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-39047b\"/\"rate-test-model-39047b-kserve-self-signed-certs\""
Sep 05 06:50:42.391556 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.391376    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-39047b\"/\"kube-root-ca.crt\""
Sep 05 06:50:42.392243 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.392221    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-39047b\"/\"default-dockercfg-cskxb\""
Sep 05 06:50:42.398258 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.398228    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"]
Sep 05 06:50:42.477707 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.477667    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-dshm\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.477707 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.477710    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-model-cache\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.477970 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.477786    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvj5l\" (UniqueName: \"kubernetes.io/projected/33c4d6de-7144-4d65-8764-0ebe903e4968-kube-api-access-mvj5l\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.477970 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.477818    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33c4d6de-7144-4d65-8764-0ebe903e4968-tls-certs\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.477970 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.477895    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-home\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.477970 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.477962    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-tmp-dir\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.579256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.579217    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-dshm\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.579256 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.579257    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-model-cache\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.579509 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.579389    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvj5l\" (UniqueName: \"kubernetes.io/projected/33c4d6de-7144-4d65-8764-0ebe903e4968-kube-api-access-mvj5l\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.579509 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.579422    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33c4d6de-7144-4d65-8764-0ebe903e4968-tls-certs\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.579509 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.579454    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-home\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.579682 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.579512    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-tmp-dir\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.579741 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.579697    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-model-cache\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.579934 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.579902    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-home\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.580022 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.579968    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-tmp-dir\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.581001 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.580973    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:50:42.581732 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.581708    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-dshm\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.581905 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.581862    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33c4d6de-7144-4d65-8764-0ebe903e4968-tls-certs\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.591790 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.591759    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvj5l\" (UniqueName: \"kubernetes.io/projected/33c4d6de-7144-4d65-8764-0ebe903e4968-kube-api-access-mvj5l\") pod \"rate-test-model-39047b-kserve-5d7cfccb7f-lt59v\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") " pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.700429 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.700398    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:42.979752 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:42.979717    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"]
Sep 05 06:50:42.981396 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:50:42.981361    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33c4d6de_7144_4d65_8764_0ebe903e4968.slice/crio-c72c80e9c63764c7e6d73a1d6c64c186b8e83b9446758269adfdb37f93594fcf WatchSource:0}: Error finding container c72c80e9c63764c7e6d73a1d6c64c186b8e83b9446758269adfdb37f93594fcf: Status 404 returned error can't find the container with id c72c80e9c63764c7e6d73a1d6c64c186b8e83b9446758269adfdb37f93594fcf
Sep 05 06:50:43.230553 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:43.230460    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v" event={"ID":"33c4d6de-7144-4d65-8764-0ebe903e4968","Type":"ContainerStarted","Data":"57b2723ebb1ad04c36fba0fc3aeccabc9aa095ed5d7204a0f319fc7837298e72"}
Sep 05 06:50:43.230553 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:43.230515    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v" event={"ID":"33c4d6de-7144-4d65-8764-0ebe903e4968","Type":"ContainerStarted","Data":"c72c80e9c63764c7e6d73a1d6c64c186b8e83b9446758269adfdb37f93594fcf"}
Sep 05 06:50:43.248477 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:43.248423    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v" podStartSLOduration=1.248402933 podStartE2EDuration="1.248402933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:50:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:50:43.247625544 +0000 UTC m=+1459.838184497" watchObservedRunningTime="2026-09-05 06:50:43.248402933 +0000 UTC m=+1459.838961889"
Sep 05 06:50:43.927274 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:43.927232    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"]
Sep 05 06:50:43.927770 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:43.927512    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24" podUID="6e8e7b38-bf46-47f7-b65b-4ce31ad05581" containerName="istio-proxy" containerID="cri-o://b9e8ba3ac842a4a3e90c27925912d8bd99c280637fed3d31b303e7a55b06b5e9" gracePeriod=30
Sep 05 06:50:49.224601 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.224555    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:50:49.271805 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.271763    2559 generic.go:356] "Generic (PLEG): container finished" podID="6e8e7b38-bf46-47f7-b65b-4ce31ad05581" containerID="b9e8ba3ac842a4a3e90c27925912d8bd99c280637fed3d31b303e7a55b06b5e9" exitCode=0
Sep 05 06:50:49.272052 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.271831    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24" event={"ID":"6e8e7b38-bf46-47f7-b65b-4ce31ad05581","Type":"ContainerDied","Data":"b9e8ba3ac842a4a3e90c27925912d8bd99c280637fed3d31b303e7a55b06b5e9"}
Sep 05 06:50:49.272052 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.271847    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"
Sep 05 06:50:49.272052 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.271905    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24" event={"ID":"6e8e7b38-bf46-47f7-b65b-4ce31ad05581","Type":"ContainerDied","Data":"57360353c1b5a8c571956692673a830e117213801df451605a670ca1e0fce98b"}
Sep 05 06:50:49.272052 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.271940    2559 scope.go:117] "RemoveContainer" containerID="b9e8ba3ac842a4a3e90c27925912d8bd99c280637fed3d31b303e7a55b06b5e9"
Sep 05 06:50:49.282321 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.282294    2559 scope.go:117] "RemoveContainer" containerID="b3ee28a0c499c273ce015ed704938095e39447711fe922e9e2a3d370fbf6dbfc"
Sep 05 06:50:49.292148 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.292124    2559 scope.go:117] "RemoveContainer" containerID="b9e8ba3ac842a4a3e90c27925912d8bd99c280637fed3d31b303e7a55b06b5e9"
Sep 05 06:50:49.292521 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:50:49.292498    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9e8ba3ac842a4a3e90c27925912d8bd99c280637fed3d31b303e7a55b06b5e9\": container with ID starting with b9e8ba3ac842a4a3e90c27925912d8bd99c280637fed3d31b303e7a55b06b5e9 not found: ID does not exist" containerID="b9e8ba3ac842a4a3e90c27925912d8bd99c280637fed3d31b303e7a55b06b5e9"
Sep 05 06:50:49.292570 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.292531    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9e8ba3ac842a4a3e90c27925912d8bd99c280637fed3d31b303e7a55b06b5e9"} err="failed to get container status \"b9e8ba3ac842a4a3e90c27925912d8bd99c280637fed3d31b303e7a55b06b5e9\": rpc error: code = NotFound desc = could not find container \"b9e8ba3ac842a4a3e90c27925912d8bd99c280637fed3d31b303e7a55b06b5e9\": container with ID starting with b9e8ba3ac842a4a3e90c27925912d8bd99c280637fed3d31b303e7a55b06b5e9 not found: ID does not exist"
Sep 05 06:50:49.292570 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.292549    2559 scope.go:117] "RemoveContainer" containerID="b3ee28a0c499c273ce015ed704938095e39447711fe922e9e2a3d370fbf6dbfc"
Sep 05 06:50:49.292981 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:50:49.292857    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3ee28a0c499c273ce015ed704938095e39447711fe922e9e2a3d370fbf6dbfc\": container with ID starting with b3ee28a0c499c273ce015ed704938095e39447711fe922e9e2a3d370fbf6dbfc not found: ID does not exist" containerID="b3ee28a0c499c273ce015ed704938095e39447711fe922e9e2a3d370fbf6dbfc"
Sep 05 06:50:49.293100 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.292983    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3ee28a0c499c273ce015ed704938095e39447711fe922e9e2a3d370fbf6dbfc"} err="failed to get container status \"b3ee28a0c499c273ce015ed704938095e39447711fe922e9e2a3d370fbf6dbfc\": rpc error: code = NotFound desc = could not find container \"b3ee28a0c499c273ce015ed704938095e39447711fe922e9e2a3d370fbf6dbfc\": container with ID starting with b3ee28a0c499c273ce015ed704938095e39447711fe922e9e2a3d370fbf6dbfc not found: ID does not exist"
Sep 05 06:50:49.348281 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.348242    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istiod-ca-cert\") pod \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") "
Sep 05 06:50:49.348281 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.348293    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p76hn\" (UniqueName: \"kubernetes.io/projected/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-kube-api-access-p76hn\") pod \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") "
Sep 05 06:50:49.348554 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.348329    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-data\") pod \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") "
Sep 05 06:50:49.348554 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.348370    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-workload-certs\") pod \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") "
Sep 05 06:50:49.348554 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.348404    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-token\") pod \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") "
Sep 05 06:50:49.348554 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.348466    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-podinfo\") pod \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") "
Sep 05 06:50:49.348554 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.348533    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-envoy\") pod \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") "
Sep 05 06:50:49.348823 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.348561    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-workload-socket\") pod \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") "
Sep 05 06:50:49.348823 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.348585    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-credential-socket\") pod \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\" (UID: \"6e8e7b38-bf46-47f7-b65b-4ce31ad05581\") "
Sep 05 06:50:49.348823 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.348685    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-data" (OuterVolumeSpecName: "istio-data") pod "6e8e7b38-bf46-47f7-b65b-4ce31ad05581" (UID: "6e8e7b38-bf46-47f7-b65b-4ce31ad05581"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:50:49.348823 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.348728    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6e8e7b38-bf46-47f7-b65b-4ce31ad05581" (UID: "6e8e7b38-bf46-47f7-b65b-4ce31ad05581"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:50:49.348823 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.348698    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6e8e7b38-bf46-47f7-b65b-4ce31ad05581" (UID: "6e8e7b38-bf46-47f7-b65b-4ce31ad05581"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:50:49.349133 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.348925    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6e8e7b38-bf46-47f7-b65b-4ce31ad05581" (UID: "6e8e7b38-bf46-47f7-b65b-4ce31ad05581"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:50:49.349133 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.348949    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6e8e7b38-bf46-47f7-b65b-4ce31ad05581" (UID: "6e8e7b38-bf46-47f7-b65b-4ce31ad05581"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:50:49.349133 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.349022    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-workload-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:49.349133 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.349038    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-workload-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:49.349133 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.349049    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istiod-ca-cert\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:49.349133 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.349058    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-data\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:49.351009 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.350966    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6e8e7b38-bf46-47f7-b65b-4ce31ad05581" (UID: "6e8e7b38-bf46-47f7-b65b-4ce31ad05581"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 06:50:49.351260 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.351220    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e8e7b38-bf46-47f7-b65b-4ce31ad05581" (UID: "6e8e7b38-bf46-47f7-b65b-4ce31ad05581"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:50:49.351370 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.351287    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-kube-api-access-p76hn" (OuterVolumeSpecName: "kube-api-access-p76hn") pod "6e8e7b38-bf46-47f7-b65b-4ce31ad05581" (UID: "6e8e7b38-bf46-47f7-b65b-4ce31ad05581"). InnerVolumeSpecName "kube-api-access-p76hn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:50:49.351370 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.351329    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6e8e7b38-bf46-47f7-b65b-4ce31ad05581" (UID: "6e8e7b38-bf46-47f7-b65b-4ce31ad05581"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:50:49.450628 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.450534    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-envoy\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:49.450628 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.450623    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-credential-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:49.450840 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.450640    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p76hn\" (UniqueName: \"kubernetes.io/projected/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-kube-api-access-p76hn\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:49.450840 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.450654    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-token\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:49.450840 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.450699    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e8e7b38-bf46-47f7-b65b-4ce31ad05581-istio-podinfo\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:50:49.600765 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.600723    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"]
Sep 05 06:50:49.606269 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.606225    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-96d352-openshift-default-66c4489854-p8g24"]
Sep 05 06:50:49.894460 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:49.894368    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e8e7b38-bf46-47f7-b65b-4ce31ad05581" path="/var/lib/kubelet/pods/6e8e7b38-bf46-47f7-b65b-4ce31ad05581/volumes"
Sep 05 06:50:52.701474 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:52.701422    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:52.701474 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:52.701486    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:52.714225 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:52.714194    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:53.309348 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:53.309315    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:50:56.092104 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:50:56.092068    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:51:07.064004 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:07.063961    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"]
Sep 05 06:51:07.064454 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:07.064208    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs" podUID="c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" containerName="istio-proxy" containerID="cri-o://022703f583f159b016ba455fd197a3ed6cff59d8558a842359825cf86b5a2653" gracePeriod=30
Sep 05 06:51:12.315899 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.315863    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:51:12.361805 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.361737    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-envoy\") pod \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") "
Sep 05 06:51:12.361805 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.361781    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-data\") pod \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") "
Sep 05 06:51:12.362061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.361828    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-workload-socket\") pod \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") "
Sep 05 06:51:12.362061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.361857    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-token\") pod \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") "
Sep 05 06:51:12.362061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.361874    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istiod-ca-cert\") pod \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") "
Sep 05 06:51:12.362061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.361918    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rxzh\" (UniqueName: \"kubernetes.io/projected/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-kube-api-access-7rxzh\") pod \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") "
Sep 05 06:51:12.362061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.361971    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-credential-socket\") pod \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") "
Sep 05 06:51:12.362061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.362019    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-workload-certs\") pod \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") "
Sep 05 06:51:12.362061 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.362047    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-podinfo\") pod \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\" (UID: \"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a\") "
Sep 05 06:51:12.362409 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.362114    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" (UID: "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:12.362409 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.362270    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" (UID: "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:12.362409 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.362290    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" (UID: "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:12.362409 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.362349    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" (UID: "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:51:12.362586 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.362422    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-credential-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:12.362586 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.362440    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-workload-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:12.362586 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.362443    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-data" (OuterVolumeSpecName: "istio-data") pod "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" (UID: "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:12.362586 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.362455    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-workload-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:12.364273 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.364248    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" (UID: "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 06:51:12.364448 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.364427    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" (UID: "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:12.364530 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.364449    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-token" (OuterVolumeSpecName: "istio-token") pod "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" (UID: "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:51:12.364530 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.364431    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-kube-api-access-7rxzh" (OuterVolumeSpecName: "kube-api-access-7rxzh") pod "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" (UID: "c4a22d30-2cb6-4b94-b1b7-4a75669dc91a"). InnerVolumeSpecName "kube-api-access-7rxzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:51:12.420621 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.420589    2559 generic.go:356] "Generic (PLEG): container finished" podID="c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" containerID="022703f583f159b016ba455fd197a3ed6cff59d8558a842359825cf86b5a2653" exitCode=0
Sep 05 06:51:12.420756 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.420642    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"
Sep 05 06:51:12.420756 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.420677    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs" event={"ID":"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a","Type":"ContainerDied","Data":"022703f583f159b016ba455fd197a3ed6cff59d8558a842359825cf86b5a2653"}
Sep 05 06:51:12.420756 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.420733    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs" event={"ID":"c4a22d30-2cb6-4b94-b1b7-4a75669dc91a","Type":"ContainerDied","Data":"b0ae8a2153418a13caba951425c712eeca0106512c9d8c4fde650952b7b6d2a1"}
Sep 05 06:51:12.420952 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.420763    2559 scope.go:117] "RemoveContainer" containerID="022703f583f159b016ba455fd197a3ed6cff59d8558a842359825cf86b5a2653"
Sep 05 06:51:12.431478 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.431461    2559 scope.go:117] "RemoveContainer" containerID="3da03b1ffcb46eb3079475bffd81e4a04114722cca529ce6b81fbfef7256083a"
Sep 05 06:51:12.439984 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.439959    2559 scope.go:117] "RemoveContainer" containerID="022703f583f159b016ba455fd197a3ed6cff59d8558a842359825cf86b5a2653"
Sep 05 06:51:12.440227 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:51:12.440206    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"022703f583f159b016ba455fd197a3ed6cff59d8558a842359825cf86b5a2653\": container with ID starting with 022703f583f159b016ba455fd197a3ed6cff59d8558a842359825cf86b5a2653 not found: ID does not exist" containerID="022703f583f159b016ba455fd197a3ed6cff59d8558a842359825cf86b5a2653"
Sep 05 06:51:12.440282 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.440235    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"022703f583f159b016ba455fd197a3ed6cff59d8558a842359825cf86b5a2653"} err="failed to get container status \"022703f583f159b016ba455fd197a3ed6cff59d8558a842359825cf86b5a2653\": rpc error: code = NotFound desc = could not find container \"022703f583f159b016ba455fd197a3ed6cff59d8558a842359825cf86b5a2653\": container with ID starting with 022703f583f159b016ba455fd197a3ed6cff59d8558a842359825cf86b5a2653 not found: ID does not exist"
Sep 05 06:51:12.440282 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.440250    2559 scope.go:117] "RemoveContainer" containerID="3da03b1ffcb46eb3079475bffd81e4a04114722cca529ce6b81fbfef7256083a"
Sep 05 06:51:12.440516 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:51:12.440501    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3da03b1ffcb46eb3079475bffd81e4a04114722cca529ce6b81fbfef7256083a\": container with ID starting with 3da03b1ffcb46eb3079475bffd81e4a04114722cca529ce6b81fbfef7256083a not found: ID does not exist" containerID="3da03b1ffcb46eb3079475bffd81e4a04114722cca529ce6b81fbfef7256083a"
Sep 05 06:51:12.440569 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.440520    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3da03b1ffcb46eb3079475bffd81e4a04114722cca529ce6b81fbfef7256083a"} err="failed to get container status \"3da03b1ffcb46eb3079475bffd81e4a04114722cca529ce6b81fbfef7256083a\": rpc error: code = NotFound desc = could not find container \"3da03b1ffcb46eb3079475bffd81e4a04114722cca529ce6b81fbfef7256083a\": container with ID starting with 3da03b1ffcb46eb3079475bffd81e4a04114722cca529ce6b81fbfef7256083a not found: ID does not exist"
Sep 05 06:51:12.452382 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.452359    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"]
Sep 05 06:51:12.455347 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.455325    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a9301f-openshift-default-7d648d4966-mcsbs"]
Sep 05 06:51:12.463293 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.463275    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-podinfo\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:12.463353 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.463295    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-envoy\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:12.463353 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.463304    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-data\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:12.463353 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.463312    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istio-token\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:12.463353 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.463320    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-istiod-ca-cert\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:12.463353 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:12.463327    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rxzh\" (UniqueName: \"kubernetes.io/projected/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a-kube-api-access-7rxzh\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:13.288772 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:13.288733    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:51:13.897817 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:13.897781    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" path="/var/lib/kubelet/pods/c4a22d30-2cb6-4b94-b1b7-4a75669dc91a/volumes"
Sep 05 06:51:17.938260 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:17.938230    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-362c80-29809845-9f9fz"]
Sep 05 06:51:17.943714 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:17.943685    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-362c80-29809845-9f9fz"]
Sep 05 06:51:19.891751 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:19.891709    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a0a67d0-eb07-4aa3-a8ae-e6201c822379" path="/var/lib/kubelet/pods/8a0a67d0-eb07-4aa3-a8ae-e6201c822379/volumes"
Sep 05 06:51:24.098382 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:24.098352    2559 scope.go:117] "RemoveContainer" containerID="17b2cc2d7aab554a29ac0aa4f820ee1401c489efb5bcd63c96a59de3b76bf0a1"
Sep 05 06:51:25.894141 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:25.894103    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:51:32.399074 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:32.399042    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"]
Sep 05 06:51:33.482568 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:33.482530    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"]
Sep 05 06:51:38.380591 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:38.380550    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:51:43.126294 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:43.126250    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"]
Sep 05 06:51:43.126761 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:43.126535    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk" podUID="d0fcf165-bddc-4260-b8e0-2cb4ffeba155" containerName="istio-proxy" containerID="cri-o://cab2cf8f70c51ffcb7b57b577227a07743a49612c19d0c9ed6acac188a661b76" gracePeriod=30
Sep 05 06:51:45.678687 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:45.678642    2559 patch_prober.go:28] interesting pod/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 05 06:51:45.679101 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:45.678722    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk" podUID="d0fcf165-bddc-4260-b8e0-2cb4ffeba155" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 05 06:51:47.418911 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.418773    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj" podUID="e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7" containerName="main" containerID="cri-o://35d6eb2b03c5a32c8204605e89a8dee262a6f7567596d8b50bc9fe2d44a5d551" gracePeriod=45
Sep 05 06:51:47.644917 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.644861    2559 generic.go:356] "Generic (PLEG): container finished" podID="e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7" containerID="35d6eb2b03c5a32c8204605e89a8dee262a6f7567596d8b50bc9fe2d44a5d551" exitCode=0
Sep 05 06:51:47.645105 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.644929    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj" event={"ID":"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7","Type":"ContainerDied","Data":"35d6eb2b03c5a32c8204605e89a8dee262a6f7567596d8b50bc9fe2d44a5d551"}
Sep 05 06:51:47.681588 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.681482    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:51:47.690622 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.690581    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-tls-certs\") pod \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") "
Sep 05 06:51:47.690767 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.690627    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-home\") pod \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") "
Sep 05 06:51:47.690767 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.690672    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-model-cache\") pod \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") "
Sep 05 06:51:47.690767 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.690714    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52vvn\" (UniqueName: \"kubernetes.io/projected/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-kube-api-access-52vvn\") pod \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") "
Sep 05 06:51:47.690976 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.690768    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-dshm\") pod \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") "
Sep 05 06:51:47.691044 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.690974    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-home" (OuterVolumeSpecName: "home") pod "e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7" (UID: "e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:47.691044 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.691020    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-model-cache" (OuterVolumeSpecName: "model-cache") pod "e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7" (UID: "e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:47.691135 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.691107    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-home\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:47.693166 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.693117    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-dshm" (OuterVolumeSpecName: "dshm") pod "e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7" (UID: "e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:47.693335 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.693207    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7" (UID: "e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:51:47.693335 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.693209    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-kube-api-access-52vvn" (OuterVolumeSpecName: "kube-api-access-52vvn") pod "e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7" (UID: "e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7"). InnerVolumeSpecName "kube-api-access-52vvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:51:47.791416 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.791368    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-tmp-dir\") pod \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\" (UID: \"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7\") "
Sep 05 06:51:47.791603 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.791591    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-tls-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:47.791649 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.791608    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-model-cache\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:47.791649 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.791620    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52vvn\" (UniqueName: \"kubernetes.io/projected/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-kube-api-access-52vvn\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:47.791649 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.791630    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-dshm\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:47.791770 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.791661    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7" (UID: "e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:47.892107 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:47.892060    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7-tmp-dir\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:48.375787 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.375755    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:51:48.397303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.396401    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-credential-socket\") pod \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") "
Sep 05 06:51:48.397303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.396455    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hnbl\" (UniqueName: \"kubernetes.io/projected/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-kube-api-access-6hnbl\") pod \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") "
Sep 05 06:51:48.397303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.396489    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-workload-socket\") pod \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") "
Sep 05 06:51:48.397303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.396527    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-envoy\") pod \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") "
Sep 05 06:51:48.397303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.396586    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istiod-ca-cert\") pod \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") "
Sep 05 06:51:48.397303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.396712    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d0fcf165-bddc-4260-b8e0-2cb4ffeba155" (UID: "d0fcf165-bddc-4260-b8e0-2cb4ffeba155"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:48.397303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.396808    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d0fcf165-bddc-4260-b8e0-2cb4ffeba155" (UID: "d0fcf165-bddc-4260-b8e0-2cb4ffeba155"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:48.397303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.396905    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-podinfo\") pod \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") "
Sep 05 06:51:48.397303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.396952    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-data\") pod \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") "
Sep 05 06:51:48.397303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.396986    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-token\") pod \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") "
Sep 05 06:51:48.397303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.397031    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d0fcf165-bddc-4260-b8e0-2cb4ffeba155" (UID: "d0fcf165-bddc-4260-b8e0-2cb4ffeba155"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:51:48.397303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.397041    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-workload-certs\") pod \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\" (UID: \"d0fcf165-bddc-4260-b8e0-2cb4ffeba155\") "
Sep 05 06:51:48.398266 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.397342    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d0fcf165-bddc-4260-b8e0-2cb4ffeba155" (UID: "d0fcf165-bddc-4260-b8e0-2cb4ffeba155"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:48.398266 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.397439    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-credential-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:48.398266 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.397471    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-workload-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:48.398266 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.397488    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istiod-ca-cert\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:48.398266 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.397501    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-workload-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:48.398266 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.398164    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-data" (OuterVolumeSpecName: "istio-data") pod "d0fcf165-bddc-4260-b8e0-2cb4ffeba155" (UID: "d0fcf165-bddc-4260-b8e0-2cb4ffeba155"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:48.400117 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.400074    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-token" (OuterVolumeSpecName: "istio-token") pod "d0fcf165-bddc-4260-b8e0-2cb4ffeba155" (UID: "d0fcf165-bddc-4260-b8e0-2cb4ffeba155"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:51:48.400393 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.400374    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d0fcf165-bddc-4260-b8e0-2cb4ffeba155" (UID: "d0fcf165-bddc-4260-b8e0-2cb4ffeba155"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:48.400514 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.400494    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-kube-api-access-6hnbl" (OuterVolumeSpecName: "kube-api-access-6hnbl") pod "d0fcf165-bddc-4260-b8e0-2cb4ffeba155" (UID: "d0fcf165-bddc-4260-b8e0-2cb4ffeba155"). InnerVolumeSpecName "kube-api-access-6hnbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:51:48.400596 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.400579    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d0fcf165-bddc-4260-b8e0-2cb4ffeba155" (UID: "d0fcf165-bddc-4260-b8e0-2cb4ffeba155"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 06:51:48.498535 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.498493    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-podinfo\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:48.498535 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.498526    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-data\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:48.498535 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.498536    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-token\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:48.498535 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.498546    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hnbl\" (UniqueName: \"kubernetes.io/projected/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-kube-api-access-6hnbl\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:48.499048 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.498555    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0fcf165-bddc-4260-b8e0-2cb4ffeba155-istio-envoy\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:48.503316 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.503272    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v" podUID="33c4d6de-7144-4d65-8764-0ebe903e4968" containerName="main" containerID="cri-o://57b2723ebb1ad04c36fba0fc3aeccabc9aa095ed5d7204a0f319fc7837298e72" gracePeriod=45
Sep 05 06:51:48.654463 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.654345    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"
Sep 05 06:51:48.654463 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.654367    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj" event={"ID":"e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7","Type":"ContainerDied","Data":"043b798221415ae47641e8546737a27ad6ac6da777863c1d3729ed3518e42122"}
Sep 05 06:51:48.654463 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.654436    2559 scope.go:117] "RemoveContainer" containerID="35d6eb2b03c5a32c8204605e89a8dee262a6f7567596d8b50bc9fe2d44a5d551"
Sep 05 06:51:48.656756 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.656720    2559 generic.go:356] "Generic (PLEG): container finished" podID="d0fcf165-bddc-4260-b8e0-2cb4ffeba155" containerID="cab2cf8f70c51ffcb7b57b577227a07743a49612c19d0c9ed6acac188a661b76" exitCode=0
Sep 05 06:51:48.656979 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.656850    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk" event={"ID":"d0fcf165-bddc-4260-b8e0-2cb4ffeba155","Type":"ContainerDied","Data":"cab2cf8f70c51ffcb7b57b577227a07743a49612c19d0c9ed6acac188a661b76"}
Sep 05 06:51:48.656979 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.656928    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk" event={"ID":"d0fcf165-bddc-4260-b8e0-2cb4ffeba155","Type":"ContainerDied","Data":"2a58664dad55151fdcd3189f3ce90b3ec9cf72927a92efa0863a9647afa91369"}
Sep 05 06:51:48.657104 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.657030    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"
Sep 05 06:51:48.659938 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.659909    2559 generic.go:356] "Generic (PLEG): container finished" podID="33c4d6de-7144-4d65-8764-0ebe903e4968" containerID="57b2723ebb1ad04c36fba0fc3aeccabc9aa095ed5d7204a0f319fc7837298e72" exitCode=0
Sep 05 06:51:48.660094 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.659931    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v" event={"ID":"33c4d6de-7144-4d65-8764-0ebe903e4968","Type":"ContainerDied","Data":"57b2723ebb1ad04c36fba0fc3aeccabc9aa095ed5d7204a0f319fc7837298e72"}
Sep 05 06:51:48.667166 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.667136    2559 scope.go:117] "RemoveContainer" containerID="cab2cf8f70c51ffcb7b57b577227a07743a49612c19d0c9ed6acac188a661b76"
Sep 05 06:51:48.676172 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.676129    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"]
Sep 05 06:51:48.678112 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.678090    2559 scope.go:117] "RemoveContainer" containerID="3facc81ab65157e1b06a3000e799c2859c0536441614e4f33463621373718806"
Sep 05 06:51:48.680461 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.680432    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-183ae6/rate-test-model-183ae6-kserve-777ddb9475-w4plj"]
Sep 05 06:51:48.691049 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.691024    2559 scope.go:117] "RemoveContainer" containerID="cab2cf8f70c51ffcb7b57b577227a07743a49612c19d0c9ed6acac188a661b76"
Sep 05 06:51:48.691461 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:51:48.691433    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cab2cf8f70c51ffcb7b57b577227a07743a49612c19d0c9ed6acac188a661b76\": container with ID starting with cab2cf8f70c51ffcb7b57b577227a07743a49612c19d0c9ed6acac188a661b76 not found: ID does not exist" containerID="cab2cf8f70c51ffcb7b57b577227a07743a49612c19d0c9ed6acac188a661b76"
Sep 05 06:51:48.691644 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.691471    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cab2cf8f70c51ffcb7b57b577227a07743a49612c19d0c9ed6acac188a661b76"} err="failed to get container status \"cab2cf8f70c51ffcb7b57b577227a07743a49612c19d0c9ed6acac188a661b76\": rpc error: code = NotFound desc = could not find container \"cab2cf8f70c51ffcb7b57b577227a07743a49612c19d0c9ed6acac188a661b76\": container with ID starting with cab2cf8f70c51ffcb7b57b577227a07743a49612c19d0c9ed6acac188a661b76 not found: ID does not exist"
Sep 05 06:51:48.691644 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.691493    2559 scope.go:117] "RemoveContainer" containerID="3facc81ab65157e1b06a3000e799c2859c0536441614e4f33463621373718806"
Sep 05 06:51:48.692102 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:51:48.692052    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3facc81ab65157e1b06a3000e799c2859c0536441614e4f33463621373718806\": container with ID starting with 3facc81ab65157e1b06a3000e799c2859c0536441614e4f33463621373718806 not found: ID does not exist" containerID="3facc81ab65157e1b06a3000e799c2859c0536441614e4f33463621373718806"
Sep 05 06:51:48.692219 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.692117    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3facc81ab65157e1b06a3000e799c2859c0536441614e4f33463621373718806"} err="failed to get container status \"3facc81ab65157e1b06a3000e799c2859c0536441614e4f33463621373718806\": rpc error: code = NotFound desc = could not find container \"3facc81ab65157e1b06a3000e799c2859c0536441614e4f33463621373718806\": container with ID starting with 3facc81ab65157e1b06a3000e799c2859c0536441614e4f33463621373718806 not found: ID does not exist"
Sep 05 06:51:48.697443 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.697395    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"]
Sep 05 06:51:48.699902 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.699853    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-5e47e8-openshift-default-b4f87654d-zfpdk"]
Sep 05 06:51:48.772817 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.772785    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:51:48.800865 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.800832    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-dshm\") pod \"33c4d6de-7144-4d65-8764-0ebe903e4968\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") "
Sep 05 06:51:48.801053 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.800952    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33c4d6de-7144-4d65-8764-0ebe903e4968-tls-certs\") pod \"33c4d6de-7144-4d65-8764-0ebe903e4968\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") "
Sep 05 06:51:48.801053 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.801003    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvj5l\" (UniqueName: \"kubernetes.io/projected/33c4d6de-7144-4d65-8764-0ebe903e4968-kube-api-access-mvj5l\") pod \"33c4d6de-7144-4d65-8764-0ebe903e4968\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") "
Sep 05 06:51:48.801053 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.801039    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-home\") pod \"33c4d6de-7144-4d65-8764-0ebe903e4968\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") "
Sep 05 06:51:48.801236 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.801067    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-model-cache\") pod \"33c4d6de-7144-4d65-8764-0ebe903e4968\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") "
Sep 05 06:51:48.801236 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.801095    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-tmp-dir\") pod \"33c4d6de-7144-4d65-8764-0ebe903e4968\" (UID: \"33c4d6de-7144-4d65-8764-0ebe903e4968\") "
Sep 05 06:51:48.801412 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.801367    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-home" (OuterVolumeSpecName: "home") pod "33c4d6de-7144-4d65-8764-0ebe903e4968" (UID: "33c4d6de-7144-4d65-8764-0ebe903e4968"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:48.801471 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.801399    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-model-cache" (OuterVolumeSpecName: "model-cache") pod "33c4d6de-7144-4d65-8764-0ebe903e4968" (UID: "33c4d6de-7144-4d65-8764-0ebe903e4968"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:48.801471 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.801409    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "33c4d6de-7144-4d65-8764-0ebe903e4968" (UID: "33c4d6de-7144-4d65-8764-0ebe903e4968"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:48.803228 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.803183    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-dshm" (OuterVolumeSpecName: "dshm") pod "33c4d6de-7144-4d65-8764-0ebe903e4968" (UID: "33c4d6de-7144-4d65-8764-0ebe903e4968"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:51:48.803228 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.803210    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33c4d6de-7144-4d65-8764-0ebe903e4968-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "33c4d6de-7144-4d65-8764-0ebe903e4968" (UID: "33c4d6de-7144-4d65-8764-0ebe903e4968"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:51:48.803392 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.803291    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33c4d6de-7144-4d65-8764-0ebe903e4968-kube-api-access-mvj5l" (OuterVolumeSpecName: "kube-api-access-mvj5l") pod "33c4d6de-7144-4d65-8764-0ebe903e4968" (UID: "33c4d6de-7144-4d65-8764-0ebe903e4968"). InnerVolumeSpecName "kube-api-access-mvj5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:51:48.902213 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.902169    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-home\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:48.902213 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.902202    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-model-cache\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:48.902213 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.902215    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-tmp-dir\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:48.902213 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.902225    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33c4d6de-7144-4d65-8764-0ebe903e4968-dshm\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:48.902493 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.902237    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33c4d6de-7144-4d65-8764-0ebe903e4968-tls-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:48.902493 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:48.902248    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvj5l\" (UniqueName: \"kubernetes.io/projected/33c4d6de-7144-4d65-8764-0ebe903e4968-kube-api-access-mvj5l\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:51:49.668702 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:49.668669    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"
Sep 05 06:51:49.669308 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:49.668701    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v" event={"ID":"33c4d6de-7144-4d65-8764-0ebe903e4968","Type":"ContainerDied","Data":"c72c80e9c63764c7e6d73a1d6c64c186b8e83b9446758269adfdb37f93594fcf"}
Sep 05 06:51:49.669308 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:49.668757    2559 scope.go:117] "RemoveContainer" containerID="57b2723ebb1ad04c36fba0fc3aeccabc9aa095ed5d7204a0f319fc7837298e72"
Sep 05 06:51:49.691368 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:49.691344    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"]
Sep 05 06:51:49.695820 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:49.695796    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-39047b/rate-test-model-39047b-kserve-5d7cfccb7f-lt59v"]
Sep 05 06:51:49.892224 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:49.892191    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33c4d6de-7144-4d65-8764-0ebe903e4968" path="/var/lib/kubelet/pods/33c4d6de-7144-4d65-8764-0ebe903e4968/volumes"
Sep 05 06:51:49.892629 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:49.892615    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0fcf165-bddc-4260-b8e0-2cb4ffeba155" path="/var/lib/kubelet/pods/d0fcf165-bddc-4260-b8e0-2cb4ffeba155/volumes"
Sep 05 06:51:49.892997 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:49.892984    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7" path="/var/lib/kubelet/pods/e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7/volumes"
Sep 05 06:51:52.983738 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:51:52.983699    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:52:03.141354 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.141319    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"]
Sep 05 06:52:03.141937 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.141912    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0fcf165-bddc-4260-b8e0-2cb4ffeba155" containerName="istio-proxy"
Sep 05 06:52:03.141937 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.141939    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0fcf165-bddc-4260-b8e0-2cb4ffeba155" containerName="istio-proxy"
Sep 05 06:52:03.142102 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.141952    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" containerName="istio-proxy"
Sep 05 06:52:03.142102 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.141962    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" containerName="istio-proxy"
Sep 05 06:52:03.142102 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.141979    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" containerName="istio-proxy"
Sep 05 06:52:03.142102 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.141988    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" containerName="istio-proxy"
Sep 05 06:52:03.142102 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.142008    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e8e7b38-bf46-47f7-b65b-4ce31ad05581" containerName="istio-proxy"
Sep 05 06:52:03.142102 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.142016    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e8e7b38-bf46-47f7-b65b-4ce31ad05581" containerName="istio-proxy"
Sep 05 06:52:03.142102 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.142035    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e8e7b38-bf46-47f7-b65b-4ce31ad05581" containerName="istio-proxy"
Sep 05 06:52:03.142102 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.142043    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e8e7b38-bf46-47f7-b65b-4ce31ad05581" containerName="istio-proxy"
Sep 05 06:52:03.142102 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.142054    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7" containerName="main"
Sep 05 06:52:03.142102 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.142064    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7" containerName="main"
Sep 05 06:52:03.142102 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.142075    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33c4d6de-7144-4d65-8764-0ebe903e4968" containerName="main"
Sep 05 06:52:03.142102 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.142084    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="33c4d6de-7144-4d65-8764-0ebe903e4968" containerName="main"
Sep 05 06:52:03.142734 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.142198    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" containerName="istio-proxy"
Sep 05 06:52:03.142734 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.142211    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0fcf165-bddc-4260-b8e0-2cb4ffeba155" containerName="istio-proxy"
Sep 05 06:52:03.142734 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.142224    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e8e7b38-bf46-47f7-b65b-4ce31ad05581" containerName="istio-proxy"
Sep 05 06:52:03.142734 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.142232    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9b9c158-ad01-4a7b-a0b7-aac13f87b2a7" containerName="main"
Sep 05 06:52:03.142734 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.142240    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="33c4d6de-7144-4d65-8764-0ebe903e4968" containerName="main"
Sep 05 06:52:03.142734 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.142249    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0fcf165-bddc-4260-b8e0-2cb4ffeba155" containerName="istio-proxy"
Sep 05 06:52:03.142734 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.142257    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e8e7b38-bf46-47f7-b65b-4ce31ad05581" containerName="istio-proxy"
Sep 05 06:52:03.142734 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.142268    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4a22d30-2cb6-4b94-b1b7-4a75669dc91a" containerName="istio-proxy"
Sep 05 06:52:03.145725 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.145691    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.148492 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.148467    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 05 06:52:03.148492 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.148484    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-b49086-openshift-default-dockercfg-jln7d\""
Sep 05 06:52:03.157048 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.157024    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"]
Sep 05 06:52:03.218456 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.218428    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-token\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.218572 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.218465    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql5fl\" (UniqueName: \"kubernetes.io/projected/09a0b23e-33dc-4fca-8113-8d536381c9e3-kube-api-access-ql5fl\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.218572 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.218515    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-envoy\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.218572 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.218542    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-data\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.218679 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.218594    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09a0b23e-33dc-4fca-8113-8d536381c9e3-istiod-ca-cert\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.218679 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.218626    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-podinfo\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.218679 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.218660    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-workload-certs\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.218775 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.218688    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-credential-socket\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.218775 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.218728    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-workload-socket\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.319490 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.319465    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-workload-certs\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.319617 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.319503    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-credential-socket\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.319617 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.319522    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-workload-socket\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.319717 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.319636    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-token\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.319717 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.319692    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql5fl\" (UniqueName: \"kubernetes.io/projected/09a0b23e-33dc-4fca-8113-8d536381c9e3-kube-api-access-ql5fl\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.319800 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.319728    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-envoy\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.319800 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.319772    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-data\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.319930 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.319839    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-workload-socket\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.319990 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.319929    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09a0b23e-33dc-4fca-8113-8d536381c9e3-istiod-ca-cert\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.319990 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.319970    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-podinfo\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.319990 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.319931    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-credential-socket\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.320117 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.320007    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-workload-certs\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.320178 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.320161    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-data\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.320643 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.320619    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09a0b23e-33dc-4fca-8113-8d536381c9e3-istiod-ca-cert\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.321974 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.321955    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-envoy\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.322377 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.322355    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-podinfo\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.332088 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.332061    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-token\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.332797 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.332776    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql5fl\" (UniqueName: \"kubernetes.io/projected/09a0b23e-33dc-4fca-8113-8d536381c9e3-kube-api-access-ql5fl\") pod \"e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") " pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.459300 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.459255    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:03.741658 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.741619    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"]
Sep 05 06:52:03.743061 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:52:03.743035    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09a0b23e_33dc_4fca_8113_8d536381c9e3.slice/crio-c791a17201e9e2e08f199870319bd71e1b4ba7880e52d79d18891ba1181ee9a7 WatchSource:0}: Error finding container c791a17201e9e2e08f199870319bd71e1b4ba7880e52d79d18891ba1181ee9a7: Status 404 returned error can't find the container with id c791a17201e9e2e08f199870319bd71e1b4ba7880e52d79d18891ba1181ee9a7
Sep 05 06:52:03.745100 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.745067    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:52:03.745202 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.745142    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:52:03.745202 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.745189    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:52:03.752081 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:03.752059    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9" event={"ID":"09a0b23e-33dc-4fca-8113-8d536381c9e3","Type":"ContainerStarted","Data":"c791a17201e9e2e08f199870319bd71e1b4ba7880e52d79d18891ba1181ee9a7"}
Sep 05 06:52:04.759285 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:04.759249    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9" event={"ID":"09a0b23e-33dc-4fca-8113-8d536381c9e3","Type":"ContainerStarted","Data":"1dc6042a72779705ae676c75dd613febb03e67eec7e532607e0ab81d520f321d"}
Sep 05 06:52:04.782733 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:04.782683    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9" podStartSLOduration=1.7826677229999999 podStartE2EDuration="1.782667723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:52:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:52:04.781079814 +0000 UTC m=+1541.371638767" watchObservedRunningTime="2026-09-05 06:52:04.782667723 +0000 UTC m=+1541.373226674"
Sep 05 06:52:05.460323 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:05.460289    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:05.464976 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:05.464947    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:05.764845 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:05.764754    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:05.765697 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:05.765679    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:12.384312 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:12.384275    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:52:21.807129 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:21.807083    2559 patch_prober.go:28] interesting pod/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 05 06:52:21.808425 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:21.807164    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9" podUID="09a0b23e-33dc-4fca-8113-8d536381c9e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 05 06:52:22.877700 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:22.877657    2559 generic.go:356] "Generic (PLEG): container finished" podID="09a0b23e-33dc-4fca-8113-8d536381c9e3" containerID="1dc6042a72779705ae676c75dd613febb03e67eec7e532607e0ab81d520f321d" exitCode=0
Sep 05 06:52:22.878206 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:22.877727    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9" event={"ID":"09a0b23e-33dc-4fca-8113-8d536381c9e3","Type":"ContainerDied","Data":"1dc6042a72779705ae676c75dd613febb03e67eec7e532607e0ab81d520f321d"}
Sep 05 06:52:22.878206 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:22.878161    2559 scope.go:117] "RemoveContainer" containerID="1dc6042a72779705ae676c75dd613febb03e67eec7e532607e0ab81d520f321d"
Sep 05 06:52:22.879248 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:22.879217    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:52:22.879309 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:22.879296    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:52:22.879347 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:22.879330    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:52:23.885424 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:23.885386    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9" event={"ID":"09a0b23e-33dc-4fca-8113-8d536381c9e3","Type":"ContainerStarted","Data":"ea95b74047c7cba81b65a2254b2d4d68bc00e385fe40fd74d9029d2e31986d61"}
Sep 05 06:52:24.129246 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:24.129193    2559 scope.go:117] "RemoveContainer" containerID="632c38c5d308f849656d7d91ca40de006896083a6c6482eba0a2fc8cac1bb9f7"
Sep 05 06:52:24.147457 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:24.147319    2559 scope.go:117] "RemoveContainer" containerID="1edbab3a9638ee6abebf492029e08a5e28ad011425d22369dba2eb9959e44a8c"
Sep 05 06:52:24.460561 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:24.460517    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:24.460824 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:24.460801    2559 patch_prober.go:28] interesting pod/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 05 06:52:24.460910 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:24.460848    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9" podUID="09a0b23e-33dc-4fca-8113-8d536381c9e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 05 06:52:25.459872 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:25.459832    2559 patch_prober.go:28] interesting pod/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 05 06:52:25.460291 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:25.459922    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9" podUID="09a0b23e-33dc-4fca-8113-8d536381c9e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 05 06:52:26.459808 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:26.459768    2559 patch_prober.go:28] interesting pod/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 05 06:52:26.460016 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:26.459835    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9" podUID="09a0b23e-33dc-4fca-8113-8d536381c9e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 05 06:52:27.459664 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:27.459626    2559 patch_prober.go:28] interesting pod/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 05 06:52:27.459871 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:27.459700    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9" podUID="09a0b23e-33dc-4fca-8113-8d536381c9e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 05 06:52:28.459736 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:28.459685    2559 patch_prober.go:28] interesting pod/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Sep 05 06:52:28.460196 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:28.459754    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9" podUID="09a0b23e-33dc-4fca-8113-8d536381c9e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Sep 05 06:52:29.463136 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:29.463101    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:29.463537 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:29.463468    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:29.464265 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:29.464245    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:52:38.585689 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.585657    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"]
Sep 05 06:52:38.586092 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.586070    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0fcf165-bddc-4260-b8e0-2cb4ffeba155" containerName="istio-proxy"
Sep 05 06:52:38.586092 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.586085    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0fcf165-bddc-4260-b8e0-2cb4ffeba155" containerName="istio-proxy"
Sep 05 06:52:38.588573 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.588550    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.592027 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.592004    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-d36c73\"/\"ipp-route-d36c73-kserve-self-signed-certs\""
Sep 05 06:52:38.592177 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.592032    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-d36c73\"/\"default-dockercfg-pxq4w\""
Sep 05 06:52:38.592177 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.592037    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-d36c73\"/\"openshift-service-ca.crt\""
Sep 05 06:52:38.592177 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.592041    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-d36c73\"/\"kube-root-ca.crt\""
Sep 05 06:52:38.602050 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.602025    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"]
Sep 05 06:52:38.636531 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.636505    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-model-cache\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.636655 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.636536    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2dbf\" (UniqueName: \"kubernetes.io/projected/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-kube-api-access-x2dbf\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.636655 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.636567    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-dshm\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.636655 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.636617    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-home\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.636820 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.636652    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-tmp-dir\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.636820 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.636759    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-tls-certs\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.738096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.738055    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-model-cache\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.738244 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.738116    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2dbf\" (UniqueName: \"kubernetes.io/projected/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-kube-api-access-x2dbf\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.738244 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.738168    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-dshm\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.738244 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.738193    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-home\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.738244 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.738217    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-tmp-dir\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.738499 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.738309    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-tls-certs\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.738565 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.738502    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-model-cache\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.738611 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.738574    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-home\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.738699 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.738679    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-tmp-dir\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.740386 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.740360    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-dshm\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.740660 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.740643    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-tls-certs\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.747555 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.747519    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2dbf\" (UniqueName: \"kubernetes.io/projected/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-kube-api-access-x2dbf\") pod \"ipp-route-d36c73-kserve-67d9649579-jbjm2\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") " pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:38.898049 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:38.897972    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:39.171185 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:39.171155    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"]
Sep 05 06:52:39.172539 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:52:39.172515    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e8c8aaf_8a98_43d1_b35a_3e3ac0e15f53.slice/crio-cb66c6e9a8513964e312be5da1d39cea67caf69265618e7991d58acb8b716835 WatchSource:0}: Error finding container cb66c6e9a8513964e312be5da1d39cea67caf69265618e7991d58acb8b716835: Status 404 returned error can't find the container with id cb66c6e9a8513964e312be5da1d39cea67caf69265618e7991d58acb8b716835
Sep 05 06:52:39.974115 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:39.974080    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2" event={"ID":"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53","Type":"ContainerStarted","Data":"82375ef79f1f5879151913f63cb4266114007b4e1231481eec7561ac1c29d7ed"}
Sep 05 06:52:39.974554 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:39.974122    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2" event={"ID":"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53","Type":"ContainerStarted","Data":"cb66c6e9a8513964e312be5da1d39cea67caf69265618e7991d58acb8b716835"}
Sep 05 06:52:39.992551 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:39.992495    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2" podStartSLOduration=1.992482387 podStartE2EDuration="1.992482387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:52:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:52:39.991977045 +0000 UTC m=+1576.582535998" watchObservedRunningTime="2026-09-05 06:52:39.992482387 +0000 UTC m=+1576.583041339"
Sep 05 06:52:43.800349 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:43.800311    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:52:48.898715 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:48.898682    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:48.899110 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:48.898751    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:48.911281 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:48.911257    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:52:49.038019 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:52:49.037987    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:53:04.387004 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:04.386965    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:53:12.409970 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.409930    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"]
Sep 05 06:53:12.413489 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.413470    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.416293 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.416270    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-9db0f4-openshift-default-dockercfg-4x9bw\""
Sep 05 06:53:12.425320 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.425294    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"]
Sep 05 06:53:12.541233 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.541195    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdrcj\" (UniqueName: \"kubernetes.io/projected/5c857f97-082a-43da-8fb2-9761a8b2541e-kube-api-access-rdrcj\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.541412 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.541248    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-credential-socket\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.541412 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.541283    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-workload-certs\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.541412 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.541339    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-workload-socket\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.541412 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.541355    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-envoy\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.541412 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.541380    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-data\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.541591 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.541426    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-token\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.541591 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.541476    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c857f97-082a-43da-8fb2-9761a8b2541e-istiod-ca-cert\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.541591 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.541501    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-podinfo\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.642022 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.641986    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-credential-socket\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.642210 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.642029    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-workload-certs\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.642210 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.642083    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-workload-socket\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.642210 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.642105    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-envoy\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.642210 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.642128    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-data\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.642210 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.642179    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-token\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.642514 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.642247    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c857f97-082a-43da-8fb2-9761a8b2541e-istiod-ca-cert\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.642514 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.642287    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-podinfo\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.642514 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.642341    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdrcj\" (UniqueName: \"kubernetes.io/projected/5c857f97-082a-43da-8fb2-9761a8b2541e-kube-api-access-rdrcj\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.642514 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.642440    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-credential-socket\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.642687 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.642509    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-workload-certs\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.642687 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.642530    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-data\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.642807 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.642744    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-workload-socket\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.643007 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.642986    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c857f97-082a-43da-8fb2-9761a8b2541e-istiod-ca-cert\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.644499 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.644479    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-envoy\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.644725 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.644707    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-podinfo\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.654027 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.654001    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-token\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.654457 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.654437    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdrcj\" (UniqueName: \"kubernetes.io/projected/5c857f97-082a-43da-8fb2-9761a8b2541e-kube-api-access-rdrcj\") pod \"e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") " pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:12.725180 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:12.725149    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:13.006136 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:13.006111    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"]
Sep 05 06:53:13.007932 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:53:13.007902    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c857f97_082a_43da_8fb2_9761a8b2541e.slice/crio-3023adb5b4e2a155b103609636bfdbc7a48c1f318a0f9d2da77bd18298ebf419 WatchSource:0}: Error finding container 3023adb5b4e2a155b103609636bfdbc7a48c1f318a0f9d2da77bd18298ebf419: Status 404 returned error can't find the container with id 3023adb5b4e2a155b103609636bfdbc7a48c1f318a0f9d2da77bd18298ebf419
Sep 05 06:53:13.010082 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:13.010053    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:53:13.010161 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:13.010113    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:53:13.010161 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:13.010146    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:53:13.176110 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:13.176067    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd" event={"ID":"5c857f97-082a-43da-8fb2-9761a8b2541e","Type":"ContainerStarted","Data":"ae5c66a8a839e1e4c66188dc496cdae72cdc392eb99fb18611f8c061a3d71dbb"}
Sep 05 06:53:13.176297 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:13.176125    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd" event={"ID":"5c857f97-082a-43da-8fb2-9761a8b2541e","Type":"ContainerStarted","Data":"3023adb5b4e2a155b103609636bfdbc7a48c1f318a0f9d2da77bd18298ebf419"}
Sep 05 06:53:13.199822 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:13.199765    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd" podStartSLOduration=1.199749865 podStartE2EDuration="1.199749865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:53:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:53:13.197194486 +0000 UTC m=+1609.787753438" watchObservedRunningTime="2026-09-05 06:53:13.199749865 +0000 UTC m=+1609.790308818"
Sep 05 06:53:13.725514 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:13.725477    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:14.731424 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:14.731388    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:15.190667 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:15.190632    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:15.193987 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:15.193955    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:25.227116 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:25.227034    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:53:40.835162 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:40.835131    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f"]
Sep 05 06:53:40.837974 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:40.837955    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f"
Sep 05 06:53:40.840781 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:40.840758    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slfqj\""
Sep 05 06:53:40.840919 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:40.840785    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 06:53:40.850423 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:40.850401    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f"]
Sep 05 06:53:40.888394 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:40.888361    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f\" (UID: \"3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f"
Sep 05 06:53:40.989284 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:40.989253    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f\" (UID: \"3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f"
Sep 05 06:53:40.989868 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:40.989846    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f\" (UID: \"3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f"
Sep 05 06:53:41.152022 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:41.151954    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f"
Sep 05 06:53:41.346332 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:41.346296    2559 generic.go:356] "Generic (PLEG): container finished" podID="5c857f97-082a-43da-8fb2-9761a8b2541e" containerID="ae5c66a8a839e1e4c66188dc496cdae72cdc392eb99fb18611f8c061a3d71dbb" exitCode=0
Sep 05 06:53:41.346501 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:41.346366    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd" event={"ID":"5c857f97-082a-43da-8fb2-9761a8b2541e","Type":"ContainerDied","Data":"ae5c66a8a839e1e4c66188dc496cdae72cdc392eb99fb18611f8c061a3d71dbb"}
Sep 05 06:53:41.346783 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:41.346769    2559 scope.go:117] "RemoveContainer" containerID="ae5c66a8a839e1e4c66188dc496cdae72cdc392eb99fb18611f8c061a3d71dbb"
Sep 05 06:53:41.347894 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:41.347854    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:53:41.347965 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:41.347946    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:53:41.348003 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:41.347982    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 05 06:53:41.428381 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:41.428348    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f"]
Sep 05 06:53:41.429492 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:53:41.429465    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c53d99f_08f8_49aa_be8d_0fdac3fe8bd8.slice/crio-8a0d2bfe63e77c53483e72aba5b7704ccc6e1203b2179c39d592fa6e0fa77283 WatchSource:0}: Error finding container 8a0d2bfe63e77c53483e72aba5b7704ccc6e1203b2179c39d592fa6e0fa77283: Status 404 returned error can't find the container with id 8a0d2bfe63e77c53483e72aba5b7704ccc6e1203b2179c39d592fa6e0fa77283
Sep 05 06:53:42.355415 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:42.355381    2559 generic.go:356] "Generic (PLEG): container finished" podID="3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8" containerID="b285a524df86b671faebd05296b4b834a343a03d03fa5a68b7197f98009030c2" exitCode=0
Sep 05 06:53:42.355920 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:42.355476    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f" event={"ID":"3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8","Type":"ContainerDied","Data":"b285a524df86b671faebd05296b4b834a343a03d03fa5a68b7197f98009030c2"}
Sep 05 06:53:42.355920 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:42.355544    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f" event={"ID":"3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8","Type":"ContainerStarted","Data":"8a0d2bfe63e77c53483e72aba5b7704ccc6e1203b2179c39d592fa6e0fa77283"}
Sep 05 06:53:42.357520 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:42.357493    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd" event={"ID":"5c857f97-082a-43da-8fb2-9761a8b2541e","Type":"ContainerStarted","Data":"080143dc01ff30e2e4cdb9d2556e24877cbdc38e0f4dfc66ac4fa4a9f363061c"}
Sep 05 06:53:42.726006 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:42.725901    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:42.727180 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:42.727149    2559 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Sep 05 06:53:42.727416 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:42.727385    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd" podUID="5c857f97-082a-43da-8fb2-9761a8b2541e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Sep 05 06:53:43.502052 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:43.502024    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f"
Sep 05 06:53:43.612045 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:43.612013    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8-maas-api-service-ca\") pod \"3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8\" (UID: \"3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8\") "
Sep 05 06:53:43.612338 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:43.612317    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8" (UID: "3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:53:43.713604 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:43.713571    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8-maas-api-service-ca\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:53:43.725695 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:43.725661    2559 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Sep 05 06:53:43.725815 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:43.725721    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd" podUID="5c857f97-082a-43da-8fb2-9761a8b2541e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Sep 05 06:53:44.370289 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:44.370258    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f"
Sep 05 06:53:44.370466 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:44.370265    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f" event={"ID":"3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8","Type":"ContainerDied","Data":"8a0d2bfe63e77c53483e72aba5b7704ccc6e1203b2179c39d592fa6e0fa77283"}
Sep 05 06:53:44.370466 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:44.370407    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a0d2bfe63e77c53483e72aba5b7704ccc6e1203b2179c39d592fa6e0fa77283"
Sep 05 06:53:44.725697 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:44.725662    2559 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Sep 05 06:53:44.726282 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:44.726251    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd" podUID="5c857f97-082a-43da-8fb2-9761a8b2541e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Sep 05 06:53:45.190738 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:45.190700    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:45.726077 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:45.726035    2559 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Sep 05 06:53:45.726712 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:45.726675    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd" podUID="5c857f97-082a-43da-8fb2-9761a8b2541e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Sep 05 06:53:46.725641 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:46.725602    2559 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Sep 05 06:53:46.725983 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:46.725949    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd" podUID="5c857f97-082a-43da-8fb2-9761a8b2541e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Sep 05 06:53:47.332815 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:47.332770    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f"]
Sep 05 06:53:47.336942 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:47.336896    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9db0f4-mfq7f"]
Sep 05 06:53:47.731096 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:47.731060    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:47.732372 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:47.732348    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:47.892689 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:47.892650    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8" path="/var/lib/kubelet/pods/3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8/volumes"
Sep 05 06:53:52.789569 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:52.789526    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:53:54.328056 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:54.328020    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"]
Sep 05 06:53:54.328505 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:54.328282    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd" podUID="5c857f97-082a-43da-8fb2-9761a8b2541e" containerName="istio-proxy" containerID="cri-o://080143dc01ff30e2e4cdb9d2556e24877cbdc38e0f4dfc66ac4fa4a9f363061c" gracePeriod=30
Sep 05 06:53:59.484798 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.484756    2559 generic.go:356] "Generic (PLEG): container finished" podID="5c857f97-082a-43da-8fb2-9761a8b2541e" containerID="080143dc01ff30e2e4cdb9d2556e24877cbdc38e0f4dfc66ac4fa4a9f363061c" exitCode=0
Sep 05 06:53:59.485232 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.484816    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd" event={"ID":"5c857f97-082a-43da-8fb2-9761a8b2541e","Type":"ContainerDied","Data":"080143dc01ff30e2e4cdb9d2556e24877cbdc38e0f4dfc66ac4fa4a9f363061c"}
Sep 05 06:53:59.485232 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.484916    2559 scope.go:117] "RemoveContainer" containerID="ae5c66a8a839e1e4c66188dc496cdae72cdc392eb99fb18611f8c061a3d71dbb"
Sep 05 06:53:59.589266 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.589209    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:53:59.682393 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.682367    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-data\") pod \"5c857f97-082a-43da-8fb2-9761a8b2541e\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") "
Sep 05 06:53:59.682600 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.682425    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-podinfo\") pod \"5c857f97-082a-43da-8fb2-9761a8b2541e\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") "
Sep 05 06:53:59.682600 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.682466    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-workload-socket\") pod \"5c857f97-082a-43da-8fb2-9761a8b2541e\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") "
Sep 05 06:53:59.682600 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.682496    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-workload-certs\") pod \"5c857f97-082a-43da-8fb2-9761a8b2541e\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") "
Sep 05 06:53:59.682600 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.682522    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c857f97-082a-43da-8fb2-9761a8b2541e-istiod-ca-cert\") pod \"5c857f97-082a-43da-8fb2-9761a8b2541e\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") "
Sep 05 06:53:59.682600 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.682563    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-credential-socket\") pod \"5c857f97-082a-43da-8fb2-9761a8b2541e\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") "
Sep 05 06:53:59.682941 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.682653    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-data" (OuterVolumeSpecName: "istio-data") pod "5c857f97-082a-43da-8fb2-9761a8b2541e" (UID: "5c857f97-082a-43da-8fb2-9761a8b2541e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:53:59.682941 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.682894    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5c857f97-082a-43da-8fb2-9761a8b2541e" (UID: "5c857f97-082a-43da-8fb2-9761a8b2541e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:53:59.682941 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.682900    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5c857f97-082a-43da-8fb2-9761a8b2541e" (UID: "5c857f97-082a-43da-8fb2-9761a8b2541e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:53:59.682941 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.682925    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5c857f97-082a-43da-8fb2-9761a8b2541e" (UID: "5c857f97-082a-43da-8fb2-9761a8b2541e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:53:59.683130 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.682993    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c857f97-082a-43da-8fb2-9761a8b2541e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5c857f97-082a-43da-8fb2-9761a8b2541e" (UID: "5c857f97-082a-43da-8fb2-9761a8b2541e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:53:59.683130 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.683078    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-token\") pod \"5c857f97-082a-43da-8fb2-9761a8b2541e\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") "
Sep 05 06:53:59.683130 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.683103    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-envoy\") pod \"5c857f97-082a-43da-8fb2-9761a8b2541e\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") "
Sep 05 06:53:59.683263 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.683133    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdrcj\" (UniqueName: \"kubernetes.io/projected/5c857f97-082a-43da-8fb2-9761a8b2541e-kube-api-access-rdrcj\") pod \"5c857f97-082a-43da-8fb2-9761a8b2541e\" (UID: \"5c857f97-082a-43da-8fb2-9761a8b2541e\") "
Sep 05 06:53:59.683397 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.683380    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-data\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:53:59.683397 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.683398    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-workload-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:53:59.683546 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.683411    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-workload-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:53:59.683546 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.683424    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c857f97-082a-43da-8fb2-9761a8b2541e-istiod-ca-cert\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:53:59.683546 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.683435    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-credential-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:53:59.684659 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.684629    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5c857f97-082a-43da-8fb2-9761a8b2541e" (UID: "5c857f97-082a-43da-8fb2-9761a8b2541e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 06:53:59.685386 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.685361    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5c857f97-082a-43da-8fb2-9761a8b2541e" (UID: "5c857f97-082a-43da-8fb2-9761a8b2541e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:53:59.685574 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.685556    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c857f97-082a-43da-8fb2-9761a8b2541e-kube-api-access-rdrcj" (OuterVolumeSpecName: "kube-api-access-rdrcj") pod "5c857f97-082a-43da-8fb2-9761a8b2541e" (UID: "5c857f97-082a-43da-8fb2-9761a8b2541e"). InnerVolumeSpecName "kube-api-access-rdrcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:53:59.685619 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.685572    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-token" (OuterVolumeSpecName: "istio-token") pod "5c857f97-082a-43da-8fb2-9761a8b2541e" (UID: "5c857f97-082a-43da-8fb2-9761a8b2541e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:53:59.784709 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.784680    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-podinfo\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:53:59.784709 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.784704    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-token\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:53:59.784709 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.784714    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c857f97-082a-43da-8fb2-9761a8b2541e-istio-envoy\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:53:59.784879 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:53:59.784722    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdrcj\" (UniqueName: \"kubernetes.io/projected/5c857f97-082a-43da-8fb2-9761a8b2541e-kube-api-access-rdrcj\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:54:00.492437 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:00.492407    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"
Sep 05 06:54:00.492853 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:00.492457    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd" event={"ID":"5c857f97-082a-43da-8fb2-9761a8b2541e","Type":"ContainerDied","Data":"3023adb5b4e2a155b103609636bfdbc7a48c1f318a0f9d2da77bd18298ebf419"}
Sep 05 06:54:00.492853 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:00.492527    2559 scope.go:117] "RemoveContainer" containerID="080143dc01ff30e2e4cdb9d2556e24877cbdc38e0f4dfc66ac4fa4a9f363061c"
Sep 05 06:54:00.518650 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:00.518619    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"]
Sep 05 06:54:00.520007 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:00.519825    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-9db0f4-openshift-default-b58c95dc8-cfgqd"]
Sep 05 06:54:01.892223 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:01.892188    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c857f97-082a-43da-8fb2-9761a8b2541e" path="/var/lib/kubelet/pods/5c857f97-082a-43da-8fb2-9761a8b2541e/volumes"
Sep 05 06:54:15.682877 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:15.682837    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:54:19.838877 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:19.838840    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"]
Sep 05 06:54:25.984900 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:25.984842    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:54:34.858589 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:34.858526    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2" podUID="7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53" containerName="main" containerID="cri-o://82375ef79f1f5879151913f63cb4266114007b4e1231481eec7561ac1c29d7ed" gracePeriod=45
Sep 05 06:54:35.101503 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.101480    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:54:35.172784 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.172713    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-tmp-dir\") pod \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") "
Sep 05 06:54:35.172784 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.172766    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-dshm\") pod \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") "
Sep 05 06:54:35.173018 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.172807    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-model-cache\") pod \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") "
Sep 05 06:54:35.173018 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.172849    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-home\") pod \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") "
Sep 05 06:54:35.173018 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.172916    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-tls-certs\") pod \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") "
Sep 05 06:54:35.173018 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.172952    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53" (UID: "7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:54:35.173018 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.173000    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2dbf\" (UniqueName: \"kubernetes.io/projected/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-kube-api-access-x2dbf\") pod \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\" (UID: \"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53\") "
Sep 05 06:54:35.173273 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.173141    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-home" (OuterVolumeSpecName: "home") pod "7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53" (UID: "7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:54:35.173273 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.173149    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-model-cache" (OuterVolumeSpecName: "model-cache") pod "7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53" (UID: "7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:54:35.173362 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.173342    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-tmp-dir\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:54:35.173406 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.173368    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-model-cache\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:54:35.173406 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.173382    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-home\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:54:35.175181 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.175153    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53" (UID: "7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 06:54:35.175292 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.175158    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-dshm" (OuterVolumeSpecName: "dshm") pod "7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53" (UID: "7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:54:35.175292 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.175262    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-kube-api-access-x2dbf" (OuterVolumeSpecName: "kube-api-access-x2dbf") pod "7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53" (UID: "7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53"). InnerVolumeSpecName "kube-api-access-x2dbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:54:35.274241 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.274218    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-tls-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:54:35.274241 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.274238    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2dbf\" (UniqueName: \"kubernetes.io/projected/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-kube-api-access-x2dbf\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:54:35.274384 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.274248    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53-dshm\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:54:35.710327 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.710288    2559 generic.go:356] "Generic (PLEG): container finished" podID="7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53" containerID="82375ef79f1f5879151913f63cb4266114007b4e1231481eec7561ac1c29d7ed" exitCode=0
Sep 05 06:54:35.710491 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.710339    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"
Sep 05 06:54:35.710491 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.710378    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2" event={"ID":"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53","Type":"ContainerDied","Data":"82375ef79f1f5879151913f63cb4266114007b4e1231481eec7561ac1c29d7ed"}
Sep 05 06:54:35.710491 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.710431    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2" event={"ID":"7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53","Type":"ContainerDied","Data":"cb66c6e9a8513964e312be5da1d39cea67caf69265618e7991d58acb8b716835"}
Sep 05 06:54:35.710491 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.710454    2559 scope.go:117] "RemoveContainer" containerID="82375ef79f1f5879151913f63cb4266114007b4e1231481eec7561ac1c29d7ed"
Sep 05 06:54:35.720714 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.720695    2559 scope.go:117] "RemoveContainer" containerID="82375ef79f1f5879151913f63cb4266114007b4e1231481eec7561ac1c29d7ed"
Sep 05 06:54:35.721020 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:54:35.721000    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82375ef79f1f5879151913f63cb4266114007b4e1231481eec7561ac1c29d7ed\": container with ID starting with 82375ef79f1f5879151913f63cb4266114007b4e1231481eec7561ac1c29d7ed not found: ID does not exist" containerID="82375ef79f1f5879151913f63cb4266114007b4e1231481eec7561ac1c29d7ed"
Sep 05 06:54:35.721079 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.721030    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82375ef79f1f5879151913f63cb4266114007b4e1231481eec7561ac1c29d7ed"} err="failed to get container status \"82375ef79f1f5879151913f63cb4266114007b4e1231481eec7561ac1c29d7ed\": rpc error: code = NotFound desc = could not find container \"82375ef79f1f5879151913f63cb4266114007b4e1231481eec7561ac1c29d7ed\": container with ID starting with 82375ef79f1f5879151913f63cb4266114007b4e1231481eec7561ac1c29d7ed not found: ID does not exist"
Sep 05 06:54:35.733220 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.733198    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"]
Sep 05 06:54:35.738463 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.738437    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-d36c73/ipp-route-d36c73-kserve-67d9649579-jbjm2"]
Sep 05 06:54:35.892457 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:54:35.892423    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53" path="/var/lib/kubelet/pods/7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53/volumes"
Sep 05 06:55:03.840325 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:03.840291    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z"]
Sep 05 06:55:03.840738 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:03.840655    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8" containerName="revoke-keys"
Sep 05 06:55:03.840738 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:03.840667    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8" containerName="revoke-keys"
Sep 05 06:55:03.840738 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:03.840679    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53" containerName="main"
Sep 05 06:55:03.840738 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:03.840685    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53" containerName="main"
Sep 05 06:55:03.840738 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:03.840696    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c857f97-082a-43da-8fb2-9761a8b2541e" containerName="istio-proxy"
Sep 05 06:55:03.840738 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:03.840701    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c857f97-082a-43da-8fb2-9761a8b2541e" containerName="istio-proxy"
Sep 05 06:55:03.840738 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:03.840720    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c857f97-082a-43da-8fb2-9761a8b2541e" containerName="istio-proxy"
Sep 05 06:55:03.840738 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:03.840726    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c857f97-082a-43da-8fb2-9761a8b2541e" containerName="istio-proxy"
Sep 05 06:55:03.841045 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:03.840775    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e8c8aaf-8a98-43d1-b35a-3e3ac0e15f53" containerName="main"
Sep 05 06:55:03.841045 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:03.840785    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c53d99f-08f8-49aa-be8d-0fdac3fe8bd8" containerName="revoke-keys"
Sep 05 06:55:03.841045 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:03.840791    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c857f97-082a-43da-8fb2-9761a8b2541e" containerName="istio-proxy"
Sep 05 06:55:03.841045 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:03.840799    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c857f97-082a-43da-8fb2-9761a8b2541e" containerName="istio-proxy"
Sep 05 06:55:03.843833 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:03.843813    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z"
Sep 05 06:55:03.846319 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:03.846294    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 05 06:55:03.846453 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:03.846325    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-slfqj\""
Sep 05 06:55:03.851113 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:03.851088    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z"]
Sep 05 06:55:04.014221 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:04.014190    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b343826c-fc4d-4f44-954c-ef0e9dbf1a94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z\" (UID: \"b343826c-fc4d-4f44-954c-ef0e9dbf1a94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z"
Sep 05 06:55:04.114653 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:04.114566    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b343826c-fc4d-4f44-954c-ef0e9dbf1a94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z\" (UID: \"b343826c-fc4d-4f44-954c-ef0e9dbf1a94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z"
Sep 05 06:55:04.115195 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:04.115175    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b343826c-fc4d-4f44-954c-ef0e9dbf1a94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z\" (UID: \"b343826c-fc4d-4f44-954c-ef0e9dbf1a94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z"
Sep 05 06:55:04.156657 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:04.156627    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z"
Sep 05 06:55:04.434429 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:04.434404    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z"]
Sep 05 06:55:04.435990 ip-10-0-130-170 kubenswrapper[2559]: W0905 06:55:04.435955    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb343826c_fc4d_4f44_954c_ef0e9dbf1a94.slice/crio-aa1e3b6227e66a2a3f7fe11db5b13bf5e2386b922e3e90497c822a4d76de31da WatchSource:0}: Error finding container aa1e3b6227e66a2a3f7fe11db5b13bf5e2386b922e3e90497c822a4d76de31da: Status 404 returned error can't find the container with id aa1e3b6227e66a2a3f7fe11db5b13bf5e2386b922e3e90497c822a4d76de31da
Sep 05 06:55:04.889856 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:04.889825    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z" event={"ID":"b343826c-fc4d-4f44-954c-ef0e9dbf1a94","Type":"ContainerStarted","Data":"e2b27ca31847de53bd00a9802d082bd797e532d2267dad38e402ec9a9c0ffe42"}
Sep 05 06:55:04.890268 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:04.889869    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z" event={"ID":"b343826c-fc4d-4f44-954c-ef0e9dbf1a94","Type":"ContainerStarted","Data":"aa1e3b6227e66a2a3f7fe11db5b13bf5e2386b922e3e90497c822a4d76de31da"}
Sep 05 06:55:04.908260 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:04.908213    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z" podStartSLOduration=1.90819475 podStartE2EDuration="1.90819475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 06:55:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 06:55:04.906673938 +0000 UTC m=+1721.497232890" watchObservedRunningTime="2026-09-05 06:55:04.90819475 +0000 UTC m=+1721.498753703"
Sep 05 06:55:05.896334 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:05.896301    2559 generic.go:356] "Generic (PLEG): container finished" podID="b343826c-fc4d-4f44-954c-ef0e9dbf1a94" containerID="e2b27ca31847de53bd00a9802d082bd797e532d2267dad38e402ec9a9c0ffe42" exitCode=0
Sep 05 06:55:05.896721 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:05.896387    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z" event={"ID":"b343826c-fc4d-4f44-954c-ef0e9dbf1a94","Type":"ContainerDied","Data":"e2b27ca31847de53bd00a9802d082bd797e532d2267dad38e402ec9a9c0ffe42"}
Sep 05 06:55:07.030422 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:07.030389    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z"
Sep 05 06:55:07.141270 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:07.141239    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b343826c-fc4d-4f44-954c-ef0e9dbf1a94-maas-api-service-ca\") pod \"b343826c-fc4d-4f44-954c-ef0e9dbf1a94\" (UID: \"b343826c-fc4d-4f44-954c-ef0e9dbf1a94\") "
Sep 05 06:55:07.141569 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:07.141546    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b343826c-fc4d-4f44-954c-ef0e9dbf1a94-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b343826c-fc4d-4f44-954c-ef0e9dbf1a94" (UID: "b343826c-fc4d-4f44-954c-ef0e9dbf1a94"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:55:07.241958 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:07.241930    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b343826c-fc4d-4f44-954c-ef0e9dbf1a94-maas-api-service-ca\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:55:07.909232 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:07.909194    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z" event={"ID":"b343826c-fc4d-4f44-954c-ef0e9dbf1a94","Type":"ContainerDied","Data":"aa1e3b6227e66a2a3f7fe11db5b13bf5e2386b922e3e90497c822a4d76de31da"}
Sep 05 06:55:07.909399 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:07.909255    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa1e3b6227e66a2a3f7fe11db5b13bf5e2386b922e3e90497c822a4d76de31da"
Sep 05 06:55:07.909399 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:07.909208    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z"
Sep 05 06:55:14.743601 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:14.743556    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z"]
Sep 05 06:55:14.745836 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:14.745812    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b49086-t8q7z"]
Sep 05 06:55:15.891863 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:15.891830    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b343826c-fc4d-4f44-954c-ef0e9dbf1a94" path="/var/lib/kubelet/pods/b343826c-fc4d-4f44-954c-ef0e9dbf1a94/volumes"
Sep 05 06:55:18.190469 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:18.190432    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:55:21.657065 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:21.657028    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"]
Sep 05 06:55:21.657501 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:21.657272    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9" podUID="09a0b23e-33dc-4fca-8113-8d536381c9e3" containerName="istio-proxy" containerID="cri-o://ea95b74047c7cba81b65a2254b2d4d68bc00e385fe40fd74d9029d2e31986d61" gracePeriod=30
Sep 05 06:55:26.897982 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.897955    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:55:26.905526 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.905498    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-credential-socket\") pod \"09a0b23e-33dc-4fca-8113-8d536381c9e3\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") "
Sep 05 06:55:26.905625 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.905538    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-token\") pod \"09a0b23e-33dc-4fca-8113-8d536381c9e3\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") "
Sep 05 06:55:26.905625 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.905556    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-workload-socket\") pod \"09a0b23e-33dc-4fca-8113-8d536381c9e3\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") "
Sep 05 06:55:26.905625 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.905585    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-data\") pod \"09a0b23e-33dc-4fca-8113-8d536381c9e3\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") "
Sep 05 06:55:26.905625 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.905601    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql5fl\" (UniqueName: \"kubernetes.io/projected/09a0b23e-33dc-4fca-8113-8d536381c9e3-kube-api-access-ql5fl\") pod \"09a0b23e-33dc-4fca-8113-8d536381c9e3\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") "
Sep 05 06:55:26.905625 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.905618    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-envoy\") pod \"09a0b23e-33dc-4fca-8113-8d536381c9e3\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") "
Sep 05 06:55:26.905898 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.905645    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09a0b23e-33dc-4fca-8113-8d536381c9e3-istiod-ca-cert\") pod \"09a0b23e-33dc-4fca-8113-8d536381c9e3\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") "
Sep 05 06:55:26.905898 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.905702    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-podinfo\") pod \"09a0b23e-33dc-4fca-8113-8d536381c9e3\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") "
Sep 05 06:55:26.905898 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.905745    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-workload-certs\") pod \"09a0b23e-33dc-4fca-8113-8d536381c9e3\" (UID: \"09a0b23e-33dc-4fca-8113-8d536381c9e3\") "
Sep 05 06:55:26.906132 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.905944    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-data" (OuterVolumeSpecName: "istio-data") pod "09a0b23e-33dc-4fca-8113-8d536381c9e3" (UID: "09a0b23e-33dc-4fca-8113-8d536381c9e3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:55:26.906132 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.905966    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "09a0b23e-33dc-4fca-8113-8d536381c9e3" (UID: "09a0b23e-33dc-4fca-8113-8d536381c9e3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:55:26.906245 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.906193    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "09a0b23e-33dc-4fca-8113-8d536381c9e3" (UID: "09a0b23e-33dc-4fca-8113-8d536381c9e3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:55:26.906245 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.906209    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09a0b23e-33dc-4fca-8113-8d536381c9e3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "09a0b23e-33dc-4fca-8113-8d536381c9e3" (UID: "09a0b23e-33dc-4fca-8113-8d536381c9e3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 05 06:55:26.906358 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.906282    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "09a0b23e-33dc-4fca-8113-8d536381c9e3" (UID: "09a0b23e-33dc-4fca-8113-8d536381c9e3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:55:26.907763 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.907720    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "09a0b23e-33dc-4fca-8113-8d536381c9e3" (UID: "09a0b23e-33dc-4fca-8113-8d536381c9e3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:55:26.907914 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.907817    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-token" (OuterVolumeSpecName: "istio-token") pod "09a0b23e-33dc-4fca-8113-8d536381c9e3" (UID: "09a0b23e-33dc-4fca-8113-8d536381c9e3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:55:26.907983 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.907939    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "09a0b23e-33dc-4fca-8113-8d536381c9e3" (UID: "09a0b23e-33dc-4fca-8113-8d536381c9e3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 05 06:55:26.908054 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:26.908000    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09a0b23e-33dc-4fca-8113-8d536381c9e3-kube-api-access-ql5fl" (OuterVolumeSpecName: "kube-api-access-ql5fl") pod "09a0b23e-33dc-4fca-8113-8d536381c9e3" (UID: "09a0b23e-33dc-4fca-8113-8d536381c9e3"). InnerVolumeSpecName "kube-api-access-ql5fl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:55:27.007077 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.007054    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-podinfo\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:55:27.007077 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.007074    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-workload-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:55:27.007237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.007084    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-credential-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:55:27.007237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.007092    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-token\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:55:27.007237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.007101    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-workload-socket\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:55:27.007237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.007110    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-data\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:55:27.007237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.007118    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql5fl\" (UniqueName: \"kubernetes.io/projected/09a0b23e-33dc-4fca-8113-8d536381c9e3-kube-api-access-ql5fl\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:55:27.007237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.007126    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/09a0b23e-33dc-4fca-8113-8d536381c9e3-istio-envoy\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:55:27.007237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.007134    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/09a0b23e-33dc-4fca-8113-8d536381c9e3-istiod-ca-cert\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:55:27.020626 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.020596    2559 generic.go:356] "Generic (PLEG): container finished" podID="09a0b23e-33dc-4fca-8113-8d536381c9e3" containerID="ea95b74047c7cba81b65a2254b2d4d68bc00e385fe40fd74d9029d2e31986d61" exitCode=0
Sep 05 06:55:27.020747 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.020643    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"
Sep 05 06:55:27.020747 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.020666    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9" event={"ID":"09a0b23e-33dc-4fca-8113-8d536381c9e3","Type":"ContainerDied","Data":"ea95b74047c7cba81b65a2254b2d4d68bc00e385fe40fd74d9029d2e31986d61"}
Sep 05 06:55:27.020747 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.020724    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9" event={"ID":"09a0b23e-33dc-4fca-8113-8d536381c9e3","Type":"ContainerDied","Data":"c791a17201e9e2e08f199870319bd71e1b4ba7880e52d79d18891ba1181ee9a7"}
Sep 05 06:55:27.020927 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.020755    2559 scope.go:117] "RemoveContainer" containerID="ea95b74047c7cba81b65a2254b2d4d68bc00e385fe40fd74d9029d2e31986d61"
Sep 05 06:55:27.030137 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.030122    2559 scope.go:117] "RemoveContainer" containerID="1dc6042a72779705ae676c75dd613febb03e67eec7e532607e0ab81d520f321d"
Sep 05 06:55:27.038462 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.038442    2559 scope.go:117] "RemoveContainer" containerID="ea95b74047c7cba81b65a2254b2d4d68bc00e385fe40fd74d9029d2e31986d61"
Sep 05 06:55:27.038723 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:55:27.038705    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea95b74047c7cba81b65a2254b2d4d68bc00e385fe40fd74d9029d2e31986d61\": container with ID starting with ea95b74047c7cba81b65a2254b2d4d68bc00e385fe40fd74d9029d2e31986d61 not found: ID does not exist" containerID="ea95b74047c7cba81b65a2254b2d4d68bc00e385fe40fd74d9029d2e31986d61"
Sep 05 06:55:27.038793 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.038734    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea95b74047c7cba81b65a2254b2d4d68bc00e385fe40fd74d9029d2e31986d61"} err="failed to get container status \"ea95b74047c7cba81b65a2254b2d4d68bc00e385fe40fd74d9029d2e31986d61\": rpc error: code = NotFound desc = could not find container \"ea95b74047c7cba81b65a2254b2d4d68bc00e385fe40fd74d9029d2e31986d61\": container with ID starting with ea95b74047c7cba81b65a2254b2d4d68bc00e385fe40fd74d9029d2e31986d61 not found: ID does not exist"
Sep 05 06:55:27.038793 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.038757    2559 scope.go:117] "RemoveContainer" containerID="1dc6042a72779705ae676c75dd613febb03e67eec7e532607e0ab81d520f321d"
Sep 05 06:55:27.039044 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:55:27.039026    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dc6042a72779705ae676c75dd613febb03e67eec7e532607e0ab81d520f321d\": container with ID starting with 1dc6042a72779705ae676c75dd613febb03e67eec7e532607e0ab81d520f321d not found: ID does not exist" containerID="1dc6042a72779705ae676c75dd613febb03e67eec7e532607e0ab81d520f321d"
Sep 05 06:55:27.039083 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.039050    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dc6042a72779705ae676c75dd613febb03e67eec7e532607e0ab81d520f321d"} err="failed to get container status \"1dc6042a72779705ae676c75dd613febb03e67eec7e532607e0ab81d520f321d\": rpc error: code = NotFound desc = could not find container \"1dc6042a72779705ae676c75dd613febb03e67eec7e532607e0ab81d520f321d\": container with ID starting with 1dc6042a72779705ae676c75dd613febb03e67eec7e532607e0ab81d520f321d not found: ID does not exist"
Sep 05 06:55:27.045243 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.045222    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"]
Sep 05 06:55:27.049111 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.049090    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-b49086-openshift-default-6fcb988874-pnxv9"]
Sep 05 06:55:27.894014 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:27.893982    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09a0b23e-33dc-4fca-8113-8d536381c9e3" path="/var/lib/kubelet/pods/09a0b23e-33dc-4fca-8113-8d536381c9e3/volumes"
Sep 05 06:55:35.388384 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:35.388345    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:55:50.876179 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:50.876138    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-39047b-29809845-l54dr"]
Sep 05 06:55:50.878308 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:50.878283    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-39047b-29809845-l54dr"]
Sep 05 06:55:51.891626 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:51.891593    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc9cd7f3-c024-4362-88c5-5ce6863721ff" path="/var/lib/kubelet/pods/fc9cd7f3-c024-4362-88c5-5ce6863721ff/volumes"
Sep 05 06:55:53.484283 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:55:53.484247    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:56:03.882011 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:56:03.881978    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:56:24.223824 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:56:24.223795    2559 scope.go:117] "RemoveContainer" containerID="7403dd2d642bfd6f324e1f4d7b209e9b77d2ec8ba4c96e250e94059135eba0cf"
Sep 05 06:56:24.232573 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:56:24.232554    2559 scope.go:117] "RemoveContainer" containerID="b3d4b8c480aab2ab8e0650ab585cbcac38bb1f8e7c96aeaa9ee0c54016dac5f8"
Sep 05 06:56:24.240395 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:56:24.240371    2559 scope.go:117] "RemoveContainer" containerID="d282909a774cfdbcd2ad4880b0e82fe84ab40fe8217185bdd863086bee1163f5"
Sep 05 06:56:24.250051 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:56:24.250033    2559 scope.go:117] "RemoveContainer" containerID="1e912877917b41564271d6ccab90f722e9509e948bdd8c246dfb0c18fc56b6b1"
Sep 05 06:57:22.679961 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:57:22.679910    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:57:49.486338 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:57:49.486255    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:57:57.086699 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:57:57.086663    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:58:04.579148 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:04.579115    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:58:11.183969 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:11.183938    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:58:23.988566 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:23.988533    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:58:31.790455 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:31.790418    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 06:58:39.328413 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:39.328369    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"]
Sep 05 06:58:39.328920 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:39.328689    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg" podUID="a96bbb54-ef69-4974-b0e2-4c165c01e8ca" containerName="manager" containerID="cri-o://17ab8ac482e4475dfc7f022d75de2ae1a1f6065880cd5155862821d5ec7bb399" gracePeriod=10
Sep 05 06:58:39.571858 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:39.571833    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"
Sep 05 06:58:39.704303 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:39.704271    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a96bbb54-ef69-4974-b0e2-4c165c01e8ca-extensions-socket-volume\") pod \"a96bbb54-ef69-4974-b0e2-4c165c01e8ca\" (UID: \"a96bbb54-ef69-4974-b0e2-4c165c01e8ca\") "
Sep 05 06:58:39.704468 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:39.704376    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8zvk\" (UniqueName: \"kubernetes.io/projected/a96bbb54-ef69-4974-b0e2-4c165c01e8ca-kube-api-access-z8zvk\") pod \"a96bbb54-ef69-4974-b0e2-4c165c01e8ca\" (UID: \"a96bbb54-ef69-4974-b0e2-4c165c01e8ca\") "
Sep 05 06:58:39.704666 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:39.704637    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a96bbb54-ef69-4974-b0e2-4c165c01e8ca-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a96bbb54-ef69-4974-b0e2-4c165c01e8ca" (UID: "a96bbb54-ef69-4974-b0e2-4c165c01e8ca"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 06:58:39.706278 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:39.706255    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a96bbb54-ef69-4974-b0e2-4c165c01e8ca-kube-api-access-z8zvk" (OuterVolumeSpecName: "kube-api-access-z8zvk") pod "a96bbb54-ef69-4974-b0e2-4c165c01e8ca" (UID: "a96bbb54-ef69-4974-b0e2-4c165c01e8ca"). InnerVolumeSpecName "kube-api-access-z8zvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 06:58:39.805237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:39.805211    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a96bbb54-ef69-4974-b0e2-4c165c01e8ca-extensions-socket-volume\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:58:39.805237 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:39.805234    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8zvk\" (UniqueName: \"kubernetes.io/projected/a96bbb54-ef69-4974-b0e2-4c165c01e8ca-kube-api-access-z8zvk\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 06:58:40.082811 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:40.082734    2559 generic.go:356] "Generic (PLEG): container finished" podID="a96bbb54-ef69-4974-b0e2-4c165c01e8ca" containerID="17ab8ac482e4475dfc7f022d75de2ae1a1f6065880cd5155862821d5ec7bb399" exitCode=0
Sep 05 06:58:40.082811 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:40.082782    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"
Sep 05 06:58:40.082811 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:40.082783    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg" event={"ID":"a96bbb54-ef69-4974-b0e2-4c165c01e8ca","Type":"ContainerDied","Data":"17ab8ac482e4475dfc7f022d75de2ae1a1f6065880cd5155862821d5ec7bb399"}
Sep 05 06:58:40.083012 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:40.082836    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg" event={"ID":"a96bbb54-ef69-4974-b0e2-4c165c01e8ca","Type":"ContainerDied","Data":"78193c35a58b2e353708c96c57f94ce6d3f186e5ca3549ecbac1b9807987d493"}
Sep 05 06:58:40.083012 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:40.082869    2559 scope.go:117] "RemoveContainer" containerID="17ab8ac482e4475dfc7f022d75de2ae1a1f6065880cd5155862821d5ec7bb399"
Sep 05 06:58:40.092682 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:40.092662    2559 scope.go:117] "RemoveContainer" containerID="17ab8ac482e4475dfc7f022d75de2ae1a1f6065880cd5155862821d5ec7bb399"
Sep 05 06:58:40.092992 ip-10-0-130-170 kubenswrapper[2559]: E0905 06:58:40.092970    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17ab8ac482e4475dfc7f022d75de2ae1a1f6065880cd5155862821d5ec7bb399\": container with ID starting with 17ab8ac482e4475dfc7f022d75de2ae1a1f6065880cd5155862821d5ec7bb399 not found: ID does not exist" containerID="17ab8ac482e4475dfc7f022d75de2ae1a1f6065880cd5155862821d5ec7bb399"
Sep 05 06:58:40.093077 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:40.093000    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17ab8ac482e4475dfc7f022d75de2ae1a1f6065880cd5155862821d5ec7bb399"} err="failed to get container status \"17ab8ac482e4475dfc7f022d75de2ae1a1f6065880cd5155862821d5ec7bb399\": rpc error: code = NotFound desc = could not find container \"17ab8ac482e4475dfc7f022d75de2ae1a1f6065880cd5155862821d5ec7bb399\": container with ID starting with 17ab8ac482e4475dfc7f022d75de2ae1a1f6065880cd5155862821d5ec7bb399 not found: ID does not exist"
Sep 05 06:58:40.102717 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:40.102693    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"]
Sep 05 06:58:40.109189 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:40.109164    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-nvfdg"]
Sep 05 06:58:41.892511 ip-10-0-130-170 kubenswrapper[2559]: I0905 06:58:41.892472    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a96bbb54-ef69-4974-b0e2-4c165c01e8ca" path="/var/lib/kubelet/pods/a96bbb54-ef69-4974-b0e2-4c165c01e8ca/volumes"
Sep 05 07:00:03.004059 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:03.004024    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 07:00:09.894631 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:09.894594    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 07:00:19.398482 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:19.398444    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 07:00:24.316712 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:24.316681    2559 scope.go:117] "RemoveContainer" containerID="b285a524df86b671faebd05296b4b834a343a03d03fa5a68b7197f98009030c2"
Sep 05 07:00:26.982421 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:26.982385    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"]
Sep 05 07:00:26.982777 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:26.982718    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b343826c-fc4d-4f44-954c-ef0e9dbf1a94" containerName="revoke-keys"
Sep 05 07:00:26.982777 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:26.982729    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b343826c-fc4d-4f44-954c-ef0e9dbf1a94" containerName="revoke-keys"
Sep 05 07:00:26.982777 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:26.982738    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a96bbb54-ef69-4974-b0e2-4c165c01e8ca" containerName="manager"
Sep 05 07:00:26.982777 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:26.982743    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a96bbb54-ef69-4974-b0e2-4c165c01e8ca" containerName="manager"
Sep 05 07:00:26.982777 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:26.982753    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09a0b23e-33dc-4fca-8113-8d536381c9e3" containerName="istio-proxy"
Sep 05 07:00:26.982777 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:26.982759    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="09a0b23e-33dc-4fca-8113-8d536381c9e3" containerName="istio-proxy"
Sep 05 07:00:26.982997 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:26.982819    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="09a0b23e-33dc-4fca-8113-8d536381c9e3" containerName="istio-proxy"
Sep 05 07:00:26.982997 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:26.982826    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a96bbb54-ef69-4974-b0e2-4c165c01e8ca" containerName="manager"
Sep 05 07:00:26.982997 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:26.982832    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b343826c-fc4d-4f44-954c-ef0e9dbf1a94" containerName="revoke-keys"
Sep 05 07:00:26.987350 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:26.987331    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:26.989658 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:26.989637    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-77503f04-kserve-self-signed-certs\""
Sep 05 07:00:26.994564 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:26.994393    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"]
Sep 05 07:00:27.070253 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.070226    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-model-cache\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.070379 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.070286    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-home\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.070379 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.070364    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-tmp-dir\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.070538 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.070413    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d93611c-e77f-4a69-823d-bca82f4b5a45-tls-certs\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.070538 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.070459    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v6bq\" (UniqueName: \"kubernetes.io/projected/6d93611c-e77f-4a69-823d-bca82f4b5a45-kube-api-access-7v6bq\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.070538 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.070498    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-dshm\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.171252 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.171216    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-dshm\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.171415 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.171273    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-model-cache\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.171415 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.171345    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-home\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.171415 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.171385    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-tmp-dir\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.171600 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.171422    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d93611c-e77f-4a69-823d-bca82f4b5a45-tls-certs\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.171600 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.171455    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v6bq\" (UniqueName: \"kubernetes.io/projected/6d93611c-e77f-4a69-823d-bca82f4b5a45-kube-api-access-7v6bq\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.171759 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.171722    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-model-cache\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.171904 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.171864    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-tmp-dir\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.171966 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.171869    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-home\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.173535 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.173511    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-dshm\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.173839 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.173819    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d93611c-e77f-4a69-823d-bca82f4b5a45-tls-certs\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.179442 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.179420    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v6bq\" (UniqueName: \"kubernetes.io/projected/6d93611c-e77f-4a69-823d-bca82f4b5a45-kube-api-access-7v6bq\") pod \"e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") " pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.297197 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.297128    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:27.567682 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.567657    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"]
Sep 05 07:00:27.569503 ip-10-0-130-170 kubenswrapper[2559]: W0905 07:00:27.569475    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d93611c_e77f_4a69_823d_bca82f4b5a45.slice/crio-c889cb23f130c24ac894cc175f31935d7875fb521427362ca1901c70267255b3 WatchSource:0}: Error finding container c889cb23f130c24ac894cc175f31935d7875fb521427362ca1901c70267255b3: Status 404 returned error can't find the container with id c889cb23f130c24ac894cc175f31935d7875fb521427362ca1901c70267255b3
Sep 05 07:00:27.675743 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.675699    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp" event={"ID":"6d93611c-e77f-4a69-823d-bca82f4b5a45","Type":"ContainerStarted","Data":"2a98b387b0488db30827d8df0ed533fefe4550fb115a6e8e64a1b9240de52ae2"}
Sep 05 07:00:27.675743 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.675744    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp" event={"ID":"6d93611c-e77f-4a69-823d-bca82f4b5a45","Type":"ContainerStarted","Data":"c889cb23f130c24ac894cc175f31935d7875fb521427362ca1901c70267255b3"}
Sep 05 07:00:27.693290 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:27.693239    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp" podStartSLOduration=1.6932243599999999 podStartE2EDuration="1.69322436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:00:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:00:27.690817355 +0000 UTC m=+2044.281376343" watchObservedRunningTime="2026-09-05 07:00:27.69322436 +0000 UTC m=+2044.283783311"
Sep 05 07:00:30.314610 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:30.314575    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 07:00:37.298020 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:37.297986    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:37.298488 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:37.298033    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:37.310728 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:37.310702    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:37.749615 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:37.749589    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:00:40.176323 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.176286    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"]
Sep 05 07:00:40.176679 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.176670    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09a0b23e-33dc-4fca-8113-8d536381c9e3" containerName="istio-proxy"
Sep 05 07:00:40.176721 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.176683    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="09a0b23e-33dc-4fca-8113-8d536381c9e3" containerName="istio-proxy"
Sep 05 07:00:40.176762 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.176740    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="09a0b23e-33dc-4fca-8113-8d536381c9e3" containerName="istio-proxy"
Sep 05 07:00:40.180228 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.180204    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.182706 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.182682    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-77503f04-kserve-self-signed-certs\""
Sep 05 07:00:40.191072 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.191046    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"]
Sep 05 07:00:40.276120 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.276089    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-model-cache\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.276278 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.276187    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-tmp-dir\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.276324 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.276279    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-home\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.276363 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.276327    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-dshm\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.276398 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.276366    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40e688a2-934d-452e-9e77-40cf77904c0c-tls-certs\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.276437 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.276392    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrszh\" (UniqueName: \"kubernetes.io/projected/40e688a2-934d-452e-9e77-40cf77904c0c-kube-api-access-mrszh\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.377203 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.377179    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-model-cache\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.377350 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.377214    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-tmp-dir\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.377350 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.377249    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-home\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.377350 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.377270    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-dshm\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.377350 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.377290    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40e688a2-934d-452e-9e77-40cf77904c0c-tls-certs\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.377350 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.377305    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrszh\" (UniqueName: \"kubernetes.io/projected/40e688a2-934d-452e-9e77-40cf77904c0c-kube-api-access-mrszh\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.377598 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.377571    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-model-cache\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.377661 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.377624    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-home\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.377661 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.377648    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-tmp-dir\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.379623 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.379596    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-dshm\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.379928 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.379908    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40e688a2-934d-452e-9e77-40cf77904c0c-tls-certs\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.385429 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.385407    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrszh\" (UniqueName: \"kubernetes.io/projected/40e688a2-934d-452e-9e77-40cf77904c0c-kube-api-access-mrszh\") pod \"e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") " pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.490169 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.490140    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:40.762007 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:40.761981    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"]
Sep 05 07:00:40.763975 ip-10-0-130-170 kubenswrapper[2559]: W0905 07:00:40.763947    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40e688a2_934d_452e_9e77_40cf77904c0c.slice/crio-145a835f978759611c3c0b3a3d63ab02def01b0334683218f61897f2c725264e WatchSource:0}: Error finding container 145a835f978759611c3c0b3a3d63ab02def01b0334683218f61897f2c725264e: Status 404 returned error can't find the container with id 145a835f978759611c3c0b3a3d63ab02def01b0334683218f61897f2c725264e
Sep 05 07:00:41.761183 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:41.761148    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7" event={"ID":"40e688a2-934d-452e-9e77-40cf77904c0c","Type":"ContainerStarted","Data":"f20aaf74f92f6e6f243e9d1d768899e81b7e9d61c7b1231f4ebae5c61acf6a56"}
Sep 05 07:00:41.761542 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:41.761191    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7" event={"ID":"40e688a2-934d-452e-9e77-40cf77904c0c","Type":"ContainerStarted","Data":"145a835f978759611c3c0b3a3d63ab02def01b0334683218f61897f2c725264e"}
Sep 05 07:00:41.778157 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:41.778108    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7" podStartSLOduration=1.7780922270000001 podStartE2EDuration="1.778092227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:00:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:00:41.775809664 +0000 UTC m=+2058.366368617" watchObservedRunningTime="2026-09-05 07:00:41.778092227 +0000 UTC m=+2058.368651233"
Sep 05 07:00:42.999125 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:42.999088    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 07:00:50.490523 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:50.490422    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:50.490523 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:50.490485    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:50.508825 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:50.508786    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:50.820468 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:50.820385    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:00:52.402869 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:52.402837    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"]
Sep 05 07:00:53.694204 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:53.694172    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"]
Sep 05 07:00:54.893514 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:00:54.893479    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 07:01:02.890913 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:02.890864    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 07:01:07.422522 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.422484    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp" podUID="6d93611c-e77f-4a69-823d-bca82f4b5a45" containerName="main" containerID="cri-o://2a98b387b0488db30827d8df0ed533fefe4550fb115a6e8e64a1b9240de52ae2" gracePeriod=45
Sep 05 07:01:07.673822 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.673763    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:01:07.815331 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.815300    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d93611c-e77f-4a69-823d-bca82f4b5a45-tls-certs\") pod \"6d93611c-e77f-4a69-823d-bca82f4b5a45\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") "
Sep 05 07:01:07.815485 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.815347    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7v6bq\" (UniqueName: \"kubernetes.io/projected/6d93611c-e77f-4a69-823d-bca82f4b5a45-kube-api-access-7v6bq\") pod \"6d93611c-e77f-4a69-823d-bca82f4b5a45\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") "
Sep 05 07:01:07.815485 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.815400    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-home\") pod \"6d93611c-e77f-4a69-823d-bca82f4b5a45\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") "
Sep 05 07:01:07.815485 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.815418    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-model-cache\") pod \"6d93611c-e77f-4a69-823d-bca82f4b5a45\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") "
Sep 05 07:01:07.815485 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.815463    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-dshm\") pod \"6d93611c-e77f-4a69-823d-bca82f4b5a45\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") "
Sep 05 07:01:07.815485 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.815480    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-tmp-dir\") pod \"6d93611c-e77f-4a69-823d-bca82f4b5a45\" (UID: \"6d93611c-e77f-4a69-823d-bca82f4b5a45\") "
Sep 05 07:01:07.815796 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.815747    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-model-cache" (OuterVolumeSpecName: "model-cache") pod "6d93611c-e77f-4a69-823d-bca82f4b5a45" (UID: "6d93611c-e77f-4a69-823d-bca82f4b5a45"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 07:01:07.815864 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.815793    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6d93611c-e77f-4a69-823d-bca82f4b5a45" (UID: "6d93611c-e77f-4a69-823d-bca82f4b5a45"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 07:01:07.815864 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.815733    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-home" (OuterVolumeSpecName: "home") pod "6d93611c-e77f-4a69-823d-bca82f4b5a45" (UID: "6d93611c-e77f-4a69-823d-bca82f4b5a45"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 07:01:07.817633 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.817601    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d93611c-e77f-4a69-823d-bca82f4b5a45-kube-api-access-7v6bq" (OuterVolumeSpecName: "kube-api-access-7v6bq") pod "6d93611c-e77f-4a69-823d-bca82f4b5a45" (UID: "6d93611c-e77f-4a69-823d-bca82f4b5a45"). InnerVolumeSpecName "kube-api-access-7v6bq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:01:07.817633 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.817606    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-dshm" (OuterVolumeSpecName: "dshm") pod "6d93611c-e77f-4a69-823d-bca82f4b5a45" (UID: "6d93611c-e77f-4a69-823d-bca82f4b5a45"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 07:01:07.817813 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.817792    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d93611c-e77f-4a69-823d-bca82f4b5a45-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6d93611c-e77f-4a69-823d-bca82f4b5a45" (UID: "6d93611c-e77f-4a69-823d-bca82f4b5a45"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:01:07.904317 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.904290    2559 generic.go:356] "Generic (PLEG): container finished" podID="6d93611c-e77f-4a69-823d-bca82f4b5a45" containerID="2a98b387b0488db30827d8df0ed533fefe4550fb115a6e8e64a1b9240de52ae2" exitCode=0
Sep 05 07:01:07.904433 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.904331    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"
Sep 05 07:01:07.904433 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.904368    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp" event={"ID":"6d93611c-e77f-4a69-823d-bca82f4b5a45","Type":"ContainerDied","Data":"2a98b387b0488db30827d8df0ed533fefe4550fb115a6e8e64a1b9240de52ae2"}
Sep 05 07:01:07.904433 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.904404    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp" event={"ID":"6d93611c-e77f-4a69-823d-bca82f4b5a45","Type":"ContainerDied","Data":"c889cb23f130c24ac894cc175f31935d7875fb521427362ca1901c70267255b3"}
Sep 05 07:01:07.904433 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.904419    2559 scope.go:117] "RemoveContainer" containerID="2a98b387b0488db30827d8df0ed533fefe4550fb115a6e8e64a1b9240de52ae2"
Sep 05 07:01:07.913160 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.913142    2559 scope.go:117] "RemoveContainer" containerID="2a98b387b0488db30827d8df0ed533fefe4550fb115a6e8e64a1b9240de52ae2"
Sep 05 07:01:07.913427 ip-10-0-130-170 kubenswrapper[2559]: E0905 07:01:07.913406    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a98b387b0488db30827d8df0ed533fefe4550fb115a6e8e64a1b9240de52ae2\": container with ID starting with 2a98b387b0488db30827d8df0ed533fefe4550fb115a6e8e64a1b9240de52ae2 not found: ID does not exist" containerID="2a98b387b0488db30827d8df0ed533fefe4550fb115a6e8e64a1b9240de52ae2"
Sep 05 07:01:07.913492 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.913434    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a98b387b0488db30827d8df0ed533fefe4550fb115a6e8e64a1b9240de52ae2"} err="failed to get container status \"2a98b387b0488db30827d8df0ed533fefe4550fb115a6e8e64a1b9240de52ae2\": rpc error: code = NotFound desc = could not find container \"2a98b387b0488db30827d8df0ed533fefe4550fb115a6e8e64a1b9240de52ae2\": container with ID starting with 2a98b387b0488db30827d8df0ed533fefe4550fb115a6e8e64a1b9240de52ae2 not found: ID does not exist"
Sep 05 07:01:07.916814 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.916796    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d93611c-e77f-4a69-823d-bca82f4b5a45-tls-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 07:01:07.916871 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.916816    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7v6bq\" (UniqueName: \"kubernetes.io/projected/6d93611c-e77f-4a69-823d-bca82f4b5a45-kube-api-access-7v6bq\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 07:01:07.916871 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.916826    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-home\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 07:01:07.916871 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.916836    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-model-cache\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 07:01:07.916871 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.916844    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-dshm\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 07:01:07.916871 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.916851    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d93611c-e77f-4a69-823d-bca82f4b5a45-tmp-dir\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 07:01:07.922315 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.922296    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"]
Sep 05 07:01:07.926758 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:07.926711    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-77503f04-kserve-545d5c8447-vrgxp"]
Sep 05 07:01:08.714281 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:08.714239    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7" podUID="40e688a2-934d-452e-9e77-40cf77904c0c" containerName="main" containerID="cri-o://f20aaf74f92f6e6f243e9d1d768899e81b7e9d61c7b1231f4ebae5c61acf6a56" gracePeriod=45
Sep 05 07:01:08.913002 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:08.912974    2559 generic.go:356] "Generic (PLEG): container finished" podID="40e688a2-934d-452e-9e77-40cf77904c0c" containerID="f20aaf74f92f6e6f243e9d1d768899e81b7e9d61c7b1231f4ebae5c61acf6a56" exitCode=0
Sep 05 07:01:08.913150 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:08.913036    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7" event={"ID":"40e688a2-934d-452e-9e77-40cf77904c0c","Type":"ContainerDied","Data":"f20aaf74f92f6e6f243e9d1d768899e81b7e9d61c7b1231f4ebae5c61acf6a56"}
Sep 05 07:01:08.962954 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:08.962932    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:01:09.127319 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.127252    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrszh\" (UniqueName: \"kubernetes.io/projected/40e688a2-934d-452e-9e77-40cf77904c0c-kube-api-access-mrszh\") pod \"40e688a2-934d-452e-9e77-40cf77904c0c\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") "
Sep 05 07:01:09.127319 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.127291    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-tmp-dir\") pod \"40e688a2-934d-452e-9e77-40cf77904c0c\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") "
Sep 05 07:01:09.127319 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.127318    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-dshm\") pod \"40e688a2-934d-452e-9e77-40cf77904c0c\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") "
Sep 05 07:01:09.127572 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.127444    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-model-cache\") pod \"40e688a2-934d-452e-9e77-40cf77904c0c\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") "
Sep 05 07:01:09.127572 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.127516    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-home\") pod \"40e688a2-934d-452e-9e77-40cf77904c0c\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") "
Sep 05 07:01:09.127572 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.127547    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40e688a2-934d-452e-9e77-40cf77904c0c-tls-certs\") pod \"40e688a2-934d-452e-9e77-40cf77904c0c\" (UID: \"40e688a2-934d-452e-9e77-40cf77904c0c\") "
Sep 05 07:01:09.127733 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.127563    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "40e688a2-934d-452e-9e77-40cf77904c0c" (UID: "40e688a2-934d-452e-9e77-40cf77904c0c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 07:01:09.127733 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.127629    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-model-cache" (OuterVolumeSpecName: "model-cache") pod "40e688a2-934d-452e-9e77-40cf77904c0c" (UID: "40e688a2-934d-452e-9e77-40cf77904c0c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 07:01:09.127822 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.127799    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-home" (OuterVolumeSpecName: "home") pod "40e688a2-934d-452e-9e77-40cf77904c0c" (UID: "40e688a2-934d-452e-9e77-40cf77904c0c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 07:01:09.127929 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.127911    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-home\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 07:01:09.127991 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.127935    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-tmp-dir\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 07:01:09.127991 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.127950    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-model-cache\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 07:01:09.129383 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.129361    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-dshm" (OuterVolumeSpecName: "dshm") pod "40e688a2-934d-452e-9e77-40cf77904c0c" (UID: "40e688a2-934d-452e-9e77-40cf77904c0c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 05 07:01:09.129484 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.129470    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40e688a2-934d-452e-9e77-40cf77904c0c-kube-api-access-mrszh" (OuterVolumeSpecName: "kube-api-access-mrszh") pod "40e688a2-934d-452e-9e77-40cf77904c0c" (UID: "40e688a2-934d-452e-9e77-40cf77904c0c"). InnerVolumeSpecName "kube-api-access-mrszh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 05 07:01:09.129544 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.129493    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40e688a2-934d-452e-9e77-40cf77904c0c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "40e688a2-934d-452e-9e77-40cf77904c0c" (UID: "40e688a2-934d-452e-9e77-40cf77904c0c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 05 07:01:09.228511 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.228473    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40e688a2-934d-452e-9e77-40cf77904c0c-tls-certs\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 07:01:09.228511 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.228506    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrszh\" (UniqueName: \"kubernetes.io/projected/40e688a2-934d-452e-9e77-40cf77904c0c-kube-api-access-mrszh\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 07:01:09.228511 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.228516    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40e688a2-934d-452e-9e77-40cf77904c0c-dshm\") on node \"ip-10-0-130-170.ec2.internal\" DevicePath \"\""
Sep 05 07:01:09.798639 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.798602    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 07:01:09.892272 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.892236    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d93611c-e77f-4a69-823d-bca82f4b5a45" path="/var/lib/kubelet/pods/6d93611c-e77f-4a69-823d-bca82f4b5a45/volumes"
Sep 05 07:01:09.918870 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.918853    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"
Sep 05 07:01:09.918999 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.918908    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7" event={"ID":"40e688a2-934d-452e-9e77-40cf77904c0c","Type":"ContainerDied","Data":"145a835f978759611c3c0b3a3d63ab02def01b0334683218f61897f2c725264e"}
Sep 05 07:01:09.918999 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.918956    2559 scope.go:117] "RemoveContainer" containerID="f20aaf74f92f6e6f243e9d1d768899e81b7e9d61c7b1231f4ebae5c61acf6a56"
Sep 05 07:01:09.937546 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.937515    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"]
Sep 05 07:01:09.940944 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:09.940923    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-77503f04-kserve-5dbf684bff-th6b7"]
Sep 05 07:01:11.892246 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:11.892213    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40e688a2-934d-452e-9e77-40cf77904c0c" path="/var/lib/kubelet/pods/40e688a2-934d-452e-9e77-40cf77904c0c/volumes"
Sep 05 07:01:19.602043 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:19.602007    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 07:01:24.337098 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:24.337065    2559 scope.go:117] "RemoveContainer" containerID="e2b27ca31847de53bd00a9802d082bd797e532d2267dad38e402ec9a9c0ffe42"
Sep 05 07:01:27.305923 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:27.305876    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 07:01:35.505448 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:35.505407    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 07:01:42.494915 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:42.493677    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-vn7br"]
Sep 05 07:01:45.126741 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:45.126708    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57cf4f5b8b-2vcjq_b9279c18-f100-497c-b1c1-01175bd1e84b/authorino/0.log"
Sep 05 07:01:50.351300 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:50.351262    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-59mrk_9c5aeb09-5dd2-49fd-9114-6791fbf0d742/manager/0.log"
Sep 05 07:01:50.477271 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:50.477243    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5rvzx_7dacb06e-b358-46b8-8508-f27c61f168eb/manager/0.log"
Sep 05 07:01:50.730703 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:50.730672    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gzgqb_99fcb1d8-584f-4b9a-beab-d0c1636b398a/server/0.log"
Sep 05 07:01:51.227355 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:51.227330    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555dc54ddd-zgvqq_4b7d6079-e03b-4bd4-99a9-982adf37a1da/copy-manifests/0.log"
Sep 05 07:01:51.240978 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:51.240955    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555dc54ddd-zgvqq_4b7d6079-e03b-4bd4-99a9-982adf37a1da/manager/0.log"
Sep 05 07:01:51.810867 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:51.810829    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29809845-2q6dp_08176018-d9cd-4592-80d4-aa6fc035d729/cleanup/0.log"
Sep 05 07:01:52.799622 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:52.799592    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57cf4f5b8b-2vcjq_b9279c18-f100-497c-b1c1-01175bd1e84b/authorino/0.log"
Sep 05 07:01:52.912171 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:52.912134    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-szsxn_c546b6c3-6b40-4eea-bbbb-66859b113974/manager/0.log"
Sep 05 07:01:53.025859 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:53.025833    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-vjkb7_0ae3f591-8cde-466c-99f3-5cbf47a5fbcf/manager/0.log"
Sep 05 07:01:53.375306 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:53.375279    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-vn7br_2e362e88-5eb3-4c01-ad5c-5978374f14a4/limitador/0.log"
Sep 05 07:01:53.493665 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:53.493628    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-v9msv_5cb8f71d-990b-4ee1-b1e3-18931667fdac/manager/0.log"
Sep 05 07:01:54.092773 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:54.092738    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bd96764bd-6zzt8_d783a45f-8564-44f1-ace1-906ef791ee4c/kube-auth-proxy/0.log"
Sep 05 07:01:54.692733 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:54.692695    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74c88b6858-trsfj_f9b3185c-0e0e-43aa-8e47-1090bfb5f016/router/0.log"
Sep 05 07:01:55.146855 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:55.146778    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-bcw86_133997e4-7df9-4d51-ae1a-315fd98e7c7f/main/0.log"
Sep 05 07:01:55.257607 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:55.257583    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-jxp9l_b1962265-ecfd-4996-9a59-c6e1fd72e72f/main/0.log"
Sep 05 07:01:55.481803 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:55.481775    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5cds9_bb22452a-2e27-4b6d-8f3a-fe66c0ed96ca/main/0.log"
Sep 05 07:01:55.589407 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:55.589379    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-2p5z4_b189488c-63bc-4885-ba0a-3a675c048bd6/main/0.log"
Sep 05 07:01:55.701589 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:01:55.701569    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-grthp_c686d0fe-cc66-4a2a-a592-48f209314c9f/main/0.log"
Sep 05 07:02:01.641111 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.641078    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tj5rw/must-gather-pkbsl"]
Sep 05 07:02:01.641501 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.641484    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d93611c-e77f-4a69-823d-bca82f4b5a45" containerName="main"
Sep 05 07:02:01.641501 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.641498    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d93611c-e77f-4a69-823d-bca82f4b5a45" containerName="main"
Sep 05 07:02:01.641576 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.641520    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40e688a2-934d-452e-9e77-40cf77904c0c" containerName="main"
Sep 05 07:02:01.641576 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.641526    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="40e688a2-934d-452e-9e77-40cf77904c0c" containerName="main"
Sep 05 07:02:01.641647 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.641586    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d93611c-e77f-4a69-823d-bca82f4b5a45" containerName="main"
Sep 05 07:02:01.641647 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.641602    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="40e688a2-934d-452e-9e77-40cf77904c0c" containerName="main"
Sep 05 07:02:01.644540 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.644521    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tj5rw/must-gather-pkbsl"
Sep 05 07:02:01.646992 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.646970    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tj5rw\"/\"default-dockercfg-9d82n\""
Sep 05 07:02:01.647126 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.646973    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tj5rw\"/\"openshift-service-ca.crt\""
Sep 05 07:02:01.647126 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.647033    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tj5rw\"/\"kube-root-ca.crt\""
Sep 05 07:02:01.653599 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.653576    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tj5rw/must-gather-pkbsl"]
Sep 05 07:02:01.735717 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.735683    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c158614a-828b-4c4d-9cef-1c372c90eb09-must-gather-output\") pod \"must-gather-pkbsl\" (UID: \"c158614a-828b-4c4d-9cef-1c372c90eb09\") " pod="openshift-must-gather-tj5rw/must-gather-pkbsl"
Sep 05 07:02:01.735821 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.735740    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db74p\" (UniqueName: \"kubernetes.io/projected/c158614a-828b-4c4d-9cef-1c372c90eb09-kube-api-access-db74p\") pod \"must-gather-pkbsl\" (UID: \"c158614a-828b-4c4d-9cef-1c372c90eb09\") " pod="openshift-must-gather-tj5rw/must-gather-pkbsl"
Sep 05 07:02:01.836772 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.836744    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c158614a-828b-4c4d-9cef-1c372c90eb09-must-gather-output\") pod \"must-gather-pkbsl\" (UID: \"c158614a-828b-4c4d-9cef-1c372c90eb09\") " pod="openshift-must-gather-tj5rw/must-gather-pkbsl"
Sep 05 07:02:01.836873 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.836791    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db74p\" (UniqueName: \"kubernetes.io/projected/c158614a-828b-4c4d-9cef-1c372c90eb09-kube-api-access-db74p\") pod \"must-gather-pkbsl\" (UID: \"c158614a-828b-4c4d-9cef-1c372c90eb09\") " pod="openshift-must-gather-tj5rw/must-gather-pkbsl"
Sep 05 07:02:01.837111 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.837092    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c158614a-828b-4c4d-9cef-1c372c90eb09-must-gather-output\") pod \"must-gather-pkbsl\" (UID: \"c158614a-828b-4c4d-9cef-1c372c90eb09\") " pod="openshift-must-gather-tj5rw/must-gather-pkbsl"
Sep 05 07:02:01.847082 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.847063    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db74p\" (UniqueName: \"kubernetes.io/projected/c158614a-828b-4c4d-9cef-1c372c90eb09-kube-api-access-db74p\") pod \"must-gather-pkbsl\" (UID: \"c158614a-828b-4c4d-9cef-1c372c90eb09\") " pod="openshift-must-gather-tj5rw/must-gather-pkbsl"
Sep 05 07:02:01.954183 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:01.954163    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tj5rw/must-gather-pkbsl"
Sep 05 07:02:02.228482 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:02.228458    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tj5rw/must-gather-pkbsl"]
Sep 05 07:02:02.230043 ip-10-0-130-170 kubenswrapper[2559]: W0905 07:02:02.230012    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc158614a_828b_4c4d_9cef_1c372c90eb09.slice/crio-bfa6e8bfeb8d765c4fa0b9e4da74b4c076f8899491d14ed2961c652e979da588 WatchSource:0}: Error finding container bfa6e8bfeb8d765c4fa0b9e4da74b4c076f8899491d14ed2961c652e979da588: Status 404 returned error can't find the container with id bfa6e8bfeb8d765c4fa0b9e4da74b4c076f8899491d14ed2961c652e979da588
Sep 05 07:02:02.232158 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:02.232141    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 05 07:02:03.202233 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:03.202206    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tj5rw/must-gather-pkbsl" event={"ID":"c158614a-828b-4c4d-9cef-1c372c90eb09","Type":"ContainerStarted","Data":"bfa6e8bfeb8d765c4fa0b9e4da74b4c076f8899491d14ed2961c652e979da588"}
Sep 05 07:02:04.213962 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:04.213910    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tj5rw/must-gather-pkbsl" event={"ID":"c158614a-828b-4c4d-9cef-1c372c90eb09","Type":"ContainerStarted","Data":"63121ccaaa6e1a499692a5e0eae29e47761cb9da594795f8ea13064b789a36c7"}
Sep 05 07:02:04.214560 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:04.214524    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tj5rw/must-gather-pkbsl" event={"ID":"c158614a-828b-4c4d-9cef-1c372c90eb09","Type":"ContainerStarted","Data":"3bd272d83cb1bc85e0277b5ac47bba6568f30cc8710b0c024fdb021485c0f145"}
Sep 05 07:02:04.232493 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:04.232435    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tj5rw/must-gather-pkbsl" podStartSLOduration=2.340550424 podStartE2EDuration="3.232415552s" totalImagesPullingTime="891.865128ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-05 07:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:02:04.230395153 +0000 UTC m=+2140.820954130" watchObservedRunningTime="2026-09-05 07:02:04.232415552 +0000 UTC m=+2140.822974506"
Sep 05 07:02:04.751033 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:04.750999    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tcnw5_c85b9378-0915-419f-9b29-47bb580dc22a/global-pull-secret-syncer/0.log"
Sep 05 07:02:04.830244 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:04.830212    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jdbcm_2d9b77a8-583d-4d7e-b62e-60fe17d73f24/konnectivity-agent/0.log"
Sep 05 07:02:04.905234 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:04.905193    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-170.ec2.internal_50db2b2cd4cf113e73677eb1ebb83d24/haproxy/0.log"
Sep 05 07:02:09.990789 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:09.990736    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57cf4f5b8b-2vcjq_b9279c18-f100-497c-b1c1-01175bd1e84b/authorino/0.log"
Sep 05 07:02:10.020234 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:10.020182    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-szsxn_c546b6c3-6b40-4eea-bbbb-66859b113974/manager/0.log"
Sep 05 07:02:10.047847 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:10.047815    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-vjkb7_0ae3f591-8cde-466c-99f3-5cbf47a5fbcf/manager/0.log"
Sep 05 07:02:10.155348 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:10.155318    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-vn7br_2e362e88-5eb3-4c01-ad5c-5978374f14a4/limitador/0.log"
Sep 05 07:02:10.227661 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:10.227628    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-v9msv_5cb8f71d-990b-4ee1-b1e3-18931667fdac/manager/0.log"
Sep 05 07:02:12.091365 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:12.091300    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-576l9_443c621a-8883-486c-9fcf-98a0a117d462/node-exporter/0.log"
Sep 05 07:02:12.115151 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:12.115108    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-576l9_443c621a-8883-486c-9fcf-98a0a117d462/kube-rbac-proxy/0.log"
Sep 05 07:02:12.137714 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:12.137689    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-576l9_443c621a-8883-486c-9fcf-98a0a117d462/init-textfile/0.log"
Sep 05 07:02:14.432856 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.432815    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"]
Sep 05 07:02:14.437646 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.437622    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:14.453926 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.449403    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"]
Sep 05 07:02:14.484568 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.484538    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/78fbb610-0376-4e2d-8668-2e186f70bccd-podres\") pod \"perf-node-gather-daemonset-n6m7l\" (UID: \"78fbb610-0376-4e2d-8668-2e186f70bccd\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:14.484828 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.484804    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/78fbb610-0376-4e2d-8668-2e186f70bccd-proc\") pod \"perf-node-gather-daemonset-n6m7l\" (UID: \"78fbb610-0376-4e2d-8668-2e186f70bccd\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:14.485010 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.484995    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78fbb610-0376-4e2d-8668-2e186f70bccd-sys\") pod \"perf-node-gather-daemonset-n6m7l\" (UID: \"78fbb610-0376-4e2d-8668-2e186f70bccd\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:14.485128 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.485111    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/78fbb610-0376-4e2d-8668-2e186f70bccd-lib-modules\") pod \"perf-node-gather-daemonset-n6m7l\" (UID: \"78fbb610-0376-4e2d-8668-2e186f70bccd\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:14.485204 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.485146    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4877c\" (UniqueName: \"kubernetes.io/projected/78fbb610-0376-4e2d-8668-2e186f70bccd-kube-api-access-4877c\") pod \"perf-node-gather-daemonset-n6m7l\" (UID: \"78fbb610-0376-4e2d-8668-2e186f70bccd\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:14.586692 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.586652    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/78fbb610-0376-4e2d-8668-2e186f70bccd-lib-modules\") pod \"perf-node-gather-daemonset-n6m7l\" (UID: \"78fbb610-0376-4e2d-8668-2e186f70bccd\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:14.586912 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.586714    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4877c\" (UniqueName: \"kubernetes.io/projected/78fbb610-0376-4e2d-8668-2e186f70bccd-kube-api-access-4877c\") pod \"perf-node-gather-daemonset-n6m7l\" (UID: \"78fbb610-0376-4e2d-8668-2e186f70bccd\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:14.586912 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.586762    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/78fbb610-0376-4e2d-8668-2e186f70bccd-podres\") pod \"perf-node-gather-daemonset-n6m7l\" (UID: \"78fbb610-0376-4e2d-8668-2e186f70bccd\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:14.586912 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.586789    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/78fbb610-0376-4e2d-8668-2e186f70bccd-proc\") pod \"perf-node-gather-daemonset-n6m7l\" (UID: \"78fbb610-0376-4e2d-8668-2e186f70bccd\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:14.586912 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.586831    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78fbb610-0376-4e2d-8668-2e186f70bccd-sys\") pod \"perf-node-gather-daemonset-n6m7l\" (UID: \"78fbb610-0376-4e2d-8668-2e186f70bccd\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:14.586912 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.586840    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/78fbb610-0376-4e2d-8668-2e186f70bccd-lib-modules\") pod \"perf-node-gather-daemonset-n6m7l\" (UID: \"78fbb610-0376-4e2d-8668-2e186f70bccd\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:14.587205 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.586942    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78fbb610-0376-4e2d-8668-2e186f70bccd-sys\") pod \"perf-node-gather-daemonset-n6m7l\" (UID: \"78fbb610-0376-4e2d-8668-2e186f70bccd\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:14.587205 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.586963    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/78fbb610-0376-4e2d-8668-2e186f70bccd-proc\") pod \"perf-node-gather-daemonset-n6m7l\" (UID: \"78fbb610-0376-4e2d-8668-2e186f70bccd\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:14.587205 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.587010    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/78fbb610-0376-4e2d-8668-2e186f70bccd-podres\") pod \"perf-node-gather-daemonset-n6m7l\" (UID: \"78fbb610-0376-4e2d-8668-2e186f70bccd\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:14.596604 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.596564    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4877c\" (UniqueName: \"kubernetes.io/projected/78fbb610-0376-4e2d-8668-2e186f70bccd-kube-api-access-4877c\") pod \"perf-node-gather-daemonset-n6m7l\" (UID: \"78fbb610-0376-4e2d-8668-2e186f70bccd\") " pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:14.762018 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:14.761978    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:15.056910 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:15.056862    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"]
Sep 05 07:02:15.058444 ip-10-0-130-170 kubenswrapper[2559]: W0905 07:02:15.058410    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod78fbb610_0376_4e2d_8668_2e186f70bccd.slice/crio-4757980e0f1f5eb84d5978fc6bbe0d0b2bd7fc370f2ecba0104aa613f594b47a WatchSource:0}: Error finding container 4757980e0f1f5eb84d5978fc6bbe0d0b2bd7fc370f2ecba0104aa613f594b47a: Status 404 returned error can't find the container with id 4757980e0f1f5eb84d5978fc6bbe0d0b2bd7fc370f2ecba0104aa613f594b47a
Sep 05 07:02:15.302497 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:15.302450    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l" event={"ID":"78fbb610-0376-4e2d-8668-2e186f70bccd","Type":"ContainerStarted","Data":"95b464917c1fe68dc4a1dc18d5a8e5d8ec75f5466259a548e2fc8576bddb63e1"}
Sep 05 07:02:15.302663 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:15.302526    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l" event={"ID":"78fbb610-0376-4e2d-8668-2e186f70bccd","Type":"ContainerStarted","Data":"4757980e0f1f5eb84d5978fc6bbe0d0b2bd7fc370f2ecba0104aa613f594b47a"}
Sep 05 07:02:15.302663 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:15.302561    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:15.320180 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:15.320073    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l" podStartSLOduration=1.320052142 podStartE2EDuration="1.320052142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-05 07:02:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-05 07:02:15.318074216 +0000 UTC m=+2151.908633169" watchObservedRunningTime="2026-09-05 07:02:15.320052142 +0000 UTC m=+2151.910611095"
Sep 05 07:02:15.385656 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:15.385565    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5cdcd47765-kxwxl_6c86a979-0d7e-4cc2-a9e9-914f7c906c8b/console/0.log"
Sep 05 07:02:16.087815 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:16.087725    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-2kg8p_81a69703-95b0-49e9-97e7-62fd2199968e/volume-data-source-validator/0.log"
Sep 05 07:02:17.200345 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:17.200288    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-s2wh5_3df3774c-273c-4b5b-9c63-785f0f683d72/dns/0.log"
Sep 05 07:02:17.222484 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:17.222462    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-s2wh5_3df3774c-273c-4b5b-9c63-785f0f683d72/kube-rbac-proxy/0.log"
Sep 05 07:02:17.349431 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:17.349403    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zs2d7_db932df4-aa61-4dd9-b590-ba22dabce334/dns-node-resolver/0.log"
Sep 05 07:02:17.943269 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:17.943240    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j8x2c_561b2ddc-4c98-434e-995b-a8338ac6cfd5/node-ca/0.log"
Sep 05 07:02:19.213073 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:19.213040    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bd96764bd-6zzt8_d783a45f-8564-44f1-ace1-906ef791ee4c/kube-auth-proxy/0.log"
Sep 05 07:02:19.371380 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:19.371351    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74c88b6858-trsfj_f9b3185c-0e0e-43aa-8e47-1090bfb5f016/router/0.log"
Sep 05 07:02:20.231832 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:20.231802    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-74htm_543b8b05-067c-49ff-81da-00f05d7e4454/serve-healthcheck-canary/0.log"
Sep 05 07:02:20.768206 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:20.768171    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-gjwjg_f8057dec-2f84-4037-9fc9-b7e0d68c5cc2/insights-operator/0.log"
Sep 05 07:02:20.769134 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:20.769110    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-gjwjg_f8057dec-2f84-4037-9fc9-b7e0d68c5cc2/insights-operator/1.log"
Sep 05 07:02:20.860908 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:20.860867    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-h6j9h_04da8f82-4950-40f1-b128-5c1fe193f655/kube-rbac-proxy/0.log"
Sep 05 07:02:20.882518 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:20.882493    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-h6j9h_04da8f82-4950-40f1-b128-5c1fe193f655/exporter/0.log"
Sep 05 07:02:20.904222 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:20.904193    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-h6j9h_04da8f82-4950-40f1-b128-5c1fe193f655/extractor/0.log"
Sep 05 07:02:21.322109 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:21.321741    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tj5rw/perf-node-gather-daemonset-n6m7l"
Sep 05 07:02:23.176979 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:23.176950    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-59mrk_9c5aeb09-5dd2-49fd-9114-6791fbf0d742/manager/0.log"
Sep 05 07:02:23.435216 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:23.435187    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5rvzx_7dacb06e-b358-46b8-8508-f27c61f168eb/manager/0.log"
Sep 05 07:02:23.506795 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:23.506767    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gzgqb_99fcb1d8-584f-4b9a-beab-d0c1636b398a/server/0.log"
Sep 05 07:02:23.800851 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:23.800813    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555dc54ddd-zgvqq_4b7d6079-e03b-4bd4-99a9-982adf37a1da/manager/0.log"
Sep 05 07:02:23.829339 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:23.829310    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-555dc54ddd-zgvqq_4b7d6079-e03b-4bd4-99a9-982adf37a1da/copy-manifests/0.log"
Sep 05 07:02:25.249844 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:25.249815    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bd96764bd-6zzt8_d783a45f-8564-44f1-ace1-906ef791ee4c/kube-auth-proxy/0.log"
Sep 05 07:02:25.373232 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:25.373147    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74c88b6858-trsfj_f9b3185c-0e0e-43aa-8e47-1090bfb5f016/router/0.log"
Sep 05 07:02:27.118504 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:27.118460    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bd96764bd-6zzt8_d783a45f-8564-44f1-ace1-906ef791ee4c/kube-auth-proxy/0.log"
Sep 05 07:02:27.239061 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:27.238999    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-74c88b6858-trsfj_f9b3185c-0e0e-43aa-8e47-1090bfb5f016/router/0.log"
Sep 05 07:02:30.281632 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:30.281602    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-z8lqv_2c624fb9-258f-4f9d-9dfc-00fa2e6ee87f/migrator/0.log"
Sep 05 07:02:30.306402 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:30.306376    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-z8lqv_2c624fb9-258f-4f9d-9dfc-00fa2e6ee87f/graceful-termination/0.log"
Sep 05 07:02:31.737036 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:31.737006    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4q5t_0a357932-d881-47a6-b6d9-ec8dab12d087/kube-multus-additional-cni-plugins/0.log"
Sep 05 07:02:31.765905 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:31.765862    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4q5t_0a357932-d881-47a6-b6d9-ec8dab12d087/egress-router-binary-copy/0.log"
Sep 05 07:02:31.788475 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:31.788453    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4q5t_0a357932-d881-47a6-b6d9-ec8dab12d087/cni-plugins/0.log"
Sep 05 07:02:31.811371 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:31.811333    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4q5t_0a357932-d881-47a6-b6d9-ec8dab12d087/bond-cni-plugin/0.log"
Sep 05 07:02:31.833011 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:31.832991    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4q5t_0a357932-d881-47a6-b6d9-ec8dab12d087/routeoverride-cni/0.log"
Sep 05 07:02:31.858349 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:31.858319    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4q5t_0a357932-d881-47a6-b6d9-ec8dab12d087/whereabouts-cni-bincopy/0.log"
Sep 05 07:02:31.882648 ip-10-0-130-170 kubenswrapper[2559]: I0905 07:02:31.882622    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j4q5t_0a357932-d881-47a6-b6d9-ec8dab12d087/whereabouts-cni/0.log"